AI Does Not Need Human Motives to Become Dangerous - Ajeya Cotra
Full episode: • Ajeya Cotra – "This might be the clearest ... Me on twitter: https://x.com/dwarkesh_sp
Member of Technical Staff, Metr
Verified interview transcripts, podcast appearances, and direct quotes from Ajeya Cotra, Member of Technical Staff at Metr. AI-verified speaker identity with full-text searchable transcripts.
Full episode: • Ajeya Cotra – "This might be the clearest ... Me on twitter: https://x.com/dwarkesh_sp
This week, we’re diving into two new reports about the OpenAI-Hugging Face hack, which fundamentally changed our understanding of what happened and upgraded our worry about A.I. Then we’re joined by Ajeya Cotra, one of the investigators at METR, to discuss the rogue agents’ message board and chain-of-thought transcripts and how the world should respond. Guest: Ajeya Cotra, co-author of the METR and Redwood Research report on the OpenAI-Hugging Face hack. 00:00 What was the Hugging Face attack? 02:22 Takeaways from the new reports 26:00 Ajeya Cotra joins us Additional Reading Brief Indepen…
Ajeya Cotra is a researcher at METR, where she works on threat modeling for loss-of-control risks from advanced AI. Before that, she led the technical AI safety program at what is now Coefficient Giving. She is one the three authors of METR and Redwood Research’s “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident”. We go through not only what she and her coauthors discovered during this investigation, but what it means for how we should train future, smarter AIs which might be involved in the process of recursive s…
00:04:15 Google I/O Reactions 00:28:20 Karpathy Joins Anthropic 00:32:35 Reactions to Spotify's New Icon 00:37:55 Birth Rate Debates 00:49:24 Jim Belosic, CEO and founder of SendCutSend, a company specializing in on-demand manufacturing services, discusses the recent achievement of securing a $110 million investment, valuing the company at $1 billion. He shares plans to utilize the funds to expand operations, including hiring additional staff and enhancing software capabilities, aiming to establish facilities near major metropolitan areas to expedite service delivery. Belosic also highlights t…
Every major AI company has the same safety plan: when AI gets crazy powerful and really dangerous, they’ll use the AI itself to figure out how to make AI safe and beneficial. It sounds circular, almost satirical. But is it actually a bad plan? Today’s guest, Ajeya Cotra, recently placed 3rd out of 413 participants forecasting AI developments and is among the most thoughtful and respected commentators on where the technology is going. She thinks there’s a meaningful chance we’ll see as much change in the next 23 years as humanity faced in the last 10,000, thanks to the arrival of artificial ge…
Ajeya is a Senior Research Analyst at Open Philanthropy. Ayeya has also written the first work on eliciting latent knowledge on language models. She’s currently thinking about the likelihood that powerful AI systems may be misaligned, and what technical work may help to reduce that risk. Previously, she worked on estimating when transformative AI might be developed and thinking about how worldview diversification could be implemented in budget allocation. She joined Open Philanthropy in July 2016 as a Research Analyst. Ajeya has a B.S. in Electrical Engineering and Computer Science from UC Ber…
Ajeya Cotra joins us to talk about thinking clearly in a rapidly changing world. Learn more about the work of Ajeya and her colleagues: https://www.openphilanthropy.org Timestamps: 00:00 Introduction 00:44 The default versus the accelerating picture of the future 04:25 The role of AI in accelerating change 06:48 Extrapolating economic growth 08:53 How do we know whether the pace of change is accelerating? 15:07 How can we cope with a rapidly changing world? 18:50 How could the future be utopian? 22:03 Is accelerating technological progress immoral? 25:43 Should we imagine concrete f…
Ajeya Cotra joins us to discuss forecasting transformative artificial intelligence. Follow the work of Ajeya and her colleagues: https://www.openphilanthropy.org Timestamps: 00:00 Introduction 00:56 What is transformative AI? 03:09 Historical growth rates 05:32 Simpler forecasting methods 09:56 Biological anchors 18:55 Different paths to transformative AI 20:06 Which year will we get transformative AI? 29:13 Expert opinion on transformative AI 34:40 Are today's machine learning techniques enough? 38:00 Will AI be limited by the physical world and regulation? 44:10 Will AI be limited by trai…
"Ajeya Cotra discusses topics including her work forecasting timelines for transformative artificial intelligence. Cotra is a Senior Research Analyst at Open Philanthropy. She is currently working on a framework for estimating when transformative AI may be developed, which is an input into how much capital should be allocated to the potential risks from advanced AI focus area relative to other areas. Previously, she worked on broad-brush estimates of the empirical returns to funding different areas, and on how worldview diversification could be implemented in budget allocation. She joined Ope…
Ajeya Cotra introduces the core principles of effective altruism. See a transcript of this talk: https://forum.effectivealtruism.org/s... See all of our favorite resources on effective altruism: https://www.effectivealtruism.org/res...
Sign in to search the full transcript archive, filter by topic, and access every quote from Ajeya Cotra.
The summary and quote tags on this profile are produced with AI assistance from verified, first-person interview transcripts, then checked by our team to confirm the speaker's identity and the accuracy of every quote. See how we verify →
Pro unlocks every transcript in full, plus unlimited search across every executive and company we cover.
50% off appliedPulling data at scale? The Data API gives you 30,000 transcripts a month over REST, MCP and JSON.
Compare all plans