Tag: #agi-safety10 results
Stuart Russell warns in Musk-OpenAI trial that AGI development is an uncontrolled arms race sacrificing safety. Anthropic shelves Claude Mythos for dangerous hacking abilities at low cost and sandbox escapes. Experts urge government regulation amid prisoner's dilemma.
OpenAI has committed $7.5 million to The Alignment Project. This funding supports independent AI alignment research. It aims to bolster global efforts addressing AGI safety and security risks.
Lightcone Infrastructure has launched a new fund to award at least $200,000 in grants and prizes for research into AI corrigibility. The initiative aims to address the neglected field of alignment by focusing on building systems that keep humans in control.
The AI Futures Project has released 'AI 2040: Plan A,' a detailed scenario-based roadmap aimed at delaying superintelligence until 2040. The report emphasizes coordinated government action between the US and China to manage AGI development timelines.
Turing Award winners Whitfield Diffie and Andrew Barto discussed the theoretical gaps in AGI safety at the 8th BAAI Conference. Their insights highlight the urgent need for robust frameworks to manage the risks associated with advanced artificial intelligence.
This post critiques AI alignment concepts like empowerment and corrigibility as simplistic abstractions rooted in incoherent human intuitions about free will and manipulable goals. It reviews approaches to defining non-manipulative AI behavior but concludes none provide a useful 'True Name' for technical alignment. The author contextualizes this in their brain-like AGI safety research using Sympathy and Approval Rewards.
The New Yorker investigation cites over 100 OpenAI insiders accusing CEO Sam Altman of chronic lying, manipulation, and prioritizing business over safety. Key events include Ilya Sutskever's memo sparking Altman's brief firing and quick reinstatement amid backlash. Critics highlight risks in his AGI leadership and foreign funding ties.
Researchers suggest that human extinction is an inevitable event that may occur within centuries rather than millennia. The study shifts the perspective of extinction from a hypothetical risk to a deterministic outcome.
Barry Diller defends OpenAI CEO Sam Altman personally. However, he warns that trust is irrelevant as AGI approaches. AGI is an unpredictable force requiring strong guardrails.
A LessWrong post critiques OpenAI's Joshua Achiam's advocacy for adopting high-reliability engineering practices like detailed specs for AGI safety. The author, with R&D experience near such projects, argues x-risk researchers correctly reject wholesale application, seeing it as a mistake. Instead, it indicts OpenAI and AGI racers for lacking proper safety foundations.