A few secretive AI companies could crush free society, researchers warn
Summary
Most of the research surrounding the risks to society of artificial intelligence tends to focus on malicious human actors using the technology for nefarious purposes, such as holding companies for ransom or nation-states conducting cyber-warfare. A new report from the security research firm Apollo Group suggests a different kind of risk may be lurking where few look: inside the companies developing the most advanced AI models, such as OpenAI and Google. It is a scenario they call an "intelligence explosion," where humans in an organization gain an advantage over the rest of society by virtue of the rising capabilities of AI. A company that is able to achieve more and more AI capabilities "in software," without the addition of vast quantities of hardware, might not raise much attention externally, they speculate. Yet another provision, they argue, is information sharing, specifically to "share critical information (internal system capabilities, evaluations, and safety measures) with select stakeholders, including cleared internal staff and relevant government agencies, through pre-internal deployment system cards and detailed safety documentation."