OpenAI Launches New “Superalignment” Team to Protect against Rogue AI

OpenAI, the research organization behind GPT-4 and ChatGPT, has announced the new team dedicated to ensuring AI does not misbehave and harm humans. The team, called Superalignment, will focus on studying and developing methods to align AI with human values ​​and goals, and prevent AI from becoming misaligned or evil.

According to OpenAI, alignment is “a property of an AI system that causes it to pursue goals that are beneficial to humans, even if those goals are not explicitly specified by the designer or user of the system.”

The team will work on the theoretical and practical aspects of AI alignmentsuch as understanding the sources and risks of misalignment, designing incentives and feedback mechanisms for AI systems, and testing and evaluating alignment of existing and future AI models.

OpenAI “dedicated 20% of the compute we have secured to date over the next four years to solving superintelligence alignment problems. Our main basic research bet is our new Superalignment team, but getting it right is critical to achieving our mission and we expect many teams to contribute, from developing new methods to improving them to implementation.”

The team will also collaborate with researchers and other stakeholders in the AI ​​community, such as ethicists, policy makers, and social scientists, to foster a culture of responsible and trustworthy AI development.

One of the main challenges the team will face is the possibility of AI systems becoming more intelligent and capable than humans, and thereby developing goals and preferences that are incompatible or even hostile to human well-being.

OpenAI Seeks a Leading Role in AI Security Actions

This problem has been widely discussed and debated by AI experts and philosophers, who have proposed various solutions and safeguards to prevent or mitigate it. However, OpenAI believes that there is no single or definitive answer to the problem of alignment and that ongoing research and experimentation is needed to find the best way to ensure that AI continues to benefit humanity.

AI can be a blessing or a curse for humanity, depending on how we develop and utilize it. That’s the message of a paper published in Nature on May 30, 2023, by some of the world’s top AI experts. They warn that AI poses a serious threat to human survival, and that we need to take urgent steps to ensure its safety and alignment with human values ​​and goals.

This paper was co-authored by more than 350 leading figures in the AI ​​space, including the CEOs of Google DeepMind, OpenAI, and Anthropicthree of the most influential and cutting-edge AI research organizations.“Reducing the extinction risk from AI should be a global priority along with other risks of social scale, such as pandemics and nuclear war,” read open letter.