Australians for AI Safety

In July 2023, Australians for AI Safety wrote to the Minister for Industry Science and Technology, highlighting the importance of AI safety.¹ The letter was in the context of the Safe and Responsible AI (SRAI) Consultation and built on similar letters from global AI leaders.²

The letter made four specific requests of the Australian Government:

• Recognise that catastrophic consequences from increasingly advanced AI systems are possible
• Take steps to mitigate these extreme risks, alongside other important work
• Work internationally to shape the global approach to these risks, and
• Increase support for research into AI safety.

Australia’s endorsement of the Bletchley Declaration on AI Safety in November 2023 was a critical step towards Australia delivering on these requests.³ By signing the Bletchley Declaration, Australia acknowledged that highly capable general-purpose AI models have the potential for serious, even catastrophic, harm. Australia resolved to mitigate these risks through global cooperation and by supporting scientific research on the safety of the most capable AI models at the frontier of research and development.

Despite our endorsement of the Bletchley Declaration, Australia has yet to position itself to learn from and contribute to growing global efforts. To achieve the economic and social benefits that AI promises, we need to be active in global action to ensure the safety of AI systems that approach or surpass human-level capabilities.

We recommend:

• The urgent creation of an Australian AI Safety Institute. The National AI Centre’s mission is driving the responsible adoption of AI by Australian industry. A separate institution is now needed to ensure the safety and security⁴ of the small number of models at the frontier of AI research and development. Countries – including the US, UK, Canada, Japan and the Republic of Korea – are establishing AI Safety Institutes to address this small class of models. An Australian AI Safety Institute would be focused on evaluating these models – including assessing models for dangerous capabilities, considering risks that could emerge from deployment in complex systems, “red-teaming” the adequacy of safeguards, and providing overall advice on their implications. An Australian AI Safety Institute would also drive AI safety research and deliver the Government's commitment to collaborate internationally on scientific research on frontier AI safety.⁵

• New safeguards must keep pace with new AI capabilities and new risks. AI is progressing rapidly, and new capabilities are hard to predict. The Government’s interim response to SRAI consultation agreed that, "Governments must respond with agility when known risks change and new risks emerge".⁶ Unfortunately, this is yet to happen. For instance, the UK and US adopted new regulations for “mail-order DNA” in response to the possibility that next-generation AI models will be able to help terrorists make biological weapons.⁷ Six months after the US took action, Australia has yet to update its equivalent biosafety regulations. This was a test of our agility, and we have failed. Australia needs a streamlined approach to risk identification and mitigation so that new safeguards keep pace with new risks in this fast-paced environment.

• Ensure the regulation of high-risk AI systems includes those that could have catastrophic consequences. Government, through the Safe and Responsible AI Consultation, identified the pressing need to regulate high-risk AI systems.⁸ In addition to the examples given in the Interim Response, “high risk” should also include systems where the consequence of something going wrong could be catastrophic. This should include highly-capable agents that can interact with other systems; autonomous goal-directed systems; and frontier models with capabilities relating to cyber offence, biotechnology, or risks to democratic institutions.⁹

Too often, lessons are learned only after something goes wrong. With AI systems that might approach or surpass human-level capabilities, we cannot afford for that to be the case.

Yours faithfully,