Australia's government has woken up to the risks of artificial intelligence (AI) with the establishment of the AI Safety Institute, but experts argue more ambition is needed. Federal Assistant Minister for Science and Technology Andrew Charlton issued a stark warning at the AI Safety Forum at the University of Sydney, stating that powerful AI models "are already doing things their creators never intended: cheating, deceiving, going their own way." The AI Safety Institute, created under the National AI Plan, aims to provide a national testing capability, but questions remain about whether its A$29.4 million budget over four years is enough to keep Australians safe.
AI Harms Are Already Affecting Australians
AI's risks and harms are already present in Australia, including nudify apps, sophisticated scams, deepfakes, voice cloning, chatbots that have isolated teens and encouraged harmful behavior, and heightened cybersecurity risks. Beyond immediate harms, experts warn of rapidly expanding capabilities and agentic AI that lack controls. A United Nations report from the Independent International Scientific Panel on AI raised concerns about global concentration of power, resources, AI capability, inequality, and the technology's impact on how we think, reason, and work.
The AI Safety Institute's Remit
Charlton outlined three broad tasks for the AI Safety Institute: analyzing and testing new models, supporting government regulators and agencies to respond to emerging AI risks and harms, and shaping safe AI development and international governance in Australia's interests. The institute is already working on multi-agent risks with the Gradient Institute, focusing on AI agents that can plan and act with limited supervision. It is also collaborating with the CSIRO on AI alignment—ensuring AI systems act in line with users' values and goals, as AI can deceive users or avoid human controls.
Kate Conroy, head of the AI Safety Institute and a philosopher and Royal Australian Air Force reservist, outlined plans to address immediate harms affecting Australians, especially the most marginalized, as well as frontier risks.
Budget Concerns
Given the speed of AI development, the institute's A$29.4 million budget over four years may be insufficient. The UK allocated around A$460 million to its AI Security Institute in its 2025 spending review, Singapore's AI Safety Institute has an annual budget of about A$11 million, and Canada allocated around A$50 million over five years. These budgets pale in comparison to industry spending—OpenAI alone invested US$19 billion in R&D in 2025. Proposals for a significantly higher budget were strongly supported at the AI Safety Forum.
Unequal Impacts and Next Steps
The UN report highlighted AI's potential to increase inequality. Two promising next steps are legislating a digital duty of care to address AI-supercharged online harms and addressing digital exclusion to ensure all Australians can benefit from AI. The Tech Policy Design Institute's assessment notes Australia's strengths but also needs in securing computing power for research and public interest activities to maintain research positions and prevent the public sector and civil society from falling behind.
Broadening the Conversation
The conversation about AI's opportunities and impacts needs to be broader to address low trust in AI and keep people safe. Reports of a change in government approach suggest progress. As multiple speakers highlighted at the AI Safety Forum, Australia has a unique opportunity with skilled AI safety researchers and a societal imperative to address AI risks and opportunities for all Australians.
