This week, Andrew Charlton, the federal assistant minister for science and technology, issued a stark warning about artificial intelligence (AI).<\/p>\n
Speaking<\/a> at the AI Safety Forum<\/a> at the University of Sydney, he said powerful AI models \u201care already doing things their creators never intended: cheating, deceiving, going their own way\u201d.<\/p>\n In response, the Australian government has established the AI Safety Institute under its National AI Plan<\/a>. Charlton described it as \u201ca national testing capability\u201d and provided the most detailed public account yet of the work it will do. <\/p>\n But will it be enough to keep Australians safe from what could be one of the most powerful, rapidly advancing technologies the world has known?<\/p>\n The harms are already here<\/p>\n AI\u2019s risks and harms are already affecting Australians. <\/p>\n There are the problems we are already seeing today: nudify apps<\/a>, more sophisticated scams<\/a> and deepfakes, voice cloning<\/a>, chatbots that have isolated teens and encouraged harms<\/a>, and heightened cybersecurity risks that are alarming our security agencies<\/a>.<\/p>\n Beyond immediate and present harms, experts are sounding the alarm about what\u2019s coming. This includes rapidly expanding capabilities and agentic AI that we don\u2019t yet have controls for<\/a>.<\/p>\n In a report<\/a> published last week, the United Nations\u2019 Independent International Scientific Panel on AI also raised concerns about global concentration of power, resources, AI capability, inequality, and the technology\u2019s impact on how we think, reason and work. <\/p>\n A significant remit<\/p>\n The AI Safety Institute has three broad tasks, as Andrew Charlton explained<\/a> at this week\u2019s forum. First, analysing and testing new models. Second, supporting government regulators and agencies to respond to emerging AI capabilities, risks, harms and trends. Third, shaping safe AI development, deployment and international governance in Australia\u2019s interests. <\/p>\n It is already doing work on multi-agent risks with the Gradient Institute<\/a>). <\/p>\n You\u2019ve probably already heard about AI agents that can book your table at a restaurant or plan your travel. The basic idea of an AI agent is an AI system that can plan and take action with limited supervision (for longer and longer periods too<\/a>). When AI agents interact with each other without supervision, unpredictable risks can arise. <\/p>\n The AI Safety Institute is also working with the CSIRO on AI alignment \u2013 the science of trying to ensure that AI systems (and AI agents) act in ways that are aligned with their users\u2019 values and goals. This matters because at this stage, we cannot fully predict how AI will act. There are plenty of experiments in which AI has deceived users<\/a> or avoided human-imposed controls<\/a>, even detecting (and underperforming)<\/a> when it is being tested. <\/p>\n The head of the AI Safety Institute, philosopher and Royal Australian Air Force reservist Kate Conroy<\/a>, also spoke at this week\u2019s forum. She outlined plans to take on both immediate harms affecting Australians, with attention to those affecting the most marginalised and vulnerable, as well as frontier risks and harms.<\/p>\n So much to do<\/p>\n The creation of the institute is a critical step towards Australia taking the impacts of AI seriously. <\/p>\n But given the speed of AI development, and the range of issues that are emerging, it would be fair to ask whether its budget of A$29.4 million over four years<\/a> is enough. <\/p>\n By contrast, the United Kingdom allocated around A$460 million<\/a> in its 2025 spending review to the its AI Security Institute. Singapore\u2019s AI Safety Institute has an annual budget of around A$11 million a year<\/a>. Canada has allocated<\/a> around A$50 million over five years to its AI Safety Institute. <\/p>\n All these budgets look insignificant next to industry spending. OpenAI alone sunk US$19 billion<\/a> into research and development in 2025. <\/p>\n Proposals that the Australian institute\u2019s budget should be significantly higher were strongly supported at this week\u2019s AI Safety Forum. <\/p>\n The unequal impacts of AI and its huge potential to increase inequality, as highlighted by the UN\u2019s new report, further highlight the fact there is much more work to do. <\/p>\n Two promising next steps would be to legislate a digital duty of care<\/a>, to help address some AI-supercharged online harms, and to address digital exclusion<\/a> to ensure all Australians can benefit from AI.<\/p>\n The Tech Policy Design Institute\u2019s recent assessment<\/a> also highlights Australia\u2019s strengths but also key needs in areas such as securing computing power for research and public interest activities. This would ensure Australia can maintain its position in research, and that the public sector and civil society don\u2019t fall behind the rest of the world. <\/p>\n Broadening the conversation<\/p>\n The conversation about AI, the opportunities it presents for society, and the impacts it will have needs to be much broader. We must consider a wider set of actions if we\u2019re going to really address Australia\u2019s low levels of trust in AI<\/a> and keep people safe from the technology\u2019s many potential harms. <\/p>\n