AI risks are becoming a major global concern as researchers, policymakers and technology experts debate how to manage the rapid growth of artificial intelligence.
A new study from MIT FutureTech and the University of Queensland has warned that some AI-related threats could cause severe harm if governments, companies and civil society fail to act early. The study involved 272 AI specialists from 37 countries, including experts from industry, academia, government and civil society.
The researchers asked the experts to assess 24 categories of AI risks and identify which threats are most urgent, which sectors are most vulnerable and who should take responsibility for reducing harm.
Why AI risks are now a global priority
Artificial intelligence is no longer a future technology. It is already shaping finance, media, defense, cybersecurity, education, health care and business operations.
That growth has created major opportunities. AI can improve productivity, speed up research, automate routine work and support better decision-making.
However, experts now warn that the same technology could also create serious dangers. These include cyberattacks, misinformation, misuse by bad actors, autonomous weapons, power concentration and systems that behave in ways humans do not fully control.
The MIT-linked study found that experts consider several AI risks to be far above normal tolerability levels. Under a “business as usual” scenario, they judged that 18 of the 24 risks had more than a 10% chance of causing catastrophic outcomes. These outcomes were defined as more than one million deaths, more than $100 billion in financial losses or comparable large-scale harm.
Most serious AI risks identified by experts
The experts identified five areas as especially severe.
The first is AI developing dangerous capabilities. This includes systems becoming more autonomous, persuasive or capable of self-improvement in ways that could be difficult to control.
The second is AI-enabled weapons and cyberattacks. Advanced AI could make it easier to attack digital systems, automate hacking attempts or support dangerous weapons development.
The third is competitive pressure. Companies and governments may race to deploy powerful AI systems quickly, even when safety testing is incomplete.
The fourth is power centralization. If a small number of companies or governments control the most advanced AI systems, they could gain enormous economic, political and strategic influence.
The fifth is sophisticated false information. AI can generate convincing text, images, audio and video, making it easier to spread misinformation at scale.
AI risks in finance, information and national security
The study found that experts view information, finance and national security as the most vulnerable sectors across AI risks.
The information sector is exposed because AI can produce and spread false content quickly. Fake news, deepfakes, impersonation scams and automated propaganda could make it harder for the public to trust what they see online.
Finance is also vulnerable. AI could be used to manipulate markets, automate fraud, attack banking systems or exploit weaknesses in digital payment networks. As financial systems become more automated, failures or attacks could spread quickly.
National security faces some of the highest risks. AI could be used in cyberwarfare, surveillance, autonomous weapons and intelligence operations. These uses could raise the chance of conflict, escalation or unintended harm.
Why catastrophic AI risks worry researchers
The word “catastrophic” can sound extreme, but researchers use it to describe large-scale harm that affects many people or causes massive economic damage.
According to the study, experts were asked to assess risks under two scenarios. The first was a “business as usual” path, where current AI development continues without major new intervention. The second included “pragmatic mitigations,” meaning reasonable and cost-effective steps to reduce harm.
Even with practical safeguards, experts still judged five AI risks as having more than a 10% chance of causing catastrophic outcomes.
That finding suggests that basic safety measures may not be enough. Stronger coordination, better oversight and clearer accountability may be needed.
AI misuse is one of the biggest concerns
One of the clearest dangers is deliberate misuse.
Bad actors could use AI to write malicious code, scan for security weaknesses, create convincing scams or automate cyberattacks. The same tools that help programmers and businesses work faster can also help criminals scale harmful activity.
Experts also raised concerns about the possible use of AI in biological or weapons-related harm. While many AI systems are built for legal and productive purposes, powerful tools can be misused when placed in the wrong hands.
This makes governance difficult. Policymakers must support innovation while reducing access to dangerous uses.
Misinformation could damage public trust
AI-generated misinformation is another major risk.
Modern AI systems can create realistic articles, images, voices and videos. This makes it easier to produce false content that looks real. During elections, conflicts or public health emergencies, such content could confuse citizens and weaken trust in institutions.
Businesses can also be affected. Fake announcements, impersonated executives and AI-generated fraud could damage companies and investors.
The information space is already crowded. AI could make it even harder for people to separate reliable information from manipulation.
Power concentration raises economic and political concerns
Advanced AI development is expensive. It requires huge amounts of computing power, skilled researchers, data and infrastructure.
Because of this, only a small number of companies and governments may be able to build and control the most powerful systems. That could concentrate power in ways that affect markets, labor, security and public policy.
If a few players dominate AI, smaller businesses, developing economies and ordinary users may have less influence over how the technology evolves.
Experts warn that this could create unfair advantages and reduce public accountability.
Who should manage AI risks?
The study asked not only which risks are most serious, but also who should be responsible for addressing them.
AI governance cannot fall on one group alone. Governments need to create rules and safety standards. Companies need to test systems responsibly and avoid reckless deployment. Researchers need to improve risk assessment. Civil society needs to defend public interest, transparency and human rights.
International cooperation will also matter. AI systems cross borders, and their effects can be global. A weak safety approach in one country can create risks for others.
What stronger AI safeguards could include
Managing AI risks may require several practical steps.
Governments could require stronger testing before powerful AI systems are released. Companies could improve transparency, safety audits and incident reporting. Researchers could develop better methods for measuring dangerous capabilities.
There may also be a need for rules around AI use in weapons, cybersecurity, finance and political communication.
For high-risk systems, independent oversight could become more important. This would help ensure that companies do not rely only on internal safety checks.
Why AI risks should not stop innovation
The warning from experts does not mean AI should be abandoned. Artificial intelligence has real benefits in medicine, education, science, business, climate research and public services.
The issue is not whether AI should develop. The issue is whether it can develop safely.
Good governance can support innovation by building public trust. When people believe AI systems are safe, fair and accountable, adoption becomes easier.
The challenge is to move fast enough to benefit from AI while being careful enough to prevent serious harm.
The future of AI risks and global safety
AI risks are likely to remain a central topic as the technology becomes more powerful.
The MIT FutureTech and University of Queensland study adds to growing calls for stronger action. Its findings show that many specialists believe the world cannot rely on business as usual.
Information systems, financial markets and national security may face the greatest pressure. But the effects of AI could reach nearly every part of society.
The message from experts is clear: AI can bring major progress, but unmanaged AI risks could also create serious danger. The next few years may decide whether governments and companies can build the safeguards needed to keep the technology beneficial and under control.
