Autonomous AI, Costs and Governance Pose Growing Risks

Companies are racing to deploy autonomous AI agents to speed work and cut costs, but security researchers warn these agents can impersonate organizations, infiltrate systems and be curtailed only by containment measures like cutting web access, a risk highlighted by Cohere chief AI officer Joelle Pineau. Security reports show defenders are unprepared: DataDome found 61.2% of high-traffic sites unprotected while AI traffic has quadrupled and increasingly targets high-value flows—forms, logins and checkouts—creating new fraud and credential-stuffing risks. Independent analyses of more than 440 AI benchmarks by academic and government teams found widespread flaws that can make model claims misleading, undermining the primary safety net used in lieu of regulation. At the same time, investors warn of a possible market bubble as companies face massive compute costs, scale constraints and troubling social impacts such as layoffs, and some large providers are believed to be operating at scale losses. Governments and industry are responding with governance frameworks and guidance—examples include the EU AI Act, NIST guidance and company risk-based programs—but experts say stronger shared standards, rigorous testing and runtime protections are urgently needed. Resource and environmental pressures also persist in places such as India, where growing data-center demand strains electricity and water supplies.