A new survey of national security practitioners identified a wide range of near- and medium-term AI risks for policymakers to consider.
Dive Brief:
- Speed and legal frameworks are the biggest impediments to confronting AI-driven threats, national security experts said ina newly released survey conducted by the Institute for Security and Technology(IST).
- Experts also worry that the U.S. government’s dependence on AI industry expertise leaves it vulnerable to regulatory capture, according to the survey, which included 111 respondents from multiple national security domains and was conducted between late April and mid-July.
- IST’s report, published on Wednesday, provides a high-quality snapshot of the fears and expectations driving AI policy conversations in cybersecurity and other areas.
Dive Insight:
One of the key findings in IST’s report is that the government lacks the ability to verify frontier AI labs’ claims about what their products can and will do.
“Respondents pointed to chronic underinvestment in test and evaluation, the absence of independent evaluation processes, and no reliable way to benchmark frontier capabilities,” the report said.
That underinvestment means that government leaders cannot verify how AI systems will behave before integrating them into military technology and other critical platforms. “Without its own benchmarks, the [government] is left to take capability claims on faith rather than measurement.”
Experts were also deeply skeptical that current AI systemscould be trustedto act independently. “Many see hallucination as intrinsic to AI models, making systems highly problematic to rely on without significant, detailed human checking,” IST observed. “As one put it, AI can make a good analysis better and a bad one worse.”
More than eight in 10 respondents said AI would “very likely” improve all of the cyber defense capabilities they were asked about — fromvulnerability discoveryto malware analysis toincident response— but 57% also said AI washelping attackers more than defendersin the near term.
The biggest cybersecurity risk associated with AI is its ability to find and exploit vulnerabilities, according to the survey, whose respondents ranked that risk first by a wide margin.
And while legal and regulatory frameworks ranked second on respondents’ list of the biggest hurdles to combating AI threats, respondents were split overthe right way to regulate AI. Roughly one-quarter of experts advocated for cross-sector federal requirements, 31% supported sector-specific requirements and 36% favored a hybrid approach.
Experts encouraged policymakers not to focus too much on AI’s unique capabilities when evaluating AI-related risks, saying thatbasic cybersecurity failureswere likely to be the biggest cause of issues such as “loss-of-control scenarios, adversarial manipulation, and the theft of model weights and weapons designs.”
“The bottleneck is deploying cyber defense at scale, not the technology itself,” IST said, adding that weak cybersecurity “will enable many of these other risks.”
Filed Under:Strategy,Threats,Leadership & Careers,Policy & Regulation
