AI Systems That Think, Team, and Fight

As AI systems become increasingly capable, the Department of War faces a critical challenge: how do we develop, rigorously evaluate, and safely deploy multiagent AI frontier systems across domains ranging from multimodal knowledge discovery to cognitive warfare? This talk presents lessons learned from building compound AI architectures that orchestrate large language models, vision-language models, and specialized agents through retrieval-augmented generation and agentic AI workflows. I will demonstrate how these systems enable cross-disciplinary knowledge synthesis for biosecurity, cognitive warfare planning and execution, and operator-AI team optimization in wargaming and readiness applications. Finally, I will present our emerging capabilities in multi-domain wargaming, where cognitively inspired AI agents execute doctrine-based maneuvers across air, space, cyber, and information domains. Evaluating these systems requires moving beyond traditional AI benchmarks. I will present our multidimensional ecosystem combining quantitative measures, qualitative SME assessments scaled through simulated domain expert agents, and causal investigations using structure learning algorithms to understand “why” behaviors emerge and “how” interventions affect mission outcomes. For safety evaluation, we examine human-agent-environment interactions holistically, addressing alignment failures, emergent capabilities under distributional shift, and systemic risks from multi-agent coordination through counterfactual “what-if” analysis and continuous monitoring. The era of scientifically grounded, operationally validated human-AI team optimization has begun, and this talk charts the path forward for defense applications.