Apostol Vassilev

Research Team Supervisor, Security Components Group, Computer Security Division, Information Technology Laboratory, NIST

Title: Maintaining Trust in Agentic AI Systems, When the Guardrails Stop Working

Abstract: TBA

Bio: Apostol Vassilev (ZP-V) is a leading expert in Trustworthy and Responsible AI and Cybersecurity at the National Institute of Standards and Technology (NIST) and the National Cybersecurity Center of Excellence (NCCoE). His work is characterized by a rare fusion of deep theoretical research and practical advances, driving the development of national and international standards that secure the next generation of AI technologies. He is the Principal Investigator of the project on Robustness of Physical AI for self-driving cars at NCCoE. For this work Apostol was named a Top 20 voice in Automotive Cybersecurity for 2026 by Automotive IQ.

Recently, Apostol made a significant contribution to the fundamental understanding of AI safety by extending Gödel's incompleteness theorem to the domain of artificial intelligence. He successfully proved that no finite set of guardrails is universally robust against adaptive adversarial prompts. This landmark result offers a formal mathematical boundary for AI Security and Alignment, suggesting that safety in current and future AI systems cannot be a static achievement but must be a dynamic, evolving process.

Beyond his theoretical breakthroughs, Apostol is a practical force in the AI security community. He serves on the Distinguished Expert Review Board of the OWASP GenAI Security Project and is a founding member of the OWASP AI Vulnerability Scoring System project. His research also focuses on Adversarial Machine Learning (AML) and Robust Physical AI for autonomous vehicles.

With a Ph.D. in Mathematics from Texas A&M University, Apostol has authored over 70 scientific papers and holds five U.S. patents. His leadership and dedication to public service have earned him numerous accolades, including a medal from the U.S. Department of Commerce. A respected authority and frequent conference speaker, his insights are regularly featured in prominent publications such as the Wall Street Journal, Politico, Fortune and Forbes.

Vladimir Pavlovic

Professor of Computer Science, Rutgers University
Former Program Director, CISE/IIS, National Science Foundation

Title: Seeing Is Not Remembering: Toward Grounded, Trustworthy Multimodal Agents

Abstract: Multimodal agents are moving from answering questions to acting on hours and days of experience. As they do, trust comes down to one question: does an agent’s behavior stay tied to what it actually saw? In this talk, I argue that trustworthy agents start with grounding, and that grounding breaks in three places: what agents perceive, what they remember, and how they reason. Drawing on our recent work, I will show that each failure can be measured, explained, and reduced. Vision-language models describe objects that are not there, and signals inside the models can expose and suppress these hallucinations without retraining. Agents also forget what they correctly saw. Rather than designing memory by hand, we can let agents search for it themselves, as long as independent reviewers keep them honest. Our LVMH harness finds memory policies that are both more accurate and far more efficient. Even a task as simple as counting reveals how fragile visual reasoning is. Interpretability lets us find where it fails inside the model and repair it with light-touch interventions. I will close with a broader view of what our community must build next for agents that are safe by design. That means evaluations that reflect long-horizon use, assurance that does not rely on self-certification, and oversight designed in from the start.

Bio: Vladimir Pavlovic is a Professor of Computer Science at Rutgers University, where for more than twenty years he has led the SEQAM Lab. His work in probabilistic machine learning covers structured and multimodal data and has increasingly focused on the reliability of foundation models and agents built on that data. His group has developed training-free methods for detecting hallucinations in large vision-language models. It has also used interpretability to identify the internal “circuits” that support visual reasoning and to make models more robust out-of-distribution. Recent work includes MemEye, a benchmark demonstrating that current multimodal agents fail to retain the visual evidence needed for long-horizon reasoning, and ECTraj, a method for fast multi-agent trajectory prediction.

From 2022 to 2026 he served as a Program Director in NSF’s Division of Information and Intelligent Systems. There, he set a national funding strategy for AI and machine learning and managed a portfolio of more than 150 awards. He also developed or co-developed programs totaling over $200M. These include the $20M Safe Learning-Enabled Systems program, created with Open Philanthropy and Good Ventures to fund research on AI systems whose safety can be assured with high confidence. They also include the $8.5M Mathematical Foundations of AI program and the $6M AI for Mathematical Reasoning program, which grew out of a National Academies workshop he helped convene. He also led the US-Singapore Workshop on Responsible AI, which brought together government, academic, and industry partners from Singapore, Thailand, Malaysia, and Vietnam.

Earlier, he directed the Samsung AI Center Cambridge and was a Research Scientist at Google Research. He served as Program Chair of CVPR 2026, where he launched the first large-scale compute-transparency initiative at a major AI conference. He has authored nearly 300 peer-reviewed publications, holds five U.S. patents, and received his Ph.D. from the University of Illinois at Urbana-Champaign.

Douglas Summers-Stay (shared with ATRACC '26)

Computer Science Researcher, Army Research Laboratory (ARL)

Title: AI Safety as an Institution: Lessons from Army Decision Systems

Abstract: Like future powerful AI systems, the U.S. Army is a powerful entity that pursues its goals with potentially catastrophic consequences. Over the centuries, the Army has built up methods for keeping the pursuit of those goals in check, through other actors with other goals and other powers that interact with the chain of command in various ways to preserve the interests of various parties. We discuss some experiments with dual models which work together to guide a text generation, and experiments with meta-reasoning for consideration of a wider set of goals.

Bio: Douglas Summers-Stay has a PhD from the University of Maryland Department of Computer Science. He has worked for the Department of Defense as a research scientist for 20 years. He is the author of Machinamenta: The thousand year quest to build a creative machine.

Reza Ghanadan (shared with ATRACC '26)

Professor and Executive Director of Innovations in AI, University of Maryland

Title: Engineering Trustworthy AI Agents: From Evaluation to Runtime Assurance

Abstract: AI agents are moving from generating responses to taking actions, using tools, and coordinating with other agents in consequential environments. This transition raises a fundamental engineering question: how do we build reliable, robust, and steerable AI systems, evaluate their behavior and limitations, and establish evidence that justifies their deployment and continued operation as conditions change?

This talk presents a systems perspective on engineering trustworthy AI agents, connecting risk-informed evaluation with runtime assurance. It examines what evaluations must establish beyond benchmark performance: capabilities and failure modes, latent capabilities that ordinary prompting may not reveal, and newly revealed or expanded capabilities as models, tools, and agent scaffolding evolve. Such capabilities may enable consequential behaviors beyond those previously tested. The talk addresses uncertainty and validity limits, including why failure to elicit a capability does not establish its absence. It also explores how AI models with strong mathematical reasoning capabilities can support mathematical formalization of agent requirements and behavioral properties, generate tests, help quantify risks, and check outcomes, with safeguards against shared blind spots.

The talk connects evaluation evidence to monitoring, explicit constraints, outcome verification, and targeted interventions for runtime control and recovery. These mechanisms aim to keep agents within defined operating limits while preserving useful capability. Multi-agent systems receive particular attention because delegation, shared state, and interacting behaviors complicate risk assessment, control, and accountability.

A running operational example illustrates how assurance evidence guides deployment decisions and responses to unexpected behavior. The talk concludes with an engineering and research agenda linking evaluation, validation, intervention, and learning from operational experience, while clarifying the limits of current methods and the roles of human oversight and organizational responsibility.

Bio: Dr. Reza Ghanadan is Professor and Executive Director of Innovations in AI at the A. James Clark School of Engineering and the Institute for Systems Research at the University of Maryland, where he leads initiatives in AI engineering, robust and trustworthy AI systems, agentic intelligence, and applied AI for science and engineering. He is an affiliated research professor with the Department of Computer Science at the University of Maryland, the Artificial Intelligence Interdisciplinary Institute at Maryland (AIM), and the Center for Machine Learning at the University of Maryland. His work focuses on advancing scalable, reliable, and deployable AI systems that bridge foundational research with real-world impact across industry, government, and mission-critical applications. He is an IEEE Fellow, recognized for leadership in robust AI technologies and applications.

Raj Dasgupta (shared with ATRACC '26)

Research Scientist, Distributed Intelligent Systems Section, Naval Research Laboratory

Title: Trustworthy Multi-Agent Autonomy in Challenged Contexts: Evidence from Maritime Capture-the-Flag

Abstract: Trustworthy multi-agent autonomy in adversarial games and other challenged contexts requires teams to demonstrate reliable behavior beyond win rates. This keynote presents the Maritime Capture-the-Flag (MCTF) challenge as a testbed for assessing AI in challenged contexts. MCTF is a multiplayer adversarial game in which teams of autonomous uncrewed surface vessels (USVs) learn and execute coordinated strategies to capture an opponent’s flag while defending their own. The challenge combines partial observability, dynamic adversaries, environmental uncertainty, heterogeneous teammates, communications constraints, and simulation-to-physical-system transfer. We assess operational trust in the MCTF challenge through stress testing across scenario variations, evaluation against unseen policies, field experiments, runtime monitoring, and characterization of residual risk. Such assessment enables us to distinguish mission effectiveness from robustness, resilience, predictability, human-oversight burden, and assurance coverage. Based on our lessons from the MCTF challenge, we argue that adversarial team games should extend beyond play-to-win to develop methods for determining when multi-agent autonomy is reliable, when it fails, and when humans should intervene.

Bio: Dr. Prithviraj (Raj) Dasgupta is the Head (Acting) of the Distributed Intelligent Systems Section at the U. S. Naval Research Laboratory, Washington, DC. His research interests are in the areas of distributed AI and game theory, focusing on multi-agent reinforcement learning, adversarial games and multi-robot task and motion planning. He has led several large projects in these areas funded by the Office of Naval Research, NASA and other federal agencies. He consistently serves on the program committees of premier AI and robotics conferences and is a senior member of IEEE. He received his Ph.D. in Computer Engineering in 2001 from the University of California, Santa Barbara.

John E. Derrick

Founder and Chief Executive Officer, Authentrics.ai

Title: Diff, Trace, and Patch: Independent Analysis, Monitoring, Control, and Correction of Fine-Tuned Open-Weight Models at the Checkpoint Level

Abstract:

Summary of Key Points

Premise: independent monitoring & control of model internals are a safety requirement.

• Input–output evaluation alone cannot reveal or remediate learned behavior; assurance requires inspection and control of internal weights.
• Tools independent of the training pipeline enable V&V, auditability, and governance of fine-tuned open-weight models.
• Local, checkpoint-based operation keeps weights and training data on-premise and supports data-removal obligations (e.g., GDPR, CCPA).

Mechanisms

• Analysis: differencing of model behavior across checkpoints; tracing through weights and activations to attribute outputs to training data.
• Correction: targeted checkpoint patching, without rollback, to remove corrupted epochs, poisoned batches, or protected data; reported correction-cost reductions exceed 90%.
• Accuracy improvement: global tuning without additional data or training cycles.

Key Takeaways

• Trustworthy agentic systems presuppose trustworthy models; assurance must extend to model internals.
• Existing training checkpoints are sufficient to analyze, monitor, and correct fine-tuned open-weight models.
• Weight-level remediation offers a lower-cost alternative to full retraining.
• Attribution and machine unlearning make regulatory compliance and auditing tractable.
• On-premise analysis reconciles data protection with oversight requirements in government and industry.

Bio: John E. Derrick founded Authentrics.ai (Knoxville, TN) in 2023 to address the opacity of neural networks. The company’s checkpoint-native platform provides weight-level attribution, monitoring, and correction for mission-critical AI without retraining, supporting verification and validation (V&V) for defense and commercial users. Mr. Derrick has 34 years of experience in information science and artificial intelligence, has advised or lead numerous commercial and government funded projects spanning multi-spectral, LLM, advanced manufacturing, and numeric use cases. Inventor on 20 patents, advised / mentored for numerous companies, and mentor/advisor for Innovation Crossroads, Yale EIR, and others.

Mayank Kejriwal

Research Associate Professor of Industrial and Systems Engineering, University of Southern California
Principal Scientist, USC Information Sciences Institute

Title: Building Trustworthy AI Agents for Science

Abstract: Scientific discovery depends on much more than experiments alone. Researchers must also keep up with a fast-moving literature, plan projects, analyze results, prepare manuscripts and proposals, and manage many time-consuming tasks that shape whether good science moves forward. In this talk, I will discuss how agentic AI can help researchers and universities work more efficiently across this broader research lifecycle. Rather than serving only as a chatbot, these systems can help with connected tasks such as literature synthesis, scientific writing, grant development, and research communication.

I will describe the emerging idea of an AI scientist and discuss how such systems are beginning to support workflows that can contribute to genuinely novel and reliable scientific findings. I will also briefly review some of the infrastructure behind this vision, including a scientific writing editor, tools for grant writing and research dissemination, and an OpenClaw-based scientific experimentation platform currently being used by researchers at 80+ institutions worldwide. Throughout the talk, I will emphasize practical opportunities for using agentic AI to enhance research productivity while preserving rigor, trust, and human judgment.

Bio: Mayank Kejriwal is a research associate professor and principal scientist at the University of Southern California, where he directs a research group on Artificial Intelligence and Complex Systems (AICS). His research has been funded by DARPA, NIH and corporate and foundation opportunities, and published in prestigious venues like Science, Nature Communications, PNAS Nexus, and AAAI. He is also CEO and co-founder of GRAIL, an autonomous science company that is building AI agents for amplifying university research.

Jim Lockett and Elena Charnetzki

The MITRE Corporation

Title: Toward System-Level Test and Evaluation of Agentic AI Systems: Assurance, Measurement, and Bounded Trust

Abstract: Agentic AI systems can plan, use tools, maintain state, and delegate tasks, but model performance alone cannot establish whether they will behave reliably in operation. We present a system-level test and evaluation (T&E) framework organized around four perspectives: defining the system under test; characterizing autonomy, authority, and workflow control; assessing emergent multi-agent risks; and measuring technical performance, human interaction, and behavior across time and context. The framework addresses whether agents stay within their authority, recover from failures, and leave enough evidence to reconstruct their actions. Simulation, observational data, benchmarking, red teaming, and post-deployment monitoring provide practical ways to build that evidence throughout the system lifecycle.

Bio: Jim Lockett has been supporting the Department of War’s Developmental Test, Evaluation, and Assessments (DTE&A) office to improve current test and evaluation (T&E) practices for artificial intelligence (AI). As Task Lead for Systems Engineering Processes to Test AI Right (SEPTAR), Jim has focused on developing systems engineering methods and best practices for evaluating AI-enabled systems. His recent work focuses on system-level testing, measurement, benchmarking, and assurance of agentic AI systems. Jim has over 30 years of systems engineering experience across MITRE and commercial industry, with extensive experience in technical leadership, digital engineering, and AI innovation.

Elena Charnetzki has been supporting the Department of War’s Developmental Test, Evaluation, and Assessments (DTE&A) office to improve current test and evaluation (T&E) practices for artificial intelligence (AI). As a strategic decision scientist in MITRE’s AI, Autonomy, and Data Division supporting Systems Engineering Processes to Test AI Right (SEPTAR), Elena’s focus is on developing decision frameworks for AI assurance. Her recent work at MITRE focuses on policy, governance, and evaluation for generative and agentic AI systems, following a 15 year career at the Department of War in strategic analysis, leading AI research and analytic modernization efforts and supporting senior defense policymakers.