Research on Trustworthy AI Showcased at Three Leading Conferences

The ability to trust an autonomous system is crucial in nearly every application, but how do we know if its trustworthy or not? From large language models reasoning through a hard problem to autonomous robots navigating the physical world, the systems tend to fail in the same unnerving way: silently and confidently.

Zhenjiang Mao, PhD student in the Trustworthy Engineered Autonomy (TEA) Lab advised by Dr. Ivan Ruchkin, investigates this problem at the intersection of natural language processing, cyber‑physical systems, and robotics. His research asks an important question: when can we trust a learned model, and can the model itself tell us when it can’t be trusted?

Along with contributions from undergraduate and master’s students including Liam (Cade) McGlothlin, Josh Kumar, Arion Stern, and Vedansh Maheshwari, Mao is working on answering these questions. Supported by funding from the National Science Foundation and Citi Bank, the research approaches this problem from two complementary directions:

1. Confidence that means something.

A model that says “92% sure” is only useful if that number is honest. For large language models, existing methods compress an entire chain of reasoning into a single confidence score, discarding the most informative part: how the model’s certainty rises and falls as it works. Mao’s approach treats confidence as a signal over time, analyzing its shape with tools borrowed from formal verification. The same principle applies to a robot’s onboard safety monitor, enabling the system to flag when conditions drift outside what the model can reliably handle, and adjust its stated confidence accordingly. This line of work leads directly into agentic AI, where systems take real actions on a user’s behalf and self-assessment becomes a safety mechanism rather than a convenience.

2. Predictive models that are physically meaningful. 

“World models” let an AI system imagine what will happen next from raw camera images. Standard world models keep this prediction in an opaque internal representation, making it impossible to verify physical plausibility. Instead, this research utilizes world models whose internal states correspond to real physical quantities such as position, velocity, and angle, and whose evolution follows known physical equations. Importantly, this is learned without exact ground-truth measurements, since real sensors give noisy ranges rather than precise values, so we train from that kind of weak, distribution-based supervision.

Conference Highlights

In 2026, Mao’s work has been accepted at three leading conferences, showing the research’s importance for high‑stakes AI applications.

“Confidence over Time: Confidence Calibration with Temporal Logic for Large Language Model Reasoning” by Zhenjiang Mao, Anirudhh Venkat, Artem Bisliouk, Akshat Kothiyal, Sindhura Kumbakonam Subramanian, Saithej Singhu, Ivan Ruchkin was presented at Findings of the Association for Computational Linguistics (ACL 2026) in San Diego, California, July 2-7, 2026. ACL is the top venue in natural language processing. This paper focuses on the approach of treating confidence as a signal over time, using Signal Temporal Logic, a formalism borrowed from formal verification, to describe its shape.

Mao explains, “a telling failure case is what we call “dip and recover,” where the model briefly falters mid-reasoning and then sounds confident again, so the average looks fine while the answer is wrong. Our method catches that, and cuts calibration error by roughly 83% relative to the standard baseline while running far faster than sampling-based alternatives.”

The second paper is “Physically Interpretable World Models via Weakly Supervised Representation Learning,” authored by Zhenjiang Mao, Mrinall Eashaan Umasudhan, Ivan Ruchkin, and presented at 2026 ACM/IEEE 17th International Conference on Cyber-Physical Systems (ICCPS 2026) in Saint-Malo, France, May 11-14, 2026. ICCPS is the flagship conference for cyber-physical systems, the intersection of computation, control, and physical processes and is held as part of CPS-IoT Week.

The paper introduces PIWM, which learns image-based predictive models whose latent variables map onto actual physical quantities. Mao and his team evaluated it on CartPole, Lunar Lander, and their physical DonkeyCar racing platform, and showed that it can recover the true physical parameters of a system, such as pole mass or engine power, purely from images plus weak supervision.

“One finding generated good discussion at the conference: the architecture matters more than we expected,” Mao said. “Separating visual perception from physical interpretation into two stages, combined with a discrete (quantized) latent space, consistently produced the most stable and physically faithful predictions.”

The third paper, “Anomaly-Informed Confidence Calibration for Vision-Based Safety Prediction,” authored by Zhenjiang Mao, Jiawen Wu, Gabriel Wagner, Zhongzheng Zhang, and Ivan Ruchkin, has been accepted to the EEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2026) in Pittsburgh, Pennsylvania, September 27 to October 1, 2026. Preprint: arXiv:2605.21109. IROS is one of the two largest robotics conferences worldwide.

“The paper addresses what we call the perception-dynamics gap. Existing anomaly detectors operate on images, so they catch a dark or blurry camera feed, but they are blind to failures where the images look perfectly normal while the vehicle’s behavior degrades, such as a miscalibrated steering actuator or processing lag,” Mao said.

This method combines perceptual and dynamical signals from a world model to adjust confidence on the fly. On a physical DonkeyCar under four real-world fault conditions never seen during training, this reduced calibration error by 37% over the best existing method. Several undergraduate students in the lab contributed to this project by running the car and collecting data.

Real-World Impact

This research has a variety of real-world applications:

• High-stakes use of LLMs: knowing when a model should abstain, when it should escalate to a human, and which of several candidate answers to trust. This matters anywhere the cost of a confident wrong answer is high, including medicine, law, finance, and education.

• Agentic AI security: as autonomous agents take real actions such as booking, purchasing, writing code, or operating other software, calibrated self-assessment becomes a core safety mechanism.

• Autonomous vehicles and drones: a runtime safety monitor that can honestly report “I am no longer reliable right now,” whether because of fog, a dirty lens, a degrading actuator, or an unfamiliar environment, and then slow down or hand control back before a failure occurs.

• Formal verification and safety certification: physically interpretable predictions can be checked against safety requirements, for example “will this vehicle leave its lane in the next two seconds?” That is a prerequisite for certifying learning-based systems in regulated domains.

• Predictive maintenance: the dynamics anomaly score detects hardware degradation that is invisible in the camera feed.

The Next Phase

This ongoing research has many more threads to follow in the future. The three papers each solve one piece of a larger problem, which Mao hopes to tackle with his dissertation. The broad scope is that a model trained and calibrated under one set of conditions may be deployed in an altered, unknown environment where obtaining ground‑truth answers would be slow, expensive, unsafe, or otherwise unattainable. Mao’s question is whether calibrated confidence can survive that. His solution framework is organized into three steps:

1. Diagnose which part of the system changed. Identify what has changed, whether the camera, the environment, the physics, or the controller itself, and distinguish changes even when they’re invisible in the input. For example, a dry road and an icy road produce identical camera images while implying very different chances of leaving the lane. Mao aims for a diagnosis that clarifies what mechanism has changed, and a system that says ‘I cannot tell these two apart’ rather than guessing.
2. Recover calibration without any new labels. Rebuild the evidence a calibrator needs through three potential methods: reweight the existing data, generate new data from a world model configured to match the diagnosed change, or transfer calibration through a representation designed to be stable under that change, choosing the method that balances cost and uncertainty.
3. Certify what can honestly be promised. Explicitly state, before any outcome from the new conditions has been observed, what level of reliability the system can actually guarantee, with uncertainty about the diagnosis making the prediction more cautious rather than quietly less trustworthy. The promise should be auditable as real outcomes are observed, and retractable if observed performance diverges.

Open challenges within this include scaling the framework to AI agents, establishing precise boundaries on when to give-up, extending validation to open-ended language generation, and ensuring computational efficiency for small onboard computers.

These studies demonstrate that by integrating continuous confidence assessment and physically-meaningful world models, AI systems can learn not only to perform better under a variety of circumstances, but also to know when they cannot be trusted, an essential step toward truly reliable autonomous technology.