This talk examines deep learning robustness from a system reliability perspective, arguing that robustness should not be studied as an isolated AI model property but in relation to higher-level system claims such as safety assurance, reliability assessment, and security certification. It first reviews common robustness problem formulations, including binary robustness verification, maximum safe radius, maximum loss, and probabilistic robustness. Among these, probabilistic robustness is highlighted as especially relevant for safety-critical AI systems because it quantifies the likelihood of adversarial examples under stochastic perturbations, thereby aligning more naturally with reliability and risk reasoning in real-world operation.

The talk then distinguishes the roles of adversarial robustness and probabilistic robustness at system level. Adversarial robustness is more closely associated with security against maliciously optimized attacks, whereas probabilistic robustness better captures reliability and safety risks arising from benign operational noise, environmental uncertainty, and unsophisticated random perturbations. This distinction motivates a broader research agenda in which robustness metrics are selected according to the system-level property being claimed.

Building on this perspective, the talk presents the presenter team’s recent work on probabilistic robustness estimation, improvement and benchmarking. The talk advocates integrating probabilistic robustness into system-level safety and reliability modelling to support more meaningful assurance arguments for AI-enabled systems.

While most on-demand mission-critical systems are engineered to be reliable to support critical tasks, occasional failures may still occur during missions. To increase system survivability, a common practice is to abort the mission before an imminent failure. We consider optimal mission abort for a system whose deterioration follows a general three-state (normal, defective, failed) semi-Markov chain. The failure is assumed self-revealed, while the healthy and defective states have to be predicted from imperfect condition monitoring data. Due to the non-Markovian process dynamics, optimal mission abort for this partially observable system is an intractable stopping problem. For a tractable solution, we introduce a novel tool of Erlang mixtures to approximate non-exponential sojourn times in the semi-Markov chain. This allows us to approximate the original process by a surrogate continuous-time Markov chain whose optimal control policy can be solved through a partially observable Markov decision process (POMDP). We show that the POMDP optimal policies converge almost surely to the optimal abort decision rules when the Erlang rate parameter diverges. This implies that the expected cost by adopting the POMDP solution converges to the optimal expected cost. Next, we provide comprehensive structural results on the optimal policy of the surrogate POMDP. Based on the results, we develop a modified point-based value iteration algorithm to numerically solve the surrogate POMDP. We further consider mission abort in a multi-task setting where a system executes several tasks consecutively before a thorough inspection. Through a case study on an unmanned aerial vehicle, we demonstrate the capability of real-time implementation of our model, even when the condition-monitoring signals are generated with high frequency.

In the field of prognostics and health management, the integration of machine learning has enabled the development of advanced predictive models that ensure the reliable and safe operation of complex assets. However, challenges such as sparse, noisy, and incomplete data necessitate the integration of prior knowledge and inductive bias to improve model generalization, interpretability, and robustness.

Inductive bias, defined as the set of assumptions embedded in machine learning models, plays a crucial role in guiding these models to generalize effectively from limited training data to real-world scenarios. In PHM applications, where physical laws and domain-specific knowledge are fundamental, the use of inductive bias can significantly enhance a model’s ability to predict system behavior under diverse operating conditions. By embedding physical principles into learning algorithms, inductive bias reduces the reliance on large datasets, ensures that model predictions are physically consistent, and enhances both the generalizability and interpretability of the models.

This talk will explore various forms of inductive bias tailored for PHM systems, with a particular focus on heterogenous-temporal graph neural networks, as well as physics-informed and algorithm-informed graph neural networks. These approaches will be applied to virtual sensing, modelling multi-body dynamical systems and anomaly detection.

In recent years, deep learning-enabled systems have made remarkable progress, powering a surge in advanced intelligent applications. This growth and its real-world impact have been further amplified by the advent of large foundation models (e.g., LLM, Stable Diffusion). Yet, the rapid evolution of these AI systems often proceeds without comprehensive quality assurance and engineering support. This gap is evident in the integration of standards for quality, reliability, and safety assurance, as well as the need for mature toolchain support that provides systematic and explainable feedback of the development lifecycle. In this talk, I will present a high-level overview of our team’s ongoing initiatives to lay the groundwork for Trustworthy Assurance of AI Systems and its industrial applications, e.g., including (1) AI software testing and analysis, (2) our latest trustworthiness assurance efforts for AI-driven Cyber-physical systems with an emphasis on sim2real transition. (3) risk and safety assessment for large foundational models, including those akin to large language models, and vision transformers.

Optimal system reliability design is an important research field in reliability engineering. Since the 1950s, extensive studies have been conducted on various aspects of this issue. This field remains highly active today due to the need to develop new generations of complex engineering systems, such as 5G telecom networks and high-performance computing clusters, which are expected to be highly reliable to meet the stringent, dynamic, and often real-time quality demands of system operators and end-users. Over the past five years, numerous new researches on optimal system reliability design have been published, addressing the theoretical challenges posed by the new engineering systems. This presentation will systematically review these works with the focus on theoretical advancements, including the models and methods for redundancy allocation problem, redundancy allocation under mixed uncertainty, joint reliability-redundancy allocation problem and joint redundancy allocation and maintenance optimization. Through analysis and discussions, we will outline future research directions.