TIGARS (Towards Identifying and closing Gaps in Assurance of autonomous Road vehicleS) was an international research project aimed at investigating the assurance gaps and challenges for first generation autonomous systems and research into techniques and engineering processes for addressing them.
The project was joint funded by the Assuring Autonomy International Programme (AAIP) and UK Centre for the Protection of National Infrastructure (CPNI).
The project was led by Adelard in collaboration with the following institutes:
We argue that assuring trust and trustworthiness through argument-based mechanisms, specifically, the Claims, Arguments, and Evidence (CAE) framework, allows for the accelerated exploration of novel mechanisms that could lead to advancements in the assurance of disruptive technologies.
This assurance approach is informed by an understanding of engineering processes and technical analysis for developing and assuring autonomous vehicles addressing resilience, formal verification, static analysis, security, and other aspects. Our project
- Identified current autonomous systems engineering approaches and their assurance gaps.
- Investigated how to address the assurance gaps with new analysis approaches based on verification of machine learning in both benign and adversarial environments, using simulation and test strategies, and an evaluation of defence in depth.
- Provided recommendations to regulatory and policy organisations and standards developers on a principles-based framework to address autonomy, as well as on a near-term interpretation of existing standards.
The project also conducted experimental trials with real life demonstrator systems, such as the TEV (TIGARS Experimental Vehicle) and Donkey car.
Outcomes and recommendations
The project concluded the research studies by reporting technical results, guidance and recommendations as a series of TIGARS Topic Notes (TTNs) - short briefing papers (7-12 pages) – on key areas.
The TTNs address the challenges faced in the current landscape regarding the noted attributes for Machine Learning (ML) based autonomous vehicles and systems. Additionally, we discuss potential solutions and recommendations proposed by a varied set of literature as well as preliminary research that we have carried out.
In addition, we also provided an overall summary of our research and longer supporting papers on experimentation, standardisation and regulatory guidance, sociotechnical perspective and SysML stochastic modelling.
For further information, please expand the section headings below.
- Part 1 addresses: Assurance-overview and issues, Resilience and Safety Requirements, Open Systems Perspective and Formal Verification and Static Analysis of ML Systems. https://arxiv.org/abs/2003.00789 (TTN part 1)
- Part 2 addresses: Simulation and Dynamic Testing, Defence in Depth and Diversity, Security-Informed Safety Analysis, Standards and Guidelines. https://arxiv.org/abs/2003.00790 (TTN part 2)
The autonomous systems field is international and has a wide variety of players of differing maturity. Some entrants are unfamiliar with classical safety engineering, yet have expertise related to AI and ML-based systems. Others are mature and familiar with classical assurance approaches but lack a grasp on the challenges autonomy brings about. Given this wide range of maturity and backgrounds, the TIGARS outputs aim to address a range of different audiences. We hope they will be accessible and of interest to both engineers as well as policy makers.
We will try to exploit the results of the project on the evaluation and development of RASs and on the development of assurance methodologies. The results will feed into projects on autonomy in the UK and a possible joint Japanese/UK project on assuring a real Mobility as a Service (MaaS) application.