In AI we don’t trust… We test
Erwan PRUD’HOMME
Statistician and Entrepreneur, co-founder of SixFoisSept
Data Science and AI
Data science converts data into actionable information, using algorithms to analyse trends, assess risks (of failure), and offer recommendations (parts needing replacing). Artificial intelligence, meanwhile, employs computational and mathematical techniques to learn from this data, in order to mimic certain human capabilities, such as recognizing, predicting, recommending, or generating content.
AI and safety in civil aviatio
In 2017, my company signed a framework agreement with OSAC to use data science for calculating aircraft non-compliance risk. AI was not yet generative, but we had all kinds of algorithms at our disposal. The challenge was ensuring result guarantees for OSAC’s risk-based oversight (RBO). In a heavily regulated aviation environment, we needed a scalable scoring system for civil aircraft and their ecosystem, using robust, reliable algorithms. Neural networks were ruled out, focusing on two main areas instead:
In a heavily regulated aviation environment, we needed a scalable scoring system for civil aircraft and their ecosystem, using robust, reliable algorithms. Neural networks were ruled out, focusing on two main areas instead:
- Quality and completeness of input data: the key challenge was defining what constituted an aircraft (the object of the risk) and the associated characteristics that influenced the risk (indicators of probability and severity), in consultation with stakeholders. These indicators were then extracted reliably from civil aviation systems, with data quality being paramount. Daily data extractions now total around a hundred, feeding a robust risk estimation system.
- Robust and explainable algorithm: Emerging algorithms, driven by global research, necessitate a fair and reliable risk scoring system. Decision trees and budget allocation methods1 have enabled a scoring system based on OSAC inspectors’ expertise, mitigating individual assessment discrepancies.
Since 2019, OSAC has employed data science and AI to enhance inspections. By 2025, the algorithm aims to detect 45% more non-conformities than those identified by expert inspections alone. This technology standardises practices, maintains risk assessment consistency, and protects against staff turnover and expertise loss.
1. The “Budget Allocation Process” is a weighting method that involves distributing a notional amount of capital among several indicators based on their relative importance, thereby enabling business-specific weighting.
Generative AI, skills – what does the future hold?
Whilst our algorithm has helped to harmonise our understanding of risk, the advent of generative AI is set to upend all our habits, instincts and ways of thinking. The Industrial Revolution automated industrial processes, and blue-collar workers were transformed into supervisors or maintenance staff, making way for robots to take on repetitive tasks… We are witnessing the same phenomenon, but this time it is affecting white-collar workers! AI assistants like ChatGPT or Claude can now generate code in seconds, eliminating the need for manual coding by interns or junior staff. It’s no longer a question of knowing where the ‘;’ is in the code syntax.
However, asking oneself about that famous ‘;’ meant one had to consider the way in which these languages functioned. And through trial and error, the trainee engineer learnt. The same was true of trainee solicitors, who had to scour the records to find the latest Court of Appeal judgements, or civil aviation inspectors, who had to read (and re-read!) the regulations and documentation, building up experience, and laying the foundations for expertise.
The challenge
The challenge ahead will be to successfully combine the use of generative AI with a high level of expertise. Generative AI is, by its very nature, prone to hallucinations: the more generative AI improves, the greater the need to be able to challenge its output; but likewise, the more generative AI is used, the more the models learn from their own outputs (including hallucinations).
When it comes to generative AI, I would mention Etienne Klein, who, during a conference on AI, concluded his speech with a comment to the tune of: “Always bear in mind that generative AI does not understand a single word of what it generates”.2
In this age of algorithms and the ever-increasing use of generative AI, we must ensure that we maintain a high level of expertise in our respective fields. This is all the more important as generative AI is gradually, and implicitly, eroding the culture of hard work! The key challenge will therefore be how to reinvent learning, both in initial and continuing education.
2. This is quoted from memory, so the wording may not be exact!
