About the use of an AI-supported inspection drone for base maintenance checks – a research project

Did you know that during recurring base maintenance checks mechanical experts thoroughly inspect the outer skin of the aircraft? That means they spend a lot of time examining the aircraft in a crouched or crawling position, secured by harnesses to stay safe at great heights. What if this process could be performed by inspection drones supported by Artificial Intelligence, that eliminates poor ergonomics, ensures job safety and digital data output?

Ideally, in such a way, the application specifically supports the competence of the mechanical experts and the drone and recording system pre-selects surface damage, so that the actual damage repair - the so-called defect rectification - can be directly prepared and performed.

That is what we were eager to find out in a research project, funded by the German Federal Ministry for Economic Affairs and Energy. Together with many partners we worked to explore the possibilities that the AI inspection drone can bring to our MRO business during base maintenance checks. We just demonstrated the use of the drone successfully at ITS congress here in Hamburg, today.

AAM and our research focus

Of course, as an aviation company, the Lufthansa Group stays up to date on Advanced Air Mobility (AAM). In the future, we may see air taxis flying to and from hubs and destinations. And the development of service offerings for MROs or IT services has already started, so that an infrastructure can be established - drones are not a thing of the future anymore!

But back to our research project: We will focus on how drones can be used primarily within a base maintenance operation as a tool to increase efficiency and further developing our vision into a holistic Tech-Ops provider with data-driven production.

Let’s get started - Setup of our research project

For us at Lufthansa Technik, we always strive to improve and digitalize production processes that lead to reduced turn-around-time (TAT) and cost savings through efficiency for our customers.

Together with the Helmut Schmidt University Hamburg, the Technical University Munich, the University of Applied Science Würzburg-Schweinfurt, Lufthansa Industry Solutions and zeroG we teamed up to explore possibilities to research innovative operation processes in aircraft inspection through the use of drones with inspection cameras and Artificial Intelligence for data analytics. Ultimately leading to the automation of inspection processes – e.g. reporting, evaluation and report generation.

Through the funding of the German Federal Ministry for Economic Affairs and Energy, we were given the opportunity to explore the following four fields of action that we identified:

1. Process optimization: Drone-based inspection process for visual verification of surface damage with focus on common damage types such as lightning strikes, dents and scratches.
2. Evaluation of existing solutions on the market: Check existing drone companies and start-ups on the market
3. Innovation: Application of Artificial Intelligence for classification and analysis of recorded damage images as well as contextual understanding in maintenance relevant documentation such as Aircraft Maintenance Manuals (AMM), Structural Repair Manual (SRM) or customer specific work packages
4. Strategy: Support for prototypical development in digital projects such as automated readout of work packages with regard to inspection tasks, Aircraft Maintenance Manuals structure for automated reading of all relevant information or automated readout of defect Job Cards regarding instructions for damage repair.

 We defined our work packages based on the core competencies of each partner and structured the project into three clusters: maintenance concepts, AI architecture, and operation & application scenarios. In this way, we were able to work our way up from concept phase to demonstration & testing to evaluation & utilization.

Contributing our expertise

As one of the world's leading MRO providers, we were able to contribute specialist knowledge of the General Visual Inspection (GVI) process as part of recurring functional inspections from our global maintenance network.

1. The objective was to verify whether visual inspection of aircraft exterior surfaces focusing on damage, defects or irregularities can be optimized by innovative solutions such as drones or robots at touching distance.
2. A prototypical development in digital projects as part of the digitalization strategy of the Aircraft Maintenance segment for automated reading of work packages with regard to inspection tasks, structure based on Aircraft Maintenance Manuals (AMM) for automated reading of all relevant information as well as automated reading of instructions for damage repair.

Fast forward: After three years of research - the results are in!

It was a long way but we were able to accomplish challenging tasks and gather important information during our three-year project together with our partners.

Our use case for the development of an autonomous flying drone with 1 m flight distance to the object and automatic collision avoidance (incl. obstacle detection), capable of capturing high-resolution damage images from a size of 5x5 mm (25 mm²) in full HD resolution was successfully fulfilled. Furthermore, the 3D imaging of hangar infrastructure, aircraft and ground support equipment enables the drone system its orientation during flight.

Artificial Intelligence (AI)

In the field of Artificial Intelligence, we accomplished the simulation of surface damages in a real-time environment and in HD resolution for generating the data material. Through a mobile device app, defects can be captured with prototypical classifications and analysis of damage patterns. In addition, we used Natural Language Processing (NLP) to analyze historical damage documentation and created an aviation-specific “MRO dictionary” for automated extraction of relevant information from the maintenance documents. This includes as well the automated reading of workpacks regarding inspection tasks and Aircraft Maintenance Manuals-based structure for automated extraction of all relevant information.

Technological concept development and prototypical integration into adjacent MRO operating processes

In order to include adjacent MRO operational processes in the technological concept and prototypical integration, the required data points for selected use cases (e.g. job card creation or defect management) were identified. This includes automated readout of workpacks related to inspection tasks and the structure based on Aircraft Maintenance Manuals for automated readout of all relevant information (unified master data sets regarding planning attributes for routine and non-routine tasks).

What’s next?

The AI inspection drone research project proves that the use of an AI inspection drone in base maintenance checks is a concept that supports our mechanical experts in detecting and classifying surface damage on aircraft. This can lead to increased efficiency, as turnaround time can be significantly reduced by allowing the experts’ attention to focus on the damage detected during the drone flight.

However, implementation cannot be accomplished overnight, as infrastructural adjustments are also necessary for integration into routine processes. And, of course, all legal requirements must be taken into account. So the next step will be to identify further use cases based on the findings from this research project. It will be interesting to follow how further data, including historical data, can be integrated into the processes. One thing is certain, however: Drones are no longer a thing of the future and will also bring benefits to the MRO industry!

Three years of research project with important insights, great colleagues and partners

During the three-year project period, we gained insights into scientific high-tech projects and potential process improvements through the application of innovative solutions and market trends. Our multiple hangar visits with the project consortium during hardware and software testing and meetings in Munich and Hamburg (before Corona in person) served as an interesting platform for technology and knowledge exchange between industry and science.

The intensive internal exchange with our mechanical and structural expert colleagues was very valuable to me to learn more about the GVI process, regulatory requirements, process potentials and a possible integration of drone solutions – but most importantly: It was a great opportunity to expand my personal knowledge in research project work and as well as in my professional network. This has given me a lot of pleasure and I’m grateful that we were able to overcome even new challenges caused by the Corona virus! Great teamwork!