Imagine, you hold up an iPad or another smart device to an aircraft, and all the locations of all the previous repairs in a 3D, interactive display pop up. Also, picture that you could view all the former repair paperwork, review handbooks and manuals, and even call someone from a pair of smart glasses to see exactly what you were looking at to get a second opinion, approval, or inspection completed. Sounds like something out of the Jetsons cartoon, huh? Well, call me June Jetson because this technology is on the precipice of our reality and some MROs are already implementing similar technologies in their repair facilities now.
A Peek into the Past
In Canada, a Supercluster project led by Boeing’s Vancouver lab, called Augmented Reality for Maintenance and Inspection project, is focused on streamlining and improving the crucial aircraft inspection and maintenance processes. Made up of Boeing Vancouver, Unity Technologies and Simon Fraser University, the project team is researching the viability of creating an augmented reality (AR) capability for aircraft maintenance and inspection.
The initial focus involves repairs to airplane surfaces. Learnings from this project could lead to other applications that leverage AR for the inspection of large objects across the transportation industry and beyond.
Currently, technicians and maintenance professionals count on written records and photographs of an aircraft, produced by previous inspections. Mechanics and technicians regularly map damage and repairs manually, and information is verified carefully to certify location accuracy. The idea is to cultivate a dedicated augmented reality engine that uses existing and historical data to compare an aircraft’s present and past states.
A View at the Present
FL Technics has been exploring AR, VR (virtual reality) and XR (cross-reality) solutions since early 2017, focusing on four key areas of operations - virtual reality training, support of augmented reality, drone inspection and automated NDT and predictive maintenance.
“We believe virtual reality to be a breakthrough technology for MRO companies,” noted an FL Technics representative. “Virtual reality training solutions are developed the most from those key areas of operations. However, due to COVID-19 impact, the further development and practical services were restrained and slowed down.”
Augmented Reality and Virtual Reality technology has been around for quite some time but using it as a tool for aircraft inspection and maintenance tasks are just now coming into play it seems. For maintenance, FL Technics started to evaluate and test technology in 2018-2019 and during a year in the pandemic. “Some significant progress was made with drone inspection and automated NDT solutions,” the MRO explained. “We develop and aim to use XR - drone inspection and automated NDT - combining advanced 3D and photo scanners with precise drone operations.”
So, how does it all work? Well, professionals are still looking into ways to optimize the technology and make it more efficient for the maintenance industry. “Key challenges are to find and develop an efficient drone-scanner setup to match the required precision and the speed of the process,” FL Technics noted.
When developed, the drone inspection and automated NDT solutions would make aircraft inspection faster and more precise compared to current processes. Additionally, the comprehensive reports would be fully digitized, streamlining the overall maintenance process to make it more efficient. “This would result in better planning and turn-around-times for clients and partners,” the company explained.
In this current, developmental stage, the key drawback, or challenge, is obtaining the required precision of scanning to meet regulations and match manual NDT procedures. Apart from drone inspections and NDT, the future focus for FL Technics would be predictive maintenance planning.
Training in a Virtual World
The MRO has also taken a peek into virtual reality for training purposes, initially – technical training and engineering training – entry level/introduction programs. “It is not a primary way of training, since most technical training and certifications are regulated by national and global organizations, requiring a wide range of procedures and standards to be met, including approvals and mandatory physical training with instructors,” noted FL Technics.
It is being tested and developed by FL Technic’s teams in its IT and technical training departments, with support from its maintenance and quality teams. Due to the restrictions from regulators regarding certifications and approvals, VR training solutions are partially limited to entry-level and supportive training courses, introducing technicians and engineers with routine and maintenance tasks.
VR reduces the cost of training and limitations related to assets and locations and allows for more flexible programs. “By using such methods, we are able to introduce the trainees to complex, rare situations,” the MRO stated. “Because virtual reality is so immersive and compelling, learners absorb information faster and retain what they have learned longer.
Additionally, it attracts and engages younger professionals.”
Is there any functionality that is not currently available? FL Technics says nothing in particular, except maybe a general source of assets, such as official 3D models of tools and aircraft, that would be available for future scaling of the product and solutions. But this would be relevant to any company developing similar solutions rather than FL Technics in particular.
Making AR Possible
AR for maintenance is being developed and used by a large pool of companies, two of which are VSight and Epson.
VSight Remote is a complete collaboration and knowledge-sharing platform that allows remote field technicians and experts to connect with each other and collaborate on maintenance, repair and operations via live video and audio powered by AR and is designed entirely for remote assistance operations. Experts have the ability to highlight and draw on top of real-world objects in a shared video.
On remote session, it is possible to draw and write notes on the live image in augmented reality in real time. Scroll, zoom, draw on the same document (.pdf, .png, .jpeg, .obf, . fbx) in real-time. The remote user sees everything from expert’s knowledge. The entire screen of a software screen or any webpage on a PC or Mac, can be shared with smartphones, tablets and smart glasses on the remote session. On a remote session, it is possible to takesnapshots from the field devices ands imultaneously draw directions, shapes, lines, texts, etc. on that image.
In light of the pandemic, digital transformation has become a necessity in many industries, with companies expediting the integration of collaboration tools into their day-to-day operations to improve service, communication and efficiency. AR smart glasses like the Epson Moverio AR smart glasses, allow aviation maintenance crews to connect with remote experts to identify equipment that needs servicing, as well as any potential problems. By seeing what the maintenance crew sees in real-time, remote experts can provide the appropriate audio or text instructions, PDF service manuals or tutorial video links to help expedite repairs. Additionally, remote inspectors can see what the on-site team sees before signing off on equipment installations, quality control audits, approving repairs completed, or suggesting corrective action. This eliminates guesswork, reduces the risk of breakdowns and allows for fast and efficient servicing.
