Can an AI Instructor Reduce Flight Training Time?

For many years, professional aviation trainers have disparaged Microsoft Flight Simulator (MFS) as non-serious: entertainment, not education.

“If you had done a web search, the results would have been that it’s terrible, that you will form so many bad habits that you will go kill yourself in an airplane,” said Brandon Seltz. “I didn’t believe it a bit.”

Seltz is CEO of TakeFlight Interactive, which has created a mix of virtual reality gaming and AI-augmented algorithms, layered onto MFS or other popular hobbyist simulation programs such as Xplane and Prepare3D, to produce a series of training modules which can reduce instructor time and in-aircraft training time.

In 1977, when Seltz was five years old, the Atari 2600 came out. “I remember walking into my uncle’s house and seeing Pong on the TV for the first time. It just captured my attention and became my life’s work.”

He became a lead designer on the Microsoft Flight Simulator development team “around the time I started taking flight lessons. I immediately discovered that I already knew how to fly. The first time I ever took off in a powered airplane and looked over the wing, I thought, oh my gosh, I have done this a million times, and it’s just not that much different.”

Seltz spent seven years with MFS before joining Redbird Flight Simulations as director of scenario development. In 2015, he formed TakeFlight in Renton, Washington (home to Microsoft and Boeing) with a handful of colleagues.

“We are changing simulation in a way that allows people to leverage it much, much better. It’s not just a sandbox anymore where you get in there and you might be doing it right, you might be doing it wrong, but you really don’t know… to a very focused training tool where you know exactly what you need to do, how to do it, what you’re doing wrong. And in the end, there’s an objective score that shows you what to work on next.”

“One of the things that we wanted to prove is that you can teach stick-and-rudder type skills in the simulator.”

EARLY ADOPTER – US NAVY

For civil aviation skeptics who might regard this as child’s play, consider that the US Navy has renewed a TakeFlight prime contract, worth $5.5 million, for their AIViator AI-based virtual instruction for Project Avenger – Navy Training Next on T-6B VR part-task training devices. A partner in the project is Advanced Simulation Technology Inc. (ASTi) to integrate their SERA (Simulated Environment for Realistic ATC) dynamic air traffic control.

“One of the reasons I’m so excited about that project is we are gathering so much data that I cannot wait to let machine learning loose and see what types of things we learn,” said Seltz.

“The things that we’re doing are on the more basic levels of AI. In game design we use reactive AI and scripted sequences and complex state machines to mimic human behaviors and non-player characters. Our goal is very similar; we strive to create what feels like a sentient human instructor in the aircraft with you.”

“There is no massive data set of what happens in a cockpit with an instructor and a student. When the instructor tells the student that they’re low, what kind of response did we get out of the student and how well did they respond? Did they over respond or under respond? And would they have responded better to a different suggestion from the instructor? That’s where it gets really interesting.”

“On the back end we’re collecting all that data. We know exactly where the student was, what the instructor said, how the student responded, how well they do over a series of 5 and 6 attempts at a scenario.”

“The subject matter experts are absolutely critical in the process. We work with our SMEs to make sure that we’re teaching the right thing. We make sure we’re hitting all of the common student errors.”

“One of the most important things in the work for the Navy is that the virtual instructor sounds like a Navy instructor and not a civilian. It’s a lot of work back and forth with the SMEs to make sure that we’re saying the right thing in a certain situation, but it is much more of a script that we write, a matrix if you will, versus teaching the AI how to be an instructor. They use the lingo, you know; they very much have their own language.”

“On the commercial side, it is all about safety. And I would never change that. But at the same time, it is a struggle to get new technologies recognized in that space.”

“Training our virtual instructor to teach based on data comes back to having those data sets. Data sets very hard to create from scratch. Those data sets just don’t exist in flight training yet. We are working hard to build them. We’re asking ourselves, what kind of data do we need to be collecting today to do all the cool things that are capable tomorrow?”

FLYING MANEUVERS with COMMENTARY

In each TakeFlight module, the virtual instructor first presents the lesson and potentially demonstrates the maneuvers. The student then flies the maneuvers with commentary and help from the instructor. Each effort then receives a score based on the FAA’s airman certification standards so students are able to track their proficiency and progress exactly as they would in a real aircraft.

“Our software is very much like a game experience,” Seltz said. One module, for example, has users guide their virtual plane through a series of glowing rings in a virtual sky. Another takes the users through the steps for a landing and scores them on how well they execute those steps.

The first stage teaches the fundamentals of flying in seven lessons and the second stage takes student pilots all the way to solo with six additional lessons that cover crosswind takeoffs and landings, traffic pattern procedures, steep turns, and abnormal/ emergency procedures.

“Our system is much more than just a computer telling the students when they are high, low, fast, or slow. We have developed complex algorithms that monitor all aspects of the flights, including whether or not the pilot is making corrections in a timely manner. The automated instructor responds appropriately, providing feedback just as a real flight instructor would,” Seltz said.

PROOF of EFFECTIVENESS

A dissertation study by Ryan Paul Gutheridge for the University of North Dakota demonstrated the efficacy of AI-based versus human flight instructor. “When considering the transfer effectiveness of an artificial intelligence-based simulator pre-training program… indicates that for each 10 hours of simulator pretraining, flight training time is reduced by 3.4 hours, on average.”

“The results of the study showed that students who conducted the pre-training lessons:

• Took fewer calendar days to complete the pre-solo block of Private Pilot training,
• Required fewer hours in the airplane during the pre-solo block of Private Pilot training, and,
• Required fewer ground training hours during the pre-solo block of Private Pilot training.

At Rainier Flight Service at Renton Municipal Airport (RNT) flight instructor Mark Thompson said, “Since introducing our system, we have helped Rainier reduce the average flight time to solo from 25 down to only 15 flight hours. Starting with a new student who has completed the first two stages on TakeFlight Interactive is like getting in the airplane with someone who already has three to four hours of quality flight instruction, but we didn’t have to spend our time doing it. That’s like adding 20 percent more instructors to your team.”

“The AI engine and courseware model provides a quantum leap over current training tools,” stated Captain Tim Adam, 787 Captain and former Director, Flight Training Center, United Airlines. “Their replacement for scarce and expensive instructor time, and an additional tool for the flight instructor, will surely have a strong positive impact on the economics and efficacy of training.”

“Every qualified pilot who could fly an airliner is getting sucked right into the industry,” Seltz noted. “So it’s leaving universities turning away students in a way they never have before because they don’t have an instructor for an airplane to meet the demand from the students.”

YOUTH MOVEMENT

The software has also been adapted for use by the Experimental Aviation Association (EAA) in its Virtual Flight Academy. “It just works great for youth environments like the EAA Young Eagles. TakeFlight really transforms one of these youth events where kids are sitting around waiting for a ride, and when they do get a ride it’s just a fire hose of information and they climb out of the airplane kind of bewildered. Like, wow, that was fun, but I have no idea what just happened.

If we can sit them on the sim and let them fly a little bit, see the cockpit for awhile, get a sense of what all the controls do… and then when they get in that airplane it is a completely transformed experience. Because now they’re not just looking at it bewildered with a pilot telling them not to touch anything. We’ve got a pilot encouraging them to touch things, and a student that’s figuring out, wow, I really can fly an airplane. And when they get out of that airplane, they’re absolutely beaming because they got to actually fly. And every time they see a little airplane fly across in the sky, they say I can do that. There’s this huge sense of pride.

For EAA Young Eagles the AI Instructor used the voice of ‘Aviore,’ the Young Eagles superhero mascot.

Excerpted from The Robot in the Simulator: Artificial Intelligence in Aviation Training by Rick Adams, FRAeS – https://aviationvoices.com/shop/