Another Set of Eyes: Why It's Time for Aviation Perception Tech

A commercial pilot once asked me why anyone would need navigational cameras on an airplane. “Once we are wheels up, we take an old newspaper and spread it across the windscreen for shade! Autopilot does the rest.”

His half-joking comment is true for greater than 90% of flying. But near airports, the story is different. Recent New York Times articles [1, 2] highlight a spike in safety incidents at airports within the United States. The causes range from understaffed air traffic control (ATC) [3], to pilot shortages, to reduced pilot experience, to the dynamic return to pre-pandemic air passenger levels. Aside from tough talk from the FAA, solutions remain sparse. Even more concerning is a quiet reality of many of the close-calls investigated by the New York Times:

A watchful pair of pilot eyes and quick thinking was the difference between a close call and disaster.

“Are hundreds of passengers depending solely on the pilots’ visual judgment?” You ask. “Are there no other safety systems in place?”

The truth is that a number of systems already work together to help aircraft avoid collisions and navigate safely from gate-to-gate. These systems include:

• GPS to measure the position of the aircraft

• ADS-B to broadcast that position to other aircraft

• TCAS to help avoid in-air collisions

• ILS to guide the aircraft to a safe landing

• PSR/SSR to coordinate traffic in the airspace

However, even with these decades-old systems, pilots still leverage their eyesight heavily, particularly in terminal area operations such as taxi, takeoff and landing. Even for large commercial flights operating under Instrument Flight Rules (IFR), visual approaches to land under visual meteorological conditions (VMC) are in fact critical to acceptable traffic throughput at busy airport hubs. Typical landing procedures during IFR flight requires additional ATC oversight, longer approach trajectories, and additional separation between inbound traffic. Once ATC grants a “visual approach” to an aircraft with “traffic in sight”, they transfer responsibility to that crew to finish the approach at the crew’s discretion. This “trick” is often used to manage ATC workload as well as provide a more streamlined approach to land, saving fuel and improving arrival times.

During terminal area operations, pilot workload is near its highest and pilot’s must balance tracking traffic and other visual queues with numerous other tasks that often pull their attention into the cockpit. Additional on-board technology that can help monitor the external environment and aid pilots in their visual tasks can only help increase the safety and efficiency of these phases of flight.

Many aerospace manufacturers are developing vision-based perception technology which leverage modern processing techniques, including “AI”, to address specific shortcomings and add redundancy. This is especially true for phases of flight at and near airports [4, 5]. Applications range from seeing other aircraft in the sky (“detect and avoid”) to augmenting GPS for positioning on final approach (“visual landing”). Most importantly, these systems work continuously as an extra set of “well-trained eyes” to add safety and situational awareness to the cockpit.

Over the next several weeks, we will be pulling back the curtain on VTI’s effort to make the skies safer with perception technology. We’ll share our successes, our challenges, and our roadmap for bringing an extra set of “eyes” to the cockpit.