For this project, I developed a quick demonstrator to showcase the capabilities of the HoloLens within the context of Air Traffic Control (ATC). The user is provided with real-time information about aircraft near the airport, specifically within the NARSIM, the in-house NLR ATC Research Simulator.
I created labels (visible at 0:51 in the video) that are displayed at the location of each aircraft. When a problem occurs, such as a runway incursion or a go-around, a pop-up appears to direct the user to the affected aircraft. Additionally, a line is drawn between the user and the aircraft to visually indicate where the problem is likely to occur.
The primary challenge of this project was the tight timeline—I had just one week to design and develop the application. The project had three key components:
- Communication with the tower to receive the locations of the aircraft and identify potential problems.
- Object placement system to accurately position all elements based on anchor locations, ensuring proper alignment in the 3D environment.
- Pop-up system that remains anchored to the screen, guiding the user towards the potential problem even as they move around.
Since I needed to demonstrate the application on multiple HoloLenses simultaneously, I had to find a way to serialize the anchors. This became particularly challenging after upgrading to Unity 2019. Fortunately, I was able to quickly recreate the necessary functionality.
Skills Used
Communication: Data flow from NARSIM to clients was managed through WebSockets, with SOAP messages.
Application Development: Unity (C#) to create the HoloLens application for the ATC operators.
See also
The published paper about the project: https://www.researchgate.net/publication/360569113_Attention_Guidance_for_Tower_ATC_Using_Augmented_Reality_Devices