Remote Command interface for Autonomous Fleet supervision
THE BACKDROP
The Remote Supervision GUI, along with our Hybrid Station serves as a tool for the RS(Remote Supervisor) to provide Carrier with remote assistance. The design of Remote supervision has to address main challenges a trained user RS might face which is to keep focus on two screens while maintaining a good judgment call when it comes to decision making in a matter of seconds.
ROLE
Lead product designer
Design system ( Dark mode)
STATUS
Shipped
TEAM
1 PM, 1 Designer, 2 Engineers
TIMELINE
May 2024- June 2024
Continuous releases ever since
PROJECT TYPE
Unity engine/mobile device
CONSTRAINTS
Human factors, accounting for time-sensitive tasks, unknown states, elongated GUI use based on the users job.
IMPACT
Operational rollout to 10+ Initial Remote Supervision Stations
Reduced supervision task load through the semantic Layer and better IA
First-of-its-Kind interface in a competitive stealth space
Ensured usability for distinct cognitive Groups through iterative testing.
DESIGING THE SCENE
The 3D scene, including the semantic map, needed to comply fully with WCAG AA 2.0. That meant the colors and layers, as well as NPCs and agents, had to be tested to have compliant contrast and distinctiblity to cater towards all varieties of accessibility needed by the users for a prolonged use of the interface during their shift in both lit or darkened amibent lighting.
The user intervenes by manipulating the way-points.
The user requested remote assistance waiting for the autonomous vehicle to receive it.
The user can pull up the map and scout the surroundings
Users can see the state of the stack at any moment
Camera feed design when front camera is focused on an road object ahead
The docking mode rear camera feed provides the user with what is needed for low-latency direct control scenarios
THE PROBLEM SPACE
When autonomous vehicles are being tested or are operating there are times that a remote human supervisor intervention is needed to help the vehicle decide or in some cases take-over. Things can get more critical when safety and regulation factors are involved. The main question is how to help the autonomous vehicle and human supervisor communicate with each other in a time-efficient manner.
CONSTRAINTS
Supervisors oversee up to +1 vehicles simultaneously, each in a different physical environment and network condition. This would change to multiple cars over time, so the system had to account that scaling from the design inception.
The system needed to:
Display multiple live feeds with minimal latency.
Support safe override actions and state recovery.
Function under low connectivity and real-time load.
Maintain clarity and calm even during system exceptions.
DESIGN APPROACH TO A COMPLEX PROBLEM
Discovery
Shadowed early test operators, mapped decision-making under uncertainty, and analyzed telemetry logs to identify intervention triggers.
Definition
Established guiding principles: clarity, control, and containment.
Built an interaction matrix defining human authority levels per autonomy state.
Design & Prototyping
Prototyped supervisory modes in Figma using simulated vehicle states.
Tested with 2 groups (5–8 participants each):
Engineering group — validating technical fidelity.
Gaming-experience group — validating situational awareness and focus.
Iterated on color tokens, alert clustering, and real-time feedback motion design.
Delivery
Delivered the first scalable front-end connected to the live fleet system, deployed across 10 supervision stations with real-time data streaming.
USABILITY TESTING ROUNDS
Usability testing was an iterative process to help quickly if the design solutions were working from a usability and human factors standpoint. It was complemented with another one which was focused on more participants' input to find more granular design bugs.
First round of testing
Second round of testing
SHOT-OUT TO THE TEAM!
