Remote Command interface for Autonomous Fleet supervision
ROLE
Lead product designer
Design system ( Dark mode)
STATUS
Shipped
TEAM
1 PM, 1 Designer, 2 Engineers
TIMELINE
May 2024- June 2024
Continuous releases ever since
PROJECT TYPE
Unity engine/mobile device
CONSTRAINTS
Human factors, accounting for time-sensitive tasks, unknown states, elongated GUI use based on the users job.
SUMMARY
When an autonomous vehicle hits a situation it can't resolve, a human has to step in — to guide it or take control — often within seconds, with safety and regulation on the line. I led design of the Remote Supervision interface: the system that lets one supervisor read a vehicle's state and intervene fast, across vehicles in different environments and network conditions.
The reframe that set the work apart: this isn't a monitoring dashboard, it's a shared-authority problem. I defined an interaction matrix mapping how much control the human holds at each autonomy state, the backbone the whole interface is built on/around three principles: clarity, control, containment.
The problem. Help a supervisor and an autonomous vehicle communicate fast enough to make safe intervention decisions — across multiple live feeds, under low connectivity, during unknown states and system exceptions, over long shifts in lit or dark environments.
KEY DECISIONS
Interaction matrix for shared authority — defined the human's control level at each autonomy state; set the logic behind every intervention flow.
WCAG AA on a live 3D scene — semantic-map colors, layers, and agents tested for contrast and distinguishability for prolonged use in any lighting. Real-time 3D accessibility is rare and hard.
Designed to scale the supervisor-to-vehicle ratio — built from inception for one → many vehicles per supervisor, the lever that drives driverless economics. (Gatik has publicly tied ~2 trucks per supervisor to roughly halving route cost.)
Direct-control modes — front-camera object focus and a low-latency rear-camera docking mode for hands-on takeover.
Validated across two cognitive profiles — an engineering group (technical fidelity) and a gaming-experience group (situational awareness under load), 5–8 each, over two rounds.
IMPACT
Operational rollout to 15+ Initial Remote Supervision Stations and then o 30 in 2026
Reduced supervision task load through the semantic Layer and better IA
First-of-its-Kind interface in a competitive stealth space
Ensured usability for distinct cognitive Groups through iterative testing.
DESIGING THE SCENE
The 3D scene, including the semantic map, needed to comply fully with WCAG AA 2.0. That meant the colors and layers, as well as NPCs and agents, had to be tested to have compliant contrast and distinctiblity to cater towards all varieties of accessibility needed by the users for a prolonged use of the interface during their shift in both lit or darkened amibent lighting.
The user intervenes by manipulating the way-points.
The user requested remote assistance waiting for the autonomous vehicle to receive it.
The user can pull up the map and scout the surroundings
Users can see the state of the stack at any moment
Camera feed design when front camera is focused on an road object ahead
The docking mode rear camera feed provides the user with what is needed for low-latency direct control scenarios
DESIGN APPROACH TO A COMPLEX PROBLEM
Discovery
Shadowed early test operators, mapped decision-making under uncertainty, and analyzed telemetry logs to identify intervention triggers.
Definition
Established guiding principles: clarity, control, and containment.
Built an interaction matrix defining human authority levels per autonomy state.
Design & Prototyping
Prototyped supervisory modes in Figma using simulated vehicle states.
Tested with 2 groups (5–8 participants each):
Engineering group — validating technical fidelity.
Gaming-experience group — validating situational awareness and focus.
Iterated on color tokens, alert clustering, and real-time feedback motion design.
Delivery
Delivered the first scalable front-end connected to the live fleet system, deployed across 10 supervision stations with real-time data streaming.
USABILITY TESTING ROUNDS
Usability testing was an iterative process to help quickly if the design solutions were working from a usability and human factors standpoint. It was complemented with another one which was focused on more participants' input to find more granular design bugs.
First round of testing
Second round of testing
SHOT-OUT TO THE TEAM!
