Human-Machine Interfaces Without a Steering Wheel

From Control to Trust: The Core HMI Challenge

The human-machine interface of a conventional vehicle is organized around a simple principle: the driver controls the machine, and the machine provides feedback about its state. Speedometer, fuel gauge, warning lights — every instrument communicates information the driver needs to operate the vehicle. When the driver is removed from the operational loop entirely, this feedback paradigm breaks down. The occupants of a fully autonomous vehicle are not operators; they are passengers. They do not need to know the vehicle's current steering angle or engine torque. But they do need to know something: that the system is functioning correctly, that it is aware of the environment around it, and that they are in safe hands.

This shift — from operational feedback to trust communication — is the central design challenge of autonomous HMI.[1] Trust is a subjective psychological state, not an objective measurement, and designing for it requires a different toolkit than designing for operational efficiency. A passenger who does not understand why the vehicle is slowing down will feel anxiety even if the slow-down is perfectly justified. A passenger who sees the vehicle correctly identify a cyclist in their peripheral vision before reacting will feel confidence that the system is attending to the right things. The HMI's job is not to provide data — it is to provide appropriate reassurance calibrated to the passenger's psychological state.

Ambient Awareness Displays

The dominant approach to communicating system state in current autonomous vehicles is the ambient awareness display: a simplified, non-intrusive visualization of the vehicle's perception of its surroundings that passengers can glance at without it demanding active attention. Waymo's Waymo One service features an in-vehicle display showing a simplified bird's-eye-view representation of nearby objects — cars, pedestrians, cyclists — as the system perceives them.[2] This display serves a dual purpose: it communicates that the system is attending to the right objects, and it provides a kind of transparency — an answer to the passenger's implicit question, "does the vehicle see what I can see?"

Research into passenger trust in autonomous vehicles consistently finds that transparency displays — showing the vehicle's intent and perception rather than just its current behavior — significantly increase perceived safety and willingness to use the service.[3] Passengers are more comfortable when they understand what the system is doing and why, even if the actual decision logic is too complex to explain in full. The ambient display provides a simplified but meaningful window into that logic.

Voice and Natural Language Interaction

Without a steering wheel or pedal assembly, the primary interactive interface for passengers in a fully autonomous vehicle is voice. This shift aligns with broader trends in consumer electronics — smart speakers, phone assistants — but the automotive context introduces specific requirements that general-purpose voice assistants do not address.[4]

Safety-relevant commands — "please pull over here," "I need to exit immediately," "I think something is wrong" — require immediate, reliable recognition and execution, not the variable latency and occasional misrecognition that is acceptable in a home assistant. The in-vehicle voice system must be robust against road noise, multiple simultaneous speakers, and accented speech. It must handle commands in multiple languages for international fleets. And it must distinguish safety-critical requests from routine preferences.

General Motors' Cruise Origin incorporated a voice interface designed specifically around ride-hailing scenarios: commands to adjust climate, change destination, call for remote assistance, or provide feedback on the service. Waymo's Waymo One vehicles feature an in-vehicle assistance button that connects to a human remote operations team — a voice interface of last resort for situations the automated system cannot handle.

Haptic Feedback: Communicating Without Screens

Haptic feedback — vibration, pressure, and texture communicated through physical surfaces — is an underexplored modality in autonomous HMI but one with significant potential, particularly for communicating safety-relevant information without requiring occupants to look at a display. Research in aviation has demonstrated that vibrotactile feedback (seat cushion vibrations indicating aircraft bank direction) can provide spatial awareness that visual displays cannot match, particularly when visual attention is occupied elsewhere.[5]

In automotive applications, haptic feedback has primarily been used in steering wheels — haptic alerts for lane departure, drowsiness detection, or collision warning. Without a steering wheel, these conventional haptic channels disappear. Their replacement requires rethinking where haptic feedback is delivered: seat cushions, armrests, and seatbelts are all candidate surfaces, each with different characteristics for the types of information they can convey.

Communicating System State: When Things Go Wrong

The HMI challenges for normal operation, though significant, are overshadowed by the challenge of communicating system states outside the norm: when the vehicle has entered a situation it cannot handle, when it is executing a minimal risk condition (emergency stop), or when it needs the passenger to take some action. These low-frequency but high-stakes interactions are the ones where HMI failure can erode trust irreparably or, in the worst case, contribute to a safety incident.

Best practice, emerging from both aviation HMI research and early AV deployment experience, emphasizes multimodal alerts (visual, auditory, and haptic simultaneously) for safety-critical situations, clear and calm language for system state announcements, and explicit communication of what the passenger should or need not do. The Waymo One interface, for example, communicates clearly when the vehicle is pulling over to address a technical issue — explaining the situation, confirming the passenger is safe, and providing instructions for contacting assistance — rather than simply stopping unexpectedly.

Communicating with Pedestrians: The External HMI Problem

Autonomous vehicles face an HMI challenge that human-driven cars do not: communicating intent to pedestrians, cyclists, and other road users who are accustomed to interpreting a human driver's behavior through eye contact, head nods, and small steering corrections. An autonomous vehicle that makes no communicative gesture is harder to read than a human driver whose behavior provides social cues.

Several manufacturers have explored external displays — LED matrices, projected light patterns, text displays — to communicate the vehicle's intent. Toyota's e-Palette concept used a light strip on the front of the vehicle to signal whether the vehicle was yielding to a pedestrian. Zoox's bidirectional vehicle uses external audio and light signals to communicate at crosswalks. The standardization of these external communication modalities is an active area of discussion in the ISO and SAE international standards bodies, as interoperability between different manufacturers' vehicles and consistent pedestrian expectations will require alignment on a common visual language.[6]