UW Unveils AI Headphones for Real-Time 3D Translation

Researchers at the University of Washington have developed Spatial Speech Translation, an AI-powered headphone system that translates multiple speakers in real time while preserving each speaker's direction, volume, and unique vocal qualities, even as they move. The system employs off-the-shelf noise-canceling headphones and advanced algorithms to separate, track, and clone speakers' voices and spatial locations for immersive translation. It currently supports English, Spanish, German, and French with a 2–4 second delay, and user studies show it performs well in both indoor and outdoor settings. The system avoids cloud-based processing to protect user privacy. The current prototype is limited to everyday speech and faces challenges with specialized terms and real-world noise. Expanding training data and real-world testing are seen as key steps for future commercial viability.