3D avatars for voice assistants
Three 3D characters that speak with the agents' voices, with lip movement matched to speech and emotions.
Runs in the browser and on an external display
Task
A voice assistant without a face is harder to follow in a conversation. The goal was lifelike facial expressions on own hardware, without paid 3D models.
Solution
Heads built in code with three.js, mouth movement computed from text and audio every 10 ms. The server renders on the GPU and streams video to a lightweight receiver.
Result
- Lip sync accurate to 10 to 30 ms
- 720p60 latency of about 83 ms
- The receiver uses 16 MB instead of 382 MB of memory
My role
Character design, animation, lip sync and streaming.
