Where it started
A talking plush toy depends on devices, networks, voice services, character content and an app. A working module alone does not guarantee a complete conversation.
My contribution
I led development of the device-to-cloud voice pipeline and contributed to device, service and app integration, character resources, factory coordination, demonstrations and customer technical communication.
The real unit of a system is a complete user experience.
Three engineering decisions
Debug the entire path
Connect ESP32-S3, WebSocket, ASR, LLM and TTS with local wake-up, cloud conversation, interruption and lighting tool calls.
Make content inspectable
Organize packs by character, voice and language, and check their contents individually. Content scale is kept separate from physical-device test coverage.
Turn incidents into engineering practice
Use state and dependency checks, touch-channel timeout isolation, deployment checks, usage accounting and circuit breaking to make troubleshooting repeatable.
What came out of it
- As of October 22, 2026, 62 characters mapped to 186 Chinese, English and Spanish resource packs with content validation.
- Historical companion-app records show 151 passing automated Flutter tests. This is not a complete app-to-robot hardware acceptance test.
- Both products were exhibited. I worked across factory coordination, demonstrations and customer communication.
Device and content checks; app tests are separate from hardware integration
What's next
Develop first-generation touch and shake interactions alongside second-generation on-device conversation research. Full on-device ASR, LLM and TTS remain subject to validation.
Project notes are grounded in reviewed source, experiment records and personal confirmations.
Content reviewed: 2026.10.28