Speech recipes

A voice agent hears and speaks. Two recipes build the audio models around it; what it decides stays in the agent (an LLM or a decision model from the recipes above).

Recipe	Builds	Starts from	Trained by	Validation
tts_piper	the voice that speaks the replies, and fixed replies as WAV files	a Piper (VITS) checkpoint	Piper's own trainer, in its own environment	Whisper listens: wer against the recordings' own reference_wer, phrase_wer
wakeword	the detector that fires when it hears the agent's name	openWakeWord's frozen speech features	a small head, in this process, on the CPU	recall, false_accepts, scenario (the name, then a command)

Both read recordings that data_speech makes from seeds (a teacher writes lines, a Piper voice reads them, Whisper checks them), or recordings of your own.

mic -> [wake word: always on, tiny] -> "Jarvis" -> [speech to text: the command] -> agent -> [TTS: the reply]
         wakeword                                                                              tts_piper

A voice agent from these recipes

The four configs below build what a voice agent says and hears; what it decides stays in the agent.

Moment	What runs	Built by
User: "Jarvis, wake up"	the wake word detector fires as "Jarvis" ends, once	wakeword (wakeword/jarvis.yaml)
	speech to text hears the command after it ("wake up"), and the agent's model picks the command wake_up	the agent (an LLM recipe can tune it)
Agent: ready	plays the read-back of wake_up, phrases/wake_up.wav	tts_piper validation.phrases
User: "Please search to the paper reference on the web"	speech to text, then the agent's model picks web_search	the agent
Agent: "Searching Reference.", then works	plays phrases/web_search.wav, then runs the search	tts_piper validation.phrases

tts/piper_jarvis.yaml holds nine read-backs: wake_up (At your service, sir.), web_search (Searching Reference.), horizontal_task (Processing Long Task.), remember_pattern (Checking the Pattern.), analysis (Target. Analysis.: a full stop so the voice pauses), focus_mode_enter (Activate Focus mode.), focus_mode_quit (Deactivate Focus mode.), persona_karen (Change to KAREN.) and persona_jarvis (Change to JARVIS.). The voice's training set (data/speech.yaml) reads them too. Which utterance means which command is the agent's to decide: no recipe here maps "enter the focus mode" to focus_mode_enter yet.

Source: README.md. This page is generated; edit the source and run npm run sync.