Speech datasets: `data_speech`

data_speech makes the recordings the speech recipes train and are measured on: a teacher writes lines (or the seeds are the lines), a Piper voice reads each one, Whisper checks what it heard, and the recordings whose transcript matches their text are written out with a manifest that carries the voice's terms.

Shipped config	Makes	Read by	For
`data/speech.yaml`	assistant lines written by a local model, plus the agent's fixed replies	one voice (`jgkawell/jarvis`)	`tts_piper`
`data/wake_phrase.yaml`	the name, "Jarvis.", as a statement, question and call	300 speakers of LibriTTS-R	`wakeword` positives
`data/wake_negatives.yaml`	near misses, the commands said alone, ordinary lines	40 speakers	`wakeword` negatives
`data/wake_nearmiss.yaml`	20 near misses of the name	120 more speakers	`wakeword` negatives
`data/wake_eval_phrase.yaml`, `wake_eval_negatives.yaml`	the name, near misses and commands	VCTK, a voice training never heard	`evaluation_wakeword`

Five stages take seeds to a directory Piper can train on. The first two are those of data_synthetic (seeds, and the texts made from them); then synthesize, verify and select. Every recording and every transcript is kept under its natural key, so a run that stops halfway continues where it stopped, and a run with stricter checks reuses the recordings it already has.

Stages and settings

pipeline: {recipe: data_speech, name: jarvis-lines}
teacher: {name: ollama, model: qwen3.5:0.8b}                # only for generators that ask a model; passthrough needs none
seeds: [{kind: situation, values: [a morning briefing, a security alert]}]
prompts: {generator: utterances, params: {n: 5, language: English, speaker: "a formal British AI butler"}}
voice:
  model: {name: jgkawell/jarvis, revision: <commit>, file: en/en_GB/jarvis/medium/jarvis-medium.onnx}   # or a local .onnx
  samples: 1                                                # recordings per text; length_scale, noise_scale, noise_w_scale
  speakers: null                                            # a multi-speaker voice: ids [3, 7] or n for ids 0..n-1
verify: {min_seconds: 0.5, max_seconds: 15, max_clipped: 0.001, max_silent: 0.5, max_wer: 0.2, asr: {language: en}}
select: {per_text: 1, max_rows: null}                       # per_text null keeps every recording of a text
licence: "terms of the voice, copied into the manifest"

A Piper training set from seeds: seeds and prompts are the stages of data_synthetic (the utterances generator asks for lines to be read aloud, one or two sentences with numbers written as words; passthrough reads the seeds as they are, so the lines of an existing metadata.csv can be read again through the delimited source; a seed of kind line, such as the agent's name or a fixed reply, is read as it is by every generator). A Piper voice reads every text and Whisper checks what it heard. The voice runs as a separate process from the Piper environment (environment/piper/setup.sh, PIPER_PYTHON; see the tts_piper section of the main README).

Stage	Does	Idempotent because
`synthesize`	the voice reads each prompt `samples` times, by each of `voice.speakers`, into `audio/<key>.wav`; `\|` and line breaks are taken out of the text first, so the text is exactly what was read	the voice, its settings and the Piper lock are in the fingerprint, so a file is named by prompt, speaker and sample alone; a rerun reads only texts without a file. Piper's noise is unseeded, so the first reading is the one kept
`verify`	length, clipped samples and silent frames first; then Whisper (`verify.asr`, pinned) transcribes the recording and the word error rate against the text decides; every rejection keeps its reason and measurements	pure checks over the files; each transcript is cached under the recording's key
`select`	per text (case and spacing ignored) the `per_text` recordings Whisper understood best; `wavs/`, `metadata.csv` (`wavs/<key>.wav\|text`), `rows.jsonl` (`audio`, `text`, paths relative to the file) and `manifest.json` with the voice, the licence and each stage's counts	pure function of the verified rows; recordings are hard-linked (copied across file systems) through a temporary name

The dataset directory is the stage directory, reached through <experiment>/select/: tts_piper reads its rows.jsonl with local_jsonl, and Piper itself reads metadata.csv with that directory as --data.audio_dir. The licence is a required key because a voice's terms bind every recording it makes; the manifest is where the next reader finds them.

Many speakers serve the wake word: data/wake_phrase.yaml has 300 speakers of one voice read the wake word and keeps every recording Whisper hears exactly, and wakeword trains on them (main README). A Piper training set stays single-voice: Piper's multi-speaker format (a speaker column) is not written yet.

Not yet: Piper's multi-speaker training format, text normalisation beyond what the generator asks for, a speaker-similarity check.

Source: data/README.md. This page is generated; edit the source and run npm run sync.

Stages and settings​

Stages and settings