What LLMClick is
LLMClick builds custom models from a config file. You write one YAML that names a base model, the rows to train on and a few settings, run one command, and get a trained checkpoint that has passed a validation gate. The same YAML run twice builds nothing the second time, and a run that crashed halfway continues where it stopped.
Three channels share one runner:
- Modeling turns a config into a trained, validated model. Recipes cover supervised fine-tuning, instruction tuning, preference tuning (DPO), reinforcement with rewards (GRPO), decision models that answer typed questions, and contrastive training of embedding models. Full fine-tuning or LoRA, on a GPU or on Apple silicon.
- Evaluation scores what Modeling built: on your own rows, on public benchmarks, against the base model, with the overlap between the evaluation items and the training rows reported next to every score.
- Data makes the rows: synthetic conversations, preferences and problems from a teacher model (an API or a local model), with the filtering and leak removal the published recipes use.
The mental model is small. A recipe is a pipeline of stages; each stage's output directory is named by a fingerprint of its inputs, so stages are cached and shared between experiments; an experiment is one config and links to the stage directories it used. Everything else (the CLI, the HTTP API with background jobs, the tracker) drives that runner.
Who it is for: people who need a small, specific model (0.6B to 2B parameters) tuned on their data and want reproducibility without writing training loops. If you already have a training script you like, LLMClick's value is the fingerprinted pipeline, the validation gate and the evaluation channel around it.
The table below lists the recipes and the models each one accepts.
Documentation: https://ash-hun.github.io/LLMClick/ (generated from this README, the channel READMEs and the code; see website/README.md).
Config-driven custom model building. One YAML describes one custom model; pipeline.recipe picks the recipe that
builds it and pipeline.stages picks the steps to run. Every recipe runs on the same pipeline runner, which skips
what is already built, shows progress, and never hands on a trained model that has not passed validation.
| Family | Recipe (pipeline.recipe) | Trains | Model catalog (model.architecture) |
|---|---|---|---|
| LLM | llm_sft | conversations, loss on every assistant turn | transformer (Qwen3-0.6B, 1.7B), hybrid (Qwen3.5-0.8B, 2B) |
| LLM | llm_instruction | instruction/input/output records | same |
| LLM | llm_dpo | chosen/rejected preference pairs | same |
| LLM | llm_grpo | prompts scored by a reward function | same |
| LLM | llm_decision_sft | decision questions: a head scores the options (Jev, jeff, Jeeves stage 1) | same, plus model.head: readout or pointer |
| LLM | llm_decision_cispo | the same questions with reasoning before the decision, by reinforcement (Jeeves stage 2) | same |
| Embedding | embedding_contrastive | query/positive/negative triples (InfoNCE) | bi_encoder (Qwen3-Embedding-0.6B) |
| TTS | tts_piper | recordings and their texts: a Piper (VITS) voice fine-tuned from a Piper checkpoint, exported to ONNX | model.quality: medium, high (Piper's sizes) |
| Wake word | wakeword | recordings of the agent's name ("Jarvis") and of everything else: an openWakeWord detector, exported to ONNX | openWakeWord's dnn head on its frozen speech features |
| Evaluation | evaluation_custom | nothing: scores a checkpoint of any experiment above on any rows, with that recipe's metrics | the experiment's |
| Evaluation | evaluation_benchmark | nothing: scores a checkpoint on public benchmarks (lm-evaluation-harness), one cached stage per benchmark | the experiment's (LLM) |
| Evaluation | evaluation_compare | nothing: several checkpoints (the base model among them) on the same benchmarks and rows; table, deltas with intervals, polygon chart | the experiments' |
| Evaluation | evaluation_decision | nothing: a decision checkpoint on Jev rows or the JevBench tiers, without and with reasoning; accuracy, NLL, ECE, per tier, latency | the experiment's (decision) |
| Evaluation | evaluation_embedding | nothing: an embedding checkpoint on MTEB tasks (mteb), one cached stage per task | the experiment's (embedding) |
| Evaluation | evaluation_wakeword | nothing: a wake word detector on recordings and hours of audio it never heard; miss rate per noise condition, false accepts per hour, the threshold sweep | the experiment's (wake word) |
| Data | data_synthetic | nothing: rows for the recipes above from seeds through a teacher model (seeds, prompts, evolve, respond, verify, select) | teachers: anthropic, openai, ollama, local |
| Data | data_speech | nothing: a Piper training set from seeds: texts (a teacher, or the seeds as they are) read by a Piper voice, checked by Whisper | voice: any Piper .onnx, one speaker or many |
The decision recipes follow the Jev request format. The readout head and one-pass training follow jeff; the pointer head, reasoning chains and CISPO follow Jeeves. Both are reimplemented on this framework's training loop, so their released checkpoints do not load here.
Source: README.md. This page is generated; edit the source and run npm run sync.