llm_instruction: instruction records

Instruction tuning fits datasets shaped as a task, an optional input and the expected output (Alpaca, most classification and extraction sets). The instruction and input become the user turn, an optional system prompt goes in front, and only the output is learned.

Rows

{"instruction": "Add the two numbers and reply with the sum only.", "input": "9 and 2", "output": "11"}

input may be left out. A Hub dataset with other column names maps them with fields (see the commented source in the config).

Config

configs/llm/instruction.yaml

# Instruction tuning: instruction/input/output records; the loss is on the output only.
pipeline:
  recipe: llm_instruction
  name: instruction-qwen3.5-0.8b
  seed: 20260920
  output_dir: ./output

model:
  architecture: hybrid              # Qwen3.5: linear-attention + full-attention layers, vision tower frozen
  name: Qwen/Qwen3.5-0.8B           # or Qwen/Qwen3.5-2B @ 15852e8c16360a2fea060d615a32b45270f8a8fc
  revision: 2fc06364715b967f1860aea9cf38778875588b17

data:
  sources:
    - name: local_jsonl
      path: samples/llm_instruction.jsonl   # rows: {"instruction": ..., "input"?: ..., "output": ...}
    # - name: huggingface                   # any Hub dataset; `fields` maps our keys to its columns
    #   dataset: owner/dataset
    #   revision: <40-character commit>
    #   split: train
    #   fields: {instruction: question, output: answer}
  validation: 0.1

method:
  system: "You are a careful calculator."   # optional system prompt in front of every instruction

training: {epochs: 1, lr: 1.0e-5, batch_size: 4, accumulation: 2, max_length: 512, max_steps: 4}

validation:
  max: {loss: 5.0}

Use

uv run llmclick run configs/llm/instruction.yaml

• Gate: validation.max: {loss: ...} on the held-out outputs; perplexity is reported too.
• Settings: method.system is the system prompt in front of every instruction; leave it out for none.
• Output: output/instruction-qwen3.5-0.8b-<hash>/validate/checkpoint.

Source: configs/llm/instruction.yaml. This page is generated; edit the source and run npm run sync.