llm_decision_cispo: reason, then decide

The second stage of a reasoning decision model (Jeeves). It starts from a stage-1 checkpoint (model.init), samples method.group_size reasoning chains per question, rewards each by the probability the head then gives the right option, and reinforces the chains that beat their group's mean (CISPO: the importance weight of each reasoning token is capped). The head's loss after reasoning and without reasoning is added, so the one-pass answer keeps working.

Use it when reasoning measurably helps on your questions: compare accuracy with think_accuracy in stage 1's validation, or run evaluation_decision with think: both.

Rows

The same Jev records as stage 1.

Config

configs/llm/decision_cispo.yaml

# Reasoning decision model, stage 2 (Jeeves CISPO): the model reasons, then decides. Reasoning that leads the head
# to the right option more surely than the other chains sampled for the same question is reinforced.
# Run configs/llm/decision_pointer.yaml first: `model.init` points at the checkpoint it writes. That path contains
# the stage-1 experiment key, which `llmclick validate configs/llm/decision_pointer.yaml` prints.
pipeline:
  recipe: llm_decision_cispo
  name: decision-cispo-qwen3.5-0.8b
  seed: 20260920
  output_dir: ./output

model:
  architecture: hybrid
  name: Qwen/Qwen3.5-0.8B
  revision: 2fc06364715b967f1860aea9cf38778875588b17
  head: pointer
  init: output/decision-pointer-qwen3.5-0.8b-e442dea1/validate/checkpoint   # exists only if stage 1 passed validation

data:
  sources:
    - name: local_jsonl
      path: samples/llm_decision.jsonl
      limit: 48
  validation: 0.15

method:
  group_size: 4                     # reasoning chains sampled per question
  max_think: 64
  temperature: 1.0
  rollout_weight: 0.5               # head loss after reasoning
  anchor_weight: 0.5                # head loss without reasoning, so the one-pass answer keeps working
  length_hinge: 48                  # chains longer than this lose up to length_penalty_cap of their reward
  calibration: 0.15

training:
  epochs: 1
  lr: 2.0e-5
  batch_size: 3                     # questions per step; each gets group_size chains
  max_length: 768
  max_steps: 4
  adapter: {name: lora, r: 16, alpha: 32}

validation:
  min: {accuracy: 0.3}            # think_accuracy (after reasoning) is reported next to accuracy

Use

uv run llmclick run configs/llm/decision_pointer.yaml    # stage 1; prints the experiment key
uv run llmclick run configs/llm/decision_cispo.yaml      # model.init points at stage 1's validate/checkpoint

• Gate: accuracy (one pass) with think_accuracy (after reasoning) next to it, both calibrated.
• Cost: each step forwards batch_size x group_size sequences; method.max_think caps a chain, and length_hinge makes long chains lose reward.
• Settings: method.iterations and method.clip reuse a sampled group for several steps (see llm_grpo).

Source: configs/llm/decision_cispo.yaml. This page is generated; edit the source and run npm run sync.