embedding_contrastive: retrieval embeddings

Contrastive learning pulls a query towards the text that answers it and away from the others in the batch, plus an optional hard negative per row. Use it to adapt an embedding model to your domain's queries and documents: search, retrieval for RAG, deduplication.

model.architecture	Model	Pooling
bi_encoder	Qwen3-Embedding-0.6B	one tower for queries and documents, last-token pooling, unit vectors

Rows

{"query": "sum of 9 and 2", "positive": "9 plus 2 equals 11.", "negative": "9 plus 2 equals 10."}

negative is optional and may be a list. Texts over training.max_length are truncated, as usual for retrieval.

Config

configs/embedding/contrastive.yaml

# Contrastive learning (InfoNCE) with in-batch negatives and optional hard negatives.
pipeline:
  recipe: embedding_contrastive
  name: contrastive-qwen3-embedding-0.6b
  seed: 20260920
  output_dir: ./output

model:
  architecture: bi_encoder          # one tower for queries and documents, last-token pooling
  name: Qwen/Qwen3-Embedding-0.6B
  revision: 97b0c614be4d77ee51c0cef4e5f07c00f9eb65b3

data:
  sources:
    - name: local_jsonl
      path: samples/embedding_contrastive.jsonl   # rows: {"query": ..., "positive": ..., "negative"?: str | [str]}
    # - name: huggingface
    #   dataset: sentence-transformers/all-nli
    #   subset: triplet
    #   revision: d482672c8e74ce18da116f430137434ba2e52fab
    #   split: train
    #   limit: 20000
    #   fields: {query: anchor}
  validation: 0.1

method:
  temperature: 0.05
  query_instruction: "Given an arithmetic question, retrieve the statement that answers it"

training: {epochs: 1, lr: 1.0e-5, batch_size: 8, max_length: 128, max_steps: 4}

validation:
  min: {accuracy: 0.5}              # share of held-out queries whose positive is ranked first

Use

uv run llmclick run configs/embedding/contrastive.yaml

• Gate: accuracy (the positive ranked first among the held-out documents) and mrr.
• Settings: method.temperature sharpens the softmax over candidates; method.query_instruction is prefixed to every query, as Qwen3-Embedding expects.
• Output: output/contrastive-qwen3-embedding-0.6b-<hash>/validate/checkpoint. Measure it on MTEB tasks with evaluation_embedding.

Source: configs/embedding/contrastive.yaml. This page is generated; edit the source and run npm run sync.