Skip to content

feat: embedding fine-tuning pipeline -- wire checkpoint lookup + CLI command #966

@Aureliolo

Description

@Aureliolo

Context

PR #962 designed the domain-specific embedding fine-tuning pipeline and shipped EmbeddingFineTuneConfig as a config stub. The config validates its invariants (required fields, path safety) but the Mem0 adapter does not consume it -- fine_tune.enabled=True raises "not yet supported" via a fail-fast validator.

This issue implements the actual pipeline.

Requirements

1. Wire checkpoint lookup into Mem0 adapter

When fine_tune.enabled=True and a valid checkpoint_path exists:

  • Load the fine-tuned model instead of the base model at backend initialization
  • If checkpoint doesn't exist at the path, log warning and fall back to base model
  • Remove the fail-fast validator on Mem0EmbedderConfig._reject_unimplemented_fine_tune

2. Fine-tuning CLI command

Implement the 4-stage offline pipeline as a CLI command (or Python script):

  1. Synthetic data generation -- LLM generates query-document pairs from org documents
  2. Hard negative mining -- base model embeds all passages, selects top-k confusing negatives
  3. Contrastive fine-tuning -- biencoder training with InfoNCE loss (tau=0.02, 3 epochs, lr=1e-5)
  4. Deploy -- save checkpoint, update config to point to fine-tuned model

Pipeline design is documented in docs/reference/embedding-evaluation.md.

3. Update docs

  • Remove "not yet implemented" notes from EmbeddingFineTuneConfig docstring, design spec, and reference page
  • Update docs to reflect actual behavior instead of planned behavior

Acceptance Criteria

  • fine_tune.enabled=True with valid checkpoint loads fine-tuned model
  • Missing checkpoint falls back to base model with logged warning
  • Fail-fast validator removed
  • CLI command runs the 4-stage pipeline end-to-end
  • Pipeline works on single GPU with no manual annotation
  • Tests cover checkpoint lookup (found, not found, disabled)
  • Docs updated to reflect implemented behavior

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    prio:mediumShould do, but not blockingscope:medium1-3 days of workspec:memoryDESIGN_SPEC Section 7 - Memory & Persistencetype:featureNew feature implementationv0.6Minor version v0.6

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions