What Gets Remembered
| Source | What’s Indexed | How Often |
|---|---|---|
| Journal entries | Features, fixes, decisions, discoveries, pivots | Every 60 seconds |
| Code headers | @purpose annotations from source files | Every 5 minutes |
| Manual memories | Insights, notes, decisions via jfl_memory_add | On demand |
How Search Works
TENET uses hybrid search — combining lexical and semantic approaches for best results:BM25+ (Always Available)
Term-frequency scoring with:- Stopword removal and phrase detection
- Adaptive document length normalization
- Query term weighting based on IDF
- Positive IDF floor (BM25+ variant) — common terms still contribute
Semantic Search (When Embeddings Available)
Cosine similarity ontext-embedding-3-small vectors:
- 1536 dimensions
- OpenAI or OpenRouter fallback
- Auto-backfill: if key was missing when indexed, embeddings are added later
Reciprocal Rank Fusion
Merges BM25 and embedding results by rank position, not raw scores. More robust than linear interpolation because it doesn’t require score normalization.Current Stats
Graph Edges
Memories aren’t isolated. They connect to each other:| Edge Type | Meaning | Example |
|---|---|---|
updates | New info supersedes old | ”CLI speed now 98ms” updates “CLI speed was 6.7s” |
contradicts | New finding invalidates old | ”Connection pooling helps” contradicts “Keep connections short” |
related_to | Topically connected | Memory about eval system → related to agent config |
caused_by | Causal relationship | ”Test failures” caused_by “Dependency upgrade” |
part_of | Hierarchical grouping | Session memories → part_of project milestone |
Code Header Indexing
Files with@purpose annotations are automatically indexed:
- Source:
file - Type:
code-header - Content:
src/lib/memory-indexer.ts: Automatic indexing of journal entries and code headers
src/, packages/, scripts/, eval/
Updates if @purpose changes. Deduped by file path.
Knowledge Doc Lifecycle
Knowledge docs (VISION.md, THESIS.md, etc.) are audited for staleness:jfl organize generates a PENDING.md with proposed updates and open questions for human review.
Search
BM25+ hybrid search internals and query optimization.
Graph Edges
Structured relationships between memories.
Embeddings
Auto-backfill, model selection, and fallback behavior.
Code Headers
Indexing @purpose annotations from source files.