Notes

Working notes on retrieval, agents and production AI. Every one is written from something I built — and most of them exist because I got it wrong first and had to find out why.

I keep these because writing an explanation is the fastest way to discover I did not understand something. If they are useful to anyone else, that is a bonus rather than the point.

New here? Start with RAG, one problem at a time — it is the map, and it says plainly which parts I have built and which I have only read about.

Foundations

Start here. Everything else assumes these.

RAG or fine-tuning — they answer different questions

One changes what the model knows. The other changes how it behaves. Choosing between them is usually a category error.

basic#rag#fine-tuning#architecture

RAG, one problem at a time

Every piece of a retrieval pipeline exists because something simpler broke first. This is the ladder, and an honest note about which rungs I have actually stood on.

basic#rag#retrieval#architecture

What an embedding actually is

A vector where distance means similarity. The useful part is knowing what that buys you and, more importantly, what it cannot do.

basic#embeddings#rag#vectors

Retrieval

Chunking, embeddings, search, ranking.

Chunking sets the ceiling on everything downstream

Retrieval cannot return a passage that chunking never produced. Whatever you do afterwards is bounded by that.

basic#chunking#rag#embeddings

A chunk has to carry its own context

What gets embedded and what gets displayed do not have to be the same string. Once you separate them, a whole class of retrieval failure disappears.

intermediate#chunking#rag#embeddings

Why hybrid search, and how rank fusion actually works

A good embedding model cannot reliably find RTN-14. Keyword search cannot find a paraphrase. The interesting part is not running both — it is that you cannot add their scores together, and what you do instead.

intermediate#rag#retrieval#ranking#bm25

Engineering

Python, APIs, testing, observability.

Counting tokens without a tokenizer

I budget context in tokens and I do not use a tokenizer to do it. That is a deliberate choice, and the reason is the same reason every model ships its own.

intermediate#tokens#cost#context#llm

Production

What breaks, what it costs, and how you find out.

Abstention is a feature, not a failure

The expensive mistake is not "I don't know". It is a confident wrong answer about somebody's money, delivered in the brand's voice.

intermediate#rag#grounding#guardrails

Things I got wrong

Bugs that produced no error message.

A comment that described code I had not written

The docstring said the blocking call was dispatched to a thread so it would not hold the event loop. It was not. The results were identical, the tests passed, and the concurrency simply never happened.

intermediate#async#python#concurrency

The chunk that inherited the wrong heading

A short section fell below my minimum chunk size, merged backwards into the section before it, and took that section's heading with it. It then embedded under the wrong topic and became unreachable by anyone asking about it. Nothing errored.

intermediate#chunking#rag#embeddings

The tests that were lying to me

I had ninety-one passing tests and I felt safe. Three of them were protecting nothing at all, and the only way I found out was by breaking my own code on purpose.

intermediate#testing#mutation-testing#rag

Still to write

Notes under /learn/ are generated from posts/*.md by build.py — edit the markdown, not the HTML.