A constant beats a vector database, until it doesn’t

Eight thousand words of domain playbook fit in a prompt. We shipped a constant instead of a retrieval system.

One of our products carries a serious body of method: how good boards run, what a real agenda looks like, what belongs in a minute. The reflex answer in 2026 is retrieval: chunk it, embed it, stand up a vector database, search at runtime.

We put it in a string constant instead. The whole playbook rides along in the system prompt, and prompt caching makes the repeat cost close to nothing. Follow-up turns pay a fraction of the first one.

Why the dumb version wins

Retrieval adds real machinery: a database to run, an indexing pipeline to maintain, a relevance problem to debug when the model answers from the wrong chunk. A constant has none of that. It cannot return the wrong passage, because it returns all of them. At eight thousand words the whole corpus fits comfortably; there is nothing to rank.

The line where it flips

Retrieval earns its keep when the corpus outgrows the prompt, somewhere past thirty thousand words, or when every customer brings their own documents and the knowledge stops being one shared playbook. Those are real thresholds, and crossing them is a product decision, not a default.

We wrote the swap into the design: the constant sits behind one function, so the day retrieval is earned, it is a query instead of a string. Until that day, the boring version is faster, cheaper, and easier to trust.