At 4:30 in the morning, behind the lost and found counter at Kamlapur, the only person awake in the building besides the boy who tends the boiler room is whoever is sorting yesterday's wrongly shelved luggage. A single bulb on a frayed cord throws a sodium pool across the wooden counter, and the cubbies on the back wall climb up to the clock, built of iron and wood, that's been keeping passably accurate time since 1969. Each bag has a tag. Each tag has a destination it didn't reach. Before the 05:40 to Sylhet starts boarding on platform four, the bags have to be sorted by why they got lost, not by where they ended up, because the second list is useless for fixing the first.
Production RAG over a brand voice corpus is the same shift.
By the time a chatbot ships, somewhere between ten and twenty percent of its retrievals have quietly ended up in the wrong cubby. Across maybe a dozen of these systems I've watched in 2024 and 2025, the number keeps coming back as fourteen percent. The user sees "you lost my bag." The clerk has to know the six reasons it happened, because the fix is different for each one.
The first failure mode is synonym drift. The corpus says "cancellation policy" in every doc on the planet, and the user asks about "refund window," and the cosine score on the right document is half a point too low because the embedder has decided the two phrases live in slightly different neighbourhoods. The bag goes to the wrong cubby. Hamel Husain's post on debugging RAG, quoted constantly at this point, walks through the same family of misses and the same fix, which is to look at your actual user queries, in their actual phrasing, not the queries you imagined when you wrote the eval set. The team that built the synthetic eval set will end up sampling synonyms too thinly, because writing fifty queries that mean the same thing in different words is boring. A real user typing with one hand into the chat widget will paraphrase the same question four ways inside ninety seconds without noticing.
The second is loss at the chunk boundary, and I have lost more weekends to it than I'd like to admit. The default text splitter (any of the three you reach for first) cuts sentences in half if you let it, and the halves embed terribly. You query for "return policy" and the chunk that ranks first starts with "Unlike our return policy, the exchange window is..." which is technically about the return policy in the same way a footnote is technically about its parent text. The cosine score lies because the embedder is reading a fragment as if it were a complete claim. The fix is unglamorous, and everyone in the field already knows it. Chunk on sentence boundaries with overlap, or, better, prepend a short summary, no longer than a paragraph, of each chunk's role in the parent document before embedding, the trick Anthropic's contextual retrieval writeup measured a 49% improvement on. Boris Power has been making the same point about chunking strategy on his podcast appearances for years; the substance of his argument is that the chunk is your real unit of retrieval, and treating it as a downstream optimization is the original sin.
The third is hallucination in the query rewrite, and this one mostly bites teams that have layered a small rewriter LLM in front of their embedder to "improve recall." The rewriter, being an LLM, will from time to time add a city that nobody asked about, or expand an acronym in the wrong direction. Sometimes it helpfully turns a question about API rate limits into a question about rate limits for renting a car, and the rewritten query then retrieves perfectly accurate documents about a problem the user does not have. The first time I saw this in production the rewriter was inflating "DOH" (the client's Department of Health document set) into "Doha" and pulling the wrong country's filings. The clerk shelves the Doha bag on platform seven; the bag was supposed to be on platform four. The fix is not to remove the rewriter but to log the rewritten queries in production and have somebody who isn't on the ML team read a hundred of them on a Friday afternoon. The exercise is dull, and it is also the only way anyone has ever caught this in practice.
The fourth is recency bias, which I keep meeting in corpora that have been through a CMS migration. The newest version of the doc keeps winning even when the user's question is about an older policy the new doc supersedes but doesn't fully replace. The bag gets shelved on platform six because that's where the most recent train from Rajshahi happened to be parked when the porter walked in. Sometimes the index isn't the culprit and whatever is breaking ties in your reranker is, which you only catch by inspecting metadata on the top-k for one suspicious query.
The fifth is flooding from documents that are nearly identical to each other, and it's the one that teams mistake for a retrieval quality problem, when what it really is, most of the time, is a corpus that was never cleaned up. Most brand corpora I've seen in 2025 contain three to five copies of every important doc that are near enough to identical, the residue of a migration from one CMS to another, with a SharePoint copy nobody deprecated in years still alive and indexed because nobody had the password to delete it. The model retrieves the same chunk nine times out of ten, the "diversity" of context is a fiction, and the answer is grounded in a version of the doc that was sunset two years ago. The clerk's shelf has six bags on it that are all labelled KHULNA, and only one is the bag the woman in the green saree is actually looking for. The fix is to dedupe the corpus before you embed it, and the fix is so boring it gets cut from the sprint plan twice before someone notices the bug.
The sixth is the near miss where cosine similarity looks close enough to pass but the topic is off entirely, the one I find most often and the one teams resist diagnosing because it looks like a model problem instead of a retrieval problem. Cosine similarity around 0.84 between two embeddings is not proof that the documents share a topic, only that the two live close enough in the embedding space that the model couldn't draw a moat between them. JESSORE and JOYDEBPUR rhyme in vector space the way they rhyme on the timetable, and the top-1 chunk is from the wrong town. Liu et al.'s Lost in the Middle paper showed in 2023 that even when the right chunk sits in your context window, the model will ignore it if it's at rank seven, which is the gentler twin of this problem. The harder version is the chunk at rank one being about the wrong city, with the model dutifully grounding an answer in it because that's what we trained it to do.
The order I work them is dedupe first, chunk boundaries second, the rewriter audit on a Friday afternoon third, and only after that the process of running both evals side by side, which catches the rest as a population instead of as complaints that show up one at a time. Synonym drift and near misses on cosine similarity mostly fall to reranking, which any pipeline this side of 2026 already has bolted on, and recency bias gets solved by the metadata audit that everybody pretends to have done and almost nobody has actually done. The bigger essay underneath all six is that production RAG fails for the same reasons production search has always failed. A corpus is messy, a query is fuzzy, and the seam where the two meet is where the bag falls off the trolley between platforms four and seven on a wet Tuesday in the monsoon.