Gabin Maxime Nguegnang
← Build notes

Post

When my AI's quality check raised an alarm, the answer's passage got through half as often as when it stayed quiet

My AI assistant has 4 steps on paper: search, rank, check, write. Measuring what each one did to the passages it reads showed that one barely chose anything and another could only lose evidence.

  • RAG
  • Evaluation
  • Information retrieval
  • AI reliability
Animated dark alarm panel. On the left, one question goes through the old 4 steps of my AI assistant. Search keeps 100 passages, and 94% of the sampled papers had 100 or fewer, like a hiring funnel whose first screen passes every applicant. Rank lets ColBERTv2 set the order. Check raises an alarm and deletes every passage after its line, including the one holding the answer. Write reads only what is left and gives a flagged refusal. On the right, the answer's passage got through 85% of the time when the check stayed quiet and 42% when it raised an alarm; 13 of 22 flagged refusals came on those questions. Below, the technical details, the limits, and the lesson: if a step can delete passages but can't fetch replacements, compare what the model reads with that step on and off. Animated dark alarm panel. On the left, one question goes through the old 4 steps of my AI assistant. Search keeps 100 passages, and 94% of the sampled papers had 100 or fewer, like a hiring funnel whose first screen passes every applicant. Rank lets ColBERTv2 set the order. Check raises an alarm and deletes every passage after its line, including the one holding the answer. Write reads only what is left and gives a flagged refusal. On the right, the answer's passage got through 85% of the time when the check stayed quiet and 42% when it raised an alarm; 13 of 22 flagged refusals came on those questions. Below, the technical details, the limits, and the lesson: if a step can delete passages but can't fetch replacements, compare what the model reads with that step on and off.
The whole post in one graphic. Open the graphic full size ↗

When my AI’s quality check raised an alarm, the answer’s passage got through half as often as when it stayed quiet.

I had called it a gate that refuses rather than guesses.

Why it matters

I build an AI assistant that answers questions about research papers and cites its sources. On my diagram it has 4 steps: search, rank, check, write.

Then I checked what each step did to the passages it reads.

Once the right paper was picked, search usually had nothing left to choose. It keeps 100 passages, and 94% of the papers in my sample had 100 or fewer. Picture a hiring funnel whose first screen passes every applicant.

The quality check was worse. It deleted passages it judged weak, and the assistant had nowhere else to look. When it raised an alarm, the answer’s passage got through 42% of the time, against 85% when it stayed quiet. 13 of 22 flagged refusals came on those questions.

Harder questions may set it off more often, which would explain part of that gap. But deleting could only lose evidence. The published method I borrowed fetches replacements from the web.

So ranking, plus how many passages I kept, decided what the assistant could read.

I now check what each step changes before I count it as part of my system.

How it works

The check is a CRAG-style gate on ColBERTv2 scores. CRAG replaces discarded passages with web results; scoped to one paper, mine had none. It now only labels, and I don’t trust the label yet: its threshold was fit on search across all papers (F1 0.55).

Stage 1 (SPECTER2 + BM25, RRF, k=100) runs inside the paper. On my old text-only index the median sampled paper had 47.5 chunks, so stage 1 usually returned the whole paper and ColBERTv2 set the order. HyDE, an extra LLM call on 75% of questions, now runs only when it can change the result. RRF as the final ranking raised table hits but cut text-evidence recall at k=20 (0.884 to 0.831), so it stays opt-in.

Limits:

  • 42% vs 85% is observational: one run, 39 of 150 questions fired, and the old gate also cut passages when quiet
  • The benchmark’s own label picks the paper, and the 94% predates tables and abstracts in the index

The lesson I took: if a step can delete passages but can’t fetch replacements, compare what the model reads with that step on and off.

The animation illustrates one question going through the old 4 steps.

If your team runs AI over its own documents and wants to know which steps earn their cost, my DMs are open.