Grounding LLM features in your own data without shipping a hallucination
Retrieval is the easy half. The hard half is deciding what your system does when it is not confident — and measuring that before launch.
Ines Duarte
ML Engineer
Most AI features fail in the same place. Not retrieval, not the prompt, but the moment the model is asked something the documents do not answer. A system with no defined behaviour for that moment will invent one.
We start every AI engagement by building an evaluation set before choosing a model. A hundred real questions with known-correct answers, labelled by someone who does the job. It takes a week and it changes every decision that follows.
The second commitment is confidence gating. A field extracted below threshold is left blank and flagged for a human, not filled with the best guess. An empty field costs a nurse ten seconds. A wrong date of birth costs considerably more.
Finally, log every human correction. Corrections are the only honest signal about where your system is weak, and they arrive free of charge from the people already using it.