Halden Pharma
Regulatory document search with citations the team trusts
Search time fell to about 2 hours a week. The evaluation set — 340 questions with known answers — runs on every deployment.
- returned per person per week
- 7 hrs
- evaluation questions, run per deploy
- 340
- of answers carry a citation
- 100%
Problem
Regulatory affairs staff spending an estimated 9 hours a week locating precedent across 240,000 submission documents spanning 22 years.
Solution
Retrieval over the submission archive with every answer citing the source paragraph, plus an evaluation set built by the regulatory team themselves.
Result
Search time fell to about 2 hours a week. The evaluation set — 340 questions with known answers — runs on every deployment.
In regulated research, an answer without a source is not an answer. That single constraint shaped the whole build.
How the work ran
- The regulatory team wrote 340 questions with known correct answers before any retrieval was built. That became the acceptance bar.
- Chunking and retrieval tuned against that set rather than against a generic benchmark.
- Every response renders the source paragraph inline, and an answer with no supporting passage returns nothing rather than a guess.
- Cost ceiling enforced per query, with the ceiling agreed before build started.
What made the difference
Returning nothing when there is no source is the feature that made this usable. It is also the one most often designed out.
“We had three vendors pitch an AI programme. VijnoraTech was the only one that arrived with an evaluation harness and a cost ceiling in the proposal.”
Marta Lindqvist
Director of Data & AI, Halden Pharma