Is checked knowledge worth more than a stronger model?

A model can recall a rule from training and get it wrong without any sign that it’s wrong. In finance a wrong answer about a rule can cost money or break the law.

fin-regbase is my test. Every page is checked by hand and cites a document and section.

What I’ve measured: on 109 questions, search finds the right pages in the top 3 for 91. The full answer eval passed 69.7% on 13th of May, 2026.

What I haven’t measured: the same questions answered by an agent with the wiki and without it. That comparison is the actual answer to this question, and I haven’t run it.