Did it answer well? Evals 101

"Looks good" is not a metric

You asked the assistant one policy question and it looked right. Ship it? No. Change a chunk size, tweak the prompt, swap the embedding — and you have no idea if you made it better or worse. You're steering by vibes.

The fix is an eval: a small fixed set of questions with known-correct answers, plus a script that scores how often the system gets them right. Run it before and after every change. Evals turn "feels better" into a number that went up or down.