Split an LLM evaluation by source document and time
Prevent evaluation leakage by grouping related questions and document revisions, using time-aware boundaries, and keeping prompt development away from the final holdout.
Foundation
The journal / Ideas in progress
Engineering & everyday learningNotes from
the work.
Software, systems, and the things I’m learning along the way. From first principles to AI in production.
Find something to readAll notes 1
A little theory. A lot of practice.