Set an LLM timeout from the user's remaining deadline
Propagate a user request deadline through retrieval and generation, reserve time for a fallback, and avoid retries that continue after the answer is no longer useful.
Foundation
The journal / Ideas in progress
Engineering & everyday learningNotes from
the work.
Software, systems, and the things I’m learning along the way. From first principles to AI in production.
Find something to readAll notes 1
A little theory. A lot of practice.