Llm

This is the retrospective for the multi-agent document analysis project. The first posts covered: why use multiple agents how the specialist agents work how the coordinator synthesizes findings This one covers what worked, what broke, and what I would change. In short, the architecture worked, the coordinator was the most valuable part, and chunking caused the worst failure mode. What worked The BaseAgent abstraction was enough. I did not need a framework. A simple base class handled the repeated LLM-call logic: model name, system prompt, max tokens, response cleaning, and JSON parsing. ...

After building the specialist agents, the output looked impressive. It was not useful enough. The system produced: 12 technical findings 14 risk findings 10 cost findings timeline findings That is a lot of analysis. It is also a lot to read. The coordinator is the piece that turns those separate findings into something a person can act on. Aggregation is not synthesis The first version of the coordinator just ran the agents and returned their results. ...

I built a document analysis system for RFPs and contracts using multiple specialist LLM agents instead of one general-purpose prompt. The architecture is simple: PDF → text extraction → Technical Analyzer → Risk Analyzer → Cost Analyzer → Timeline Analyzer → Coordinator synthesis → final report The interesting part is not that it calls an LLM. That’s easy. The interesting part is how much the output changes when the model is forced to analyze the same document through different lenses before producing a final answer. ...