Case · 02 — Production
Auditable research system with agents
Personal use
A plain-language request, a team of agents assembled to fit from 16 specialists, and a report that shows what percentage of its data was confirmed against the source.
US$0,10average cost per run
The problem
AI-generated reports sound just as convincing when they are right as when they make things up: a number with no source, a trend nobody measured, a competitor that does not exist.
What I built
A planner builds the team from a catalogue of archetypes and calls on only the ones it needs. Outputs are validated against 49 Pydantic models, a verifier checks each data point against its source, and a three-layer judge allows bounded retries. Cost is estimated before each run, with model tiers and caps.
What changed
Evaluated with an offline golden set and online evals of the planner, and covered by 517 tests. Backtested over 8 years of data with anti-leakage controls, the predictor tied with "the price always goes up"; behind a filter requiring less than a 1% probability of chance, it now almost never makes a prediction.
Tools
- Python
- CrewAI
- Gemini
- Pydantic