Quality Metrics
Every eval run pushes the agent through a golden dataset of research queries, scores the answers with an independent LLM judge (Gemini, a different model family than the agent's Llama), and checks whether the agent's self-critique agrees with the external judge.
Are the answer's claims grounded in retrieved context? (ragas, LLM judge)
How much of the retrieved context is actually relevant? (ragas)
Fraction of expected golden facts covered by the answer
Rubric-scored relevance, completeness, specificity, coherence
—
Pearson correlation between the agent critic's own quality score and the external judge. Higher means the agent knows when its own answers are good.
0.80
Agent critic's average confidence
2.0
Self-reflection loops per query
104s
End-to-end per research query
Per-Query Results
What is Microsoft's corporate mission?
Error: judge: Gemini free-tier daily quota exhausted (429 RESOURCE_EXHAUSTED)