Talent→Research Conference CircuitBlogNewsletterMarketplace
Resources+
AI Learning GuideEvals GuidesAI Circle Human Data LandscapeMarket maps+
Human Data for Physical AIPhysical AI AtlasMultilinguality MapCoding AtlasTrust & Safety MapExpert Data MapRL Environments AtlasEvals & Benchmarking AtlasAudio, Image & Video MapData Licensing MapLabeling & Workforces MapGuides
Practical, field-tested guides for teams building and evaluating AI systems. Written by the Head of Training at Data Gradient, published for the AI Circle community.
Volume 01, Model Evaluation5 guides
- No. 01GuideEvals 101 to ProductionDesigning an evaluation system from scratch.7 chapters
- No. 02GuideLLM-as-a-Judge HandbookRubric design, position bias, calibration, and reliability.6 chapters
- No. 03GuideBenchmark Design PlaybookDataset construction, leakage, contamination, and statistical validity.6 chapters
- No. 04GuideHuman Evaluation HandbookExpert recruitment, annotation design, agreement, and quality control.6 chapters
- No. 05DictionaryThe Evals Metrics DictionaryPass@k, Elo, win rates, confidence intervals, Krippendorff’s alpha, and more.5 terms
Volume 01, openedGuides 01 to 05
Volume 01
Model Evaluation
Guides 01 to 05, bound as a set.
Written by Head of Training at Data Gradient
Forthcoming
More guides are on the way.
Want a topic covered, or want to contribute one? Tell us.
GuidesWritten by Head of Training at Data GradientPublished for the AI Circle community
Data Gradient