Topic guide
8 posts
Model Evaluation: Tests and Methods
LLM evaluation notes by Jack: reproducible task tests, benchmark interpretation, cost comparisons, and methods for checking model results and their limits.
Where to start
Compare models using tasks you can check. These posts cover test design, benchmark limits, and the tradeoffs between accuracy, time, and cost.
Start here: How to build a custom LLM evaluation
All posts on this topic
-
September 2026 AI Model Rankings: Reading the Chart
Read the September 2026 AI model rankings as a dated tier chart, with its attribution, missing methodology, and limits made clear before choosing a model.
-
AI Workflows: How to Choose Models and Agents
A practical AI workflows guide for choosing models, designing agents, controlling costs, and evaluating real work without trusting benchmarks alone.
-
GPT-5.6 Sol vs. Terra vs. Luna: Which Should You Use?
Fresh CodexRadar data shows when GPT-5.6 Sol, Terra, or Luna makes sense, separating repeatable task coverage from latest-run cost and speed.
-
DeepSeek V4 Flash + GPT-5.6 Luna: A $0.42 Sol Alternative?
DeepSeek V4 Flash and GPT-5.6 Luna theoretically match Sol xhigh's coverage for about $0.42, but only if a verifier can identify the correct patch.
-
GPT-5.6 Sol Max vs. xhigh: Why Max Isn't My Default
GPT-5.6 Sol passed fewer repository tasks at max than xhigh while costing 46% more. This benchmark maps the full Codex effort curve across 112 tasks.
-
GPT-5.6 Juice Values for Sol, Terra, and Luna
Personal GPT-5.6 Juice value observations for Sol, Terra, and Luna across reasoning effort levels in Codex, plus prompts used to reproduce them.
-
How to Build a Custom LLM Evaluation You Can Trust
Build a custom LLM evaluation with hidden tests, mechanical scoring, blind review, and checks for variance, leakage, and framework confounders.
-
Claude Opus 5 vs. Claude Fable 5: Eight Tests, Same Prompts
Claude Opus 5 and Claude Fable 5 tied on most of eight tests. Fable won the decision memo, while Opus delivered the same objective score at half the price.