TL;DR
The short version
Leaderboards can't tell you whether a new model is better for your work. A small set of your own tasks, run the same way every time and scored without knowing which model wrote what, can. Switch only when the gap is worth the cost of changing habits.
On The AI Daily Brief, Nathaniel Whittemore and Nufar Gaspar walk through a five-step personal benchmark: choose tasks, choose candidates, run them identically, score them blind, then decide to switch, split or stay.
Built on the 8 October Operator's Cut of The AI Daily Brief, a recording of the AIDB × Superintelligent personal-benchmark webinar led by Nufar Gaspar, with release-cadence data from ARK Invest and Artificial Analysis.
Why the leaderboard isn't your answer
New frontier models now arrive about every 11 days. That is the median gap in 2026, down from 37.5 days in 2023, and each one ships with a table of benchmark scores.
Whittemore's case against leaning on those tables has two parts. Many public benchmarks have leaked into training data, so the older a test is, the less it tells you. And the gaps that matter now are less about one model being strictly better. They are about how a model feels and how it fits the rest of your setup.
His conclusion is blunt: "There's kind of no shortcut solution to figuring out how a new model is or isn't going to fit into your process other than using it." Gaspar's method turns "just use it" into something you can repeat.
Pick tasks you would actually act on
Gaspar calls task choice the most important step. She suggests four to six tasks that you do often, that span your real work, and where a clear win would make you change tools. If AI does a lot of your personal admin, include that too.
Two selection rules stand out. First, favour tasks where failure is obvious: "If you can evaluate what bad looks like, sometimes it's a better use case than the one where you have lots of nuances on what good looks like." Second, include at least one wish-list task, something AI has let you down on before. Hers is a full weekly plan that comes out right in one go. That task is how you notice when the frontier actually moves.
Write each task as a measurable brief. "Saying create a good blog post is not measurable," she says. Word limits, required parts and things to avoid make it scoreable.
Run it small, frozen and blind
Compare your current setup against up to about three new candidates. Gaspar ran seven models across her tasks and calls that "way too much" to review.
Keep everything else fixed. Send the identical prompt to each model in a fresh chat. Hide the model names, whether through a friend, a spreadsheet shuffle or a script. She built one that runs prompts through OpenRouter, a service that reaches many model vendors with one API key, and shows the answers side by side as A to G. For high-stakes tasks, run each prompt more than once, because the same model gives different answers on different runs.
Rank side by side rather than scoring each answer alone. She finds absolute scores hard and comparisons easy. Include something visual, such as a web page, if you ever use AI that way, because taste shows up fastest there.
You are the judge. The AI judge is a second opinion.
An AI judge with a written rubric can cross-check your scores. Gaspar's caveats are sharp. Use a judge from a different model family, because models tend to favour their own family's work. That bias is documented: Panickssery and colleagues found GPT-4 and Llama 2 rate their own outputs higher even when humans rate them equal. And test the judge itself:
If the judge is completely not correlated to your taste, it's a bad judge and just throw it away.
Nufar Gaspar on The AI Daily Brief
Her Gemini judge agreed with her on just one task, pricing pushback. It also judged the website task from code alone, without seeing the rendered page.
The blind reveal is what changed her mind. GPT won her negotiation task. Grok won weekly planning. Opus 5.5 built the best course website. No model wrote a LinkedIn post she would publish as is. Fable 5.1 topped the judge's average but cost far more. In her own scores it tied with GPT-6.1 Sol, which cost roughly a tenth as much. Kimi K3 was cheapest and fastest. She is now weighing moving more of her daily work to GPT, which she did not expect.
Switch, split or stay
Gaspar ends with three choices. Switch when the new option is clearly better. Split when different tasks have different winners. Stay when the gap doesn't justify the change. Company rules, terms, plan limits, cost, speed and features all weigh in, and "habit formation is an expensive thing."
Two boundaries from the Q&A matter. For routine work like email or basic research, any decent model looks the same, and open-weight models are good enough when the surrounding tooling is good. Tasks that depend on your own setup, with its connectors and context, have to be tested inside that setup. And cost is easy to misread on a flat-rate plan, which may be hiding much higher token prices. She expects those subsidies to shrink.
Build the set once, keep it fixed, and rerun it when a release looks like it might matter.
