Read evaluation results
View evaluation results, compare scores and costs, and understand variation and missing data.
An evaluation shows how well an agent completed your tasks and, where available, how much time and money it spent. Start with the summary, then check individual scenarios for failures or differences that the average may hide.
If you have not run an evaluation yet, see Prove a skill works using evaluation.
View a run
Open your most recent run or choose one by ID:
$ tessl eval view --last
$ tessl eval view <run-id>The output includes a comparison of the test variants, followed by a table of results for each variant broken down by scenario.
This example is real output from a run that compared four models on the same plugin, with three repeats each:
Eval Results: tile-analysis — model comparison
Repo: tesslio/monorepo
Agent: claude
Model: claude-sonnet-4-6
Variants: Opus 5, Sonnet 4 6, Haiku 4 5, Deepseek V4 Flash
Status: ✔ Completed
Test variant comparison
Test variant Score Cost / solve Duration Tokens / solve Turns / solve
opus-5 100% ± 0.0pp $0.9835 ± $0.1550 183.3s ± 17.5s 730,012 26.67 ± 1.63
sonnet-4-6 98% ± 2.8pp $0.3698 ± $0.0935 126.6s ± 27.0s 533,055 24 ± 3.02
haiku-4-5 92% ± 7.9pp $0.1258 ± $0.0635 50.3s ± 42.2s 451,771 15.22 ± 13.45
deepseek-v4-flash 98% ± 2.7pp $1.3125 ± $0.3342 245.8s ± 84.0s 801,544 43 ± 9.94Score, cost, duration, and turns show a mean (average) followed by a standard deviation (SD), which describes how much the measurements vary. For example, 92% ± 7.9pp means an average score of 92% and an SD of 7.9 percentage points. It is not a guaranteed range for future scores.
The score mean is truncated to a whole number in this view: 84.7 appears as 84%. Tokens appear as a rounded mean without an SD. When fewer repeats have results than expected, a cell includes a count such as (n=2 of 3).
View a run in the web app
tessl eval view prints a link to the run. The page has these sections, from top to bottom:
Result: one sentence that states the outcome, for example which variant scored highest.
Metrics: score, cost per solve, time per solve, and tokens per solve for each variant. Each value is a mean across repeats, drawn with its spread.
Did the agent actually use the skill?: the share of repeats in which the agent loaded the skill, for each variant that has it. See How activation is measured.
By scenario: one row for each scenario. Select a row to open its criteria and its repeats. Each repeat shows its score, cost, time, tokens, and whether the skill activated, and members of the workspace can download the solution.
Setup: what each variant did differently and what stayed the same, the total cost of the run, and how many attempts were reused from earlier runs and not billed again.
When a difference is within noise
The page treats two means as different only when the gap between them is larger than their pooled standard deviation, the square root of the sum of the two variances. A grey band on each metric shows that threshold. In a default run, the band sits around the result without the skill, and a change inside it reads Unchanged. In a run with your own variants, the band sits around the best variant, and variants that trail it by more than the band are faded.
A difference inside the band is not proof that the variants are equivalent. Add repeats to narrow the spread.
Activation states
A percentage, such as
62%, is the share of measured repeats in which the agent loaded the skill.Not measured means the agent does not report activation. It does not count as a zero.
Not loaded on a repeat means the skill was not installed for that variant, or it was installed and the agent did not load it.
Understand what is being counted
A run contains test variants, scenarios, and repeats:
Every variant attempts every scenario. The rubric defines what a successful response must do, and the judge scores the response against it. -n, or --runs, sets how many times each variant attempts each scenario.
Each attempt is a solution, whether or not scoring completes. The number of attempts is:
For example, four models on three scenarios with three repeats create 36 attempts. This is also the number counted toward the daily quota.
A default run has two test variants: without the plugin and with it. A run declared with --variant-json has the variants you list. Each can change the model, agent, plugin, or other fixtures.
What each measurement means
Score
Points awarded as a percentage of the rubric's maximum
Cost
Cost of solving the task, in dollars, where recorded
Duration
Wall-clock duration and the agent's own time, recorded separately
Tokens
Input, output, cache read, and cache creation token counts
Turns
Number of agent turns used to solve the task
Skill activation
Names of skills the agent invoked, where supported
The judge's time, cost, and tokens are recorded separately. The cost in the variant comparison table covers solving the task; it does not include scoring it.
Numeric measurements store a count, missing count, minimum, quartiles, median, mean, maximum, and standard deviation. Activation is a separate record of skill names. See How activation is measured for its limitations.
Inspect variation within a result
Use --full to see more than the average:
This expands the per-scenario cells in each variant's table. The top-level comparison of variants stays compact.
An expanded score cell from the run above, for one scenario solved three times by haiku-4-5, looks like this:
min and max are the lowest and highest values. med is the median, or middle value. q1 and q3 mark the lower and upper quartiles, with the middle half of the results between them. sd is the standard deviation.
Use these statistics to decide which individual attempts to inspect. A low minimum may point to a failed attempt, while a wide range may reflect differences between tasks or between repeats of the same task. The summary alone does not explain the cause.
Avoid ranking variants by the average alone
The model comparison in What you can compare with an evaluation had these unrounded scores:
A cheaper hosted model
85.0
15.8
A frontier model
84.7
20.0
A mid-tier model
73.0
34.6
A small fast model
72.9
21.8
The means differ by about 12 points from highest to lowest, while the standard deviations range from about 16 to 35. With that much variation, the averages alone are a weak basis for choosing a model.
For example, the mid-tier model's individual scores ranged from 0 to 100. Its average of 73.0 hides that variation. Look at the per-scenario results to see which tasks failed and whether the failures recur across repeats.
Start with at least three repeats when you intend to share a conclusion. Add more when the result is variable or the decision needs greater confidence. Comparing standard deviations is useful for spotting uncertainty, but it does not replace a statistical test of the difference.
Check cost and time alongside quality
That run also shows why the highest score may not be the best choice for your workload:
A cheaper hosted model
85.0
$1.572
313s
61.3
A frontier model
84.7
$0.940
150s
17.4
A small fast model
72.9
$0.107
43s
10.3
The small fast model scored lower, but used about 11% of the frontier model's cost and less than a third of its time. It may be suitable for tasks where that quality is enough.
The cheaper hosted model had a lower list price than the frontier model, yet cost 67% more per completed task and took about twice as long. Its higher turn count helps explain why list price alone is not enough to estimate the cost of a task.
Measure the improvement over a baseline
To measure how much a skill helps, include a variant without it:
Setting the context fixture to null removes the plugin under test. Other fixtures, such as a codebase, remain available. This gives you a baseline for the same tasks without the plugin you are testing.
In one evaluation, five skill rewrites scored between 95 and 98, but the no-skill variant already scored 90.2. The measured improvements were therefore between 4.8 and 7.8 points. Those scenarios left little room to distinguish successful rewrites because the agent could already do most of the work without the skill.
On a different skill, the shipped version scored 86.4 against a baseline of 39.4. The best rewrite scored 97.2. The rewrite improved on the shipped version by 10.8 points; the baseline showed that both versions also helped substantially compared with having no skill.
A baseline provides context for a skill's contribution. You do not need it merely to calculate the difference between two versions.
Compare separate runs
Use compare to put two to ten runs side by side:
The output has one column per run and shows the configuration dimensions that differ between them. Check those differences before attributing a score change to your edit.
The compact statistical cells in compare and view both show the mean with the standard deviation.
With compare --full, every statistical cell in the comparison table expands to show the full distribution. With view --full, only the per-scenario tables expand.
Confirm results with another run
The seven examples in What you can compare with an evaluation were each run twice with the same configuration. Two conclusions changed on the second run: the order of the ablation variants reversed, and the improvement from a rewrite changed by an order of magnitude.
Repeats help reveal variation within a run. A separate run checks whether your conclusion holds again. If it does not, investigate the individual attempts and the rubric before making a change based on the first result.
Handle missing data
A missing measurement is different from a measured zero. Missing cells display a dash, and partially missing results include a count such as n=1 missing. Check these counts before comparing averages.
Activation
Only the claude agent reports activation. For other agents it is recorded as not measured. This does not mean the skill failed to activate.
Cost
Some agents do not use the billing path that records per-solution cost. An empty cell does not mean the task was free.
Cost or judge statistics on a shared run
These fields are visible only to members of the workspace that owns the run, even when the run is public. Check workspace access before assuming data was not recorded.
Score
A solution may complete without receiving a score. Treat it as missing data, not a zero or a no-skill baseline.
Per-turn or tool-usage statistics
These require an agent log. Attempts without one are reported as unavailable.
Run limits
Attempts per run (variants × scenarios × repeats)
2,000
Repeats (-n, --runs)
1 to 10
localPath fixture references per run
50
Distinct local sources
20
Scenarios per run
More than 50 requires --yes
Attempts per organization per day
1,000, resetting at 00:00 UTC
The daily cap counts attempts, not runs. Four variants on three scenarios with three repeats use 36 of the 1,000 available attempts.
Next steps
What you can compare with an evaluation gives examples of model, agent, and skill comparisons.
Prove a skill works using evaluation covers generating scenarios and running evaluations.
Troubleshooting reviews and evaluations helps with failed runs and unexpected scores.
Check a skill's quality using review explains how to review a skill before evaluating it.
Last updated