Benchmark · run 15 Nov 2026 · reproducible
How we scored the skill, and what the score does not prove
A 50-check rubric in eight dimensions, built from FAIM's own SEO, blog, conversion and website briefs. The script is in the download, so you can run it yourself.
Total coverage score
By dimension
Limits, in plain words
- It is circular. We wrote the rubric and v2 against the same briefs. 100 means v2 covers FAIM's requirements, not that it is the best SEO skill anywhere.
- It checks instructions, not results. A skill can contain the right words and still produce weak work.
- Four candidates, all ours. There is no comparative evidence for a "#1" claim, so we make none.
- Cost. v2 is about 6.5 times longer than v1. References are split so Claude reads only what a task needs.
Run it yourself
python3 scripts/benchmark.py SKILL.md references
Next we will run three real tasks with v1 and v2 as separate fresh agents and grade them blind. That is the result that could show v2 performs better. The task cases are in the download.
BreakdownStep by step
- Know what the benchmark measures
A 50-check rubric across eight areas: whether the skill's instructions cover evidence, priority, research, technical, local, AI search, content and governance.
- Know what it does not measure
It does not measure rankings or the quality of a finished page. Those need live tests on real sites.
- Run it yourself
The script is in the download. Run it on any skill folder and read the failed checks.
Worked example
Why we will not say "number one"
The rubric shows our package covers all 50 checks it was written to test. That is not a comparison with other SEO skills, so we do not claim to be the best. A fair comparison would run the same tasks through several skills and have independent people grade the output.
TipsWhat works in practice
AvoidCommon mistakes
- Treating a rubric you wrote as proof of quality.
- Comparing scores across different rubrics.
Quest · Judge badge
Honest scores
What would justify a "number one" claim?
Why: Only comparison on the same tasks, graded independently, could support a ranking claim.