FixLedger HomeAttach MCP
OPEN-SOURCE REPRODUCIBLE BENCHMARK

The AI Coding Tax Benchmark

We ran 25 real post-cutoff compiler, SDK, and protocol breaking changes across Next.js 15, Tailwind v4, MCP Streamable HTTP, x402 EIP-3009, Vercel AI SDK v4, Prisma Edge, and Pydantic v2. Spending $0.125 total on FixLedger eliminated 476,464 retry tokens and cut debugging time from 34.1 minutes to 5.8 minutes.

Token Reduction
-90.4%
527,000 vs 50,536 tok
Net Cost ROI (Sonnet 4.5)
15x ROI
$0.125 fees saved $1.88 net
Wall-Clock Speedup
5.9x
34.1 min down to 5.8 min
Single-Session Pass Rate
100%
vs 72% with blind web search

Cumulative Token & Dollar Burn Across 25 Bugs

Reproduce locally: npm run benchmark
Control Group (Frontier Agent + Free Web Search)527,000 tokens ($2.242)
Test Group (Frontier Agent + FixLedger MCP $0.005 Unlocks)50,536 tokens ($0.363 all-in)

All 25 Post-Cutoff Bug Scenarios

Mean peek latency: 0.84ms
IDStackScenarioControl TokFixLedger TokControl USDAll-In USDNet Saved
BM-01x402Base Mainnet USDC EIP-3009 EIP-712 Domain Name Mismatch29,5002,019$0.126$0.015$0.111
BM-02x402EIP-3009 bytes32 Random Nonce vs Sequential uint256 Permit Nonce22,1001,988$0.094$0.014$0.080
BM-03x402Base USDC 6-Decimal Atomic Units vs 18-Decimal parseEther Overflow17,0501,985$0.073$0.014$0.059
BM-04x402Cross-Origin Browser Agent Missing Access-Control-Expose-Headers21,0002,042$0.088$0.015$0.074
BM-05x402Base L2 Sequencer Clock Skew on EIP-3009 validAfter Timestamp22,8001,998$0.097$0.015$0.083
BM-06mcpMCP Streamable HTTP Session Loss Across Stateless Serverless Isolates27,1502,037$0.117$0.015$0.102
BM-07mcpconsole.log / dotenv Banner Corrupting MCP Stdio JSON-RPC Frame24,0002,029$0.102$0.015$0.087
BM-08mcpMCP Streamable HTTP notifications/initialized Rejects id:null JSON Body21,8002,039$0.092$0.015$0.077
BM-09mcpzodToJsonSchema $ref Definitions Rejected by Strict LLM Tool Routers18,7002,009$0.079$0.015$0.064
BM-10nextjsNext.js 15 Asynchronous headers(), cookies(), and params Breaking Change16,6501,991$0.071$0.015$0.056
BM-11nextjsNext.js Server Action redirect() Caught Inside try/catch Without unstable_rethrow18,3502,080$0.079$0.015$0.064
BM-12nextjsReact 19 useActionState Migration From Removed react-dom useFormState15,8502,021$0.067$0.015$0.053
BM-13nextjsNext.js 15.5 Concurrent Dev/Build Collision on /_not-found Manifest35,0002,116$0.151$0.015$0.136
BM-14tailwindTailwind CSS v4 @tailwindcss/postcss & @import Migration20,6502,006$0.088$0.015$0.073
BM-15tailwindTailwind CSS v4 @apply Unknown Utility Inside CSS Modules Without @reference25,3001,980$0.108$0.014$0.094
BM-16ai-sdkVercel AI SDK v4 Unified tool({ parameters }) & maxSteps Migration22,5002,037$0.096$0.015$0.082
BM-17ai-sdkOpenAI Strict Structured Outputs Rejects Zod .optional() Without .nullable()18,0501,983$0.076$0.014$0.062
BM-18ai-sdkLangGraph Parallel Fan-Out InvalidUpdateError Missing Annotated Reducer20,5001,996$0.087$0.015$0.072
BM-19llm-apiRetired Pre-Cutoff LLM Model Snapshot HTTP 404 Replacement14,7001,987$0.062$0.014$0.048
BM-20llm-apiOpenAI o-Series Reasoning Models Reject temperature & max_tokens15,7502,008$0.067$0.015$0.052
BM-21llm-apiAnthropic Extended Thinking max_tokens > budget_tokens & temperature=1 Rule19,9502,026$0.084$0.015$0.070
BM-22llm-apiGoogle Unified @google/genai SDK Migration From Retired @google/generative-ai21,6502,081$0.092$0.015$0.077
BM-23stripeStripe Webhook Raw UTF-8 Body constructEventAsync in App Router17,9502,038$0.076$0.015$0.061
BM-24prismaPrisma Client on Vercel Edge Runtime Without @prisma/adapter-neon24,6502,036$0.105$0.015$0.090
BM-25fastapiPydantic v2 ConfigDict(from_attributes=True) & model_validate Migration15,4002,004$0.065$0.015$0.051