AIBRIEF
The Y Combinator backed tool reads every session on MCP, the connector standard that lets LLMs use software, and surfaces failures hidden behind 200 OK responses — the standard HTTP success code that can mask real agent errors.
Armature, a Y Combinator-launched analytics and evals product, is one of the first tools built to watch what AI agents do when they call into apps through the Model Context Protocol (MCP), the connector standard that lets apps expose their functions to large language models.
The product, free in self-serve beta at armature.tech, reads every agent session that flows through a company's MCP server and groups them by use case. An example account display shows four use-case buckets: create and send invoices (38%), reconcile payments (22%), bulk refunds (14%), and export revenue reports (9%).
Failure detection scans every session for loops, dead ends, and root causes even when API responses return 200 OK. A sample issue list from the account display shows agent loops on a missing auth scope, search misses for the word "refund," truncated exports, and rate-limit hits on bulk updates.
Session replay scores each run for whether the user got what they asked for and exposes the full trace, including the ask, the agent's reasoning, and every tool call. PII and secrets are redacted before storage, a deliberate stance for sessions that can carry sensitive data. The evals surface auto-drafts tests from common use cases or accepts a single user-style prompt; real agents run each eval end-to-end across major models.
Co-founders Theodore and Louis introduced the tool in a YC Launches post. Pricing tiers and design partners are not yet disclosed.