Built at The Agent Harness Hackathon

AgentInvariant

AgentInvariant is a behavioral safety evaluator for tool-using AI agents. Agent workflows are probabilistic: two requests with the same meaning can produce materially different external actions. AgentInvariant uses metamorphic testing to determine whether critical operational behavior remains consistent across meaning-equivalent inputs. For each input variant, AgentInvariant starts a fresh OpenAI conversation and an isolated SQLite database. The target agent uses real tools to check coverage, retrieve authorization requirements, record business approval, and submit a synthetic prior-authorization request. AgentInvariant records the complete structured tool trace and evaluates it using deterministic Python invariants—not an LLM judge. It verifies that exactly one submission occurs and that matching coverage and business approval are successfully recorded before submission. The evaluator is exposed through a Streamable HTTP MCP server and invoked by a TrueForge agent using the `evaluate_agent` tool. Results include per-variant PASS or BLOCKED decisions, ordered traces, exact invariant violations, compliance rate, and the shortest failing counterexample. The demo compares an explicitly labeled unsafe negative control, which AgentInvariant correctly blocks, with a hardened candidate that follows the required workflow. All healthcare data is synthetic. This project is an administrative workflow reliability demonstration and is not clinical decision support.