WhetstoneCyberelf Labs // deployment authority live · stateless workbench · private bank absent

Cyberelf Labs // Whetstone // release decision

Before you ship an AI change, check what it broke.

The old behavior is part of the contract.

Run the current and candidate system on the same checks. Whetstone shows every gain and regression, then returns PASS, HOLD, or BLOCK with the exact path behind the decision.

Compare two agents Open the verifier workbench
No accountsNo cookiesWorkbench payloads not retainedNo private bankExact receipts
Public-demo boundary. Use disposable or sanitized inputs here. Benchmark publication is explicit and stores only the public manifest and sanitized receipt. Private exams belong inside your trust boundary.

Receipts, not slogans

The mechanisms were measured before they were wrapped.

These numbers are loaded from committed, sanitized experiment receipts. The public service never sees the private items behind them.

—loading receipt evidence

The verifier workbench

Run the mechanism. Inspect the proof.

Every sample is disposable, editable, and verified by the same deterministic core exposed through the API.

01 / 08

Whetstone Inspector

exact inputs

Drop your filesor click to choose JSON / JSONL
Request payload JSON · editable before anything is sent

Load the verified sample, replace it with sanitized data, then run.

EVIDENCE RECEIPT

Ready for a run

IDLE

Run the verified sample to see the complete decision path, not just a score.

A hard trust boundary

The public demo cannot serve what it never loaded.

The hosted process is stateless and has no examiner bank. For a real promotion decision, the CLI, MCP server, ACP candidate adapter, and localhost service keep private items inside your infrastructure.

YOUR BANKprivate items
→
WHETSTONEserver-side grading
→
YOUR CIreceipt + exit code
whetstone inspect --exam private.jsonl --exposure training.jsonl \
  --baseline v1.json --candidate v2.json
# 0 PASS · 2 HOLD · 3 BLOCK

Plain answers

Frequently asked questions

What is Whetstone?

Whetstone is a promotion gate for AI agents. It tests whether a new agent or model version genuinely improved using declared-exposure audits, paired outcomes, exact statistics, and evidence commitments, then returns PASS, HOLD, or BLOCK with auditable receipts. These controls surface known contamination; they do not prove absence from every training corpus.

How is this different from eval dashboards and observability tools?

Observability tools show what happened; Whetstone issues a release decision. Its exam items are structurally withheld from the system under exam, checked for contamination against actual training artifacts, and retired when they saturate. Every verdict ships with an item-level decision path and SHA-256 commitments.

How do I connect an AI agent?

Point any MCP client at https://whetstone.cyberelf.link/mcp (Streamable HTTP, no auth): claude mcp add --transport http whetstone https://whetstone.cyberelf.link/mcp. Or POST JSON to the REST endpoints under /api/ — GET /api/examples returns a complete working payload for every tool. Full agent documentation: /for-agents.

Can Whetstone grade my agent right now?

Yes. Open Promotion Bench gives a baseline and candidate the same six scope-integrity tasks, then counts gains and regressions. Agents can also call open_bench_start and open_bench_submit over MCP. The existing disposable graph-repair report card remains available through report_card_start.

Do you store my data?

Workbench and disposable report-card request bodies, prompts, answers, and results are not persisted. If you explicitly publish a benchmark run, Whetstone stores only the system manifests, item transitions, counts, hashes, and verdict—never the tasks or answer patches. Privacy-preserving usage metrics and standard Nginx access logs are retained.

What does it cost?

The public demo is free with no signup. The core is open source under AGPL-3.0 on GitHub; commercial licensing and design-partner pilots are available from the author.