Repair Bench · Performed demo
A broken webhook HMAC verifier, fixed: 3/6 → 6/6
A Stripe-style signed webhook receiver with three real-world bugs. We wrote a six-check test suite, watched it fail, fixed the verifier, and re-ran it. Below is the actual output — run the script yourself: hmac_demo.py.
The failing run
--- broken: 3/6 --- [FAIL] valid signature accepted [PASS] tampered payload rejected [PASS] wrong secret rejected [PASS] missing header rejected [FAIL] stale timestamp (replay) rejected [FAIL] constant-time comparison
The three bugs
Bug 1 — verifies the wrong bytes. The receiver parsed the JSON body and re-serialized it before checking the HMAC. Whitespace and key order differ from the raw bytes the sender signed, so legitimate webhooks were rejected. Fix: verify over the raw request body, untouched.
Bug 2 — no replay protection. The timestamp in the signature header was parsed but never range-checked, so a captured webhook could be replayed an hour later and accepted. Fix: reject anything outside a 5-minute window.
Bug 3 — timing-leaking comparison. Digests were compared with
==, which short-circuits on the first differing byte and leaks the secret to a timing attack. Fix: hmac.compare_digest.The patch
- canonical = json.dumps(json.loads(raw_body), separators=(",", ":")).encode() - expected = hmac.new(SECRET, f"{ts}.".encode() + canonical, hashlib.sha256).hexdigest() - return expected == sig + if abs(now - ts) > TOLERANCE_S: + return False + expected = hmac.new(SECRET, f"{ts}.".encode() + raw_body, hashlib.sha256).hexdigest() + return hmac.compare_digest(expected, sig)
The passing run
--- fixed: 6/6 --- [PASS] valid signature accepted [PASS] tampered payload rejected [PASS] wrong secret rejected [PASS] missing header rejected [PASS] stale timestamp (replay) rejected [PASS] constant-time comparison
This is the class of failure the Repair Bench takes on: one failing webhook or callback, reproduced in 24 hours or a full refund. See the offer.