Answers from our docs and articlesesc to close
Try asking
Create an accountSign up

Seeded bug of the week: a charge retry that bills twice

A retry loop around a charge drops the idempotency key, so a timeout after success bills twice. Why it passes review, and how Auditor reported it.

Key takeaways

  • Adding a retry around a payment call is only safe if every attempt sends the same idempotency key; this change dropped it.
  • The bug is easy to miss because the key vanishes inside a rewritten line, and the rule that requires it lives in a file the diff doesn't touch.
  • In our Oct 11, 2026 eval run, both Standard and Thorough reviews reported it as High, with the double charge spelled out.
  • The case is seeded and small, and the answer key is ours, so treat it as an example, not a measure of every review.

Seeded bug of the week is a short series. Each one takes a case from the eval set we use to tune Auditor's deep review: a small pull request with one bug planted in it. We show the diff, why it gets past a reader, and what Auditor said about it.

This week's case is called data-nonidempotent-retry. The pull request is titled "Retry flaky charges".

The change

The repository has a payments client whose contract is written in a comment:

export interface Payments {
  /** Creates a charge. Safe to retry only with the same idempotencyKey. */
  createCharge(input: { customer: string; cents: number; idempotencyKey?: string }): Promise<{ id: string }>;
}

Before the pull request, checkout made one call, keyed by the order:

export async function checkout(payments: Payments, orderId: string, customer: string, cents: number) {
  return payments.createCharge({ customer, cents, idempotencyKey: `order:${orderId}` });
}

After it, checkout retries up to three times:

export async function checkout(payments: Payments, orderId: string, customer: string, cents: number) {
  for (let attempt = 0; ; attempt++) {
    try {
      return await payments.createCharge({ customer, cents });
    } catch (err) {
      if (attempt >= 2) throw err;
    }
  }
}

Why it looks fine

The pull request does what its title says, and the loop is the standard shape. A reader checks the attempt count, the await and the rethrow on the last try, and all three are right.

What's missing is on the line that was rewritten. In the diff, one line goes and seven come in, and the call that went in no longer has idempotencyKey. The rule that makes it matter is in client.ts, which the pull request doesn't touch. The key is optional in the type, so it still compiles, and orderId going unused raises no error under default TypeScript settings. A test with a mock that fails twice and then succeeds passes.

How it fails

A timeout doesn't mean the charge failed. Say the first call reaches the payment provider and succeeds, but the response is lost and the call throws. The loop sends the same customer and amount again, now with no key, so the provider can't tell it's a retry. The customer is charged twice for one order, or three times if the second response is lost too.

The fix

Build the key once, outside the loop, and send it on every attempt:

export async function checkout(payments: Payments, orderId: string, customer: string, cents: number) {
  const idempotencyKey = `order:${orderId}`;
  for (let attempt = 0; ; attempt++) {
    try {
      return await payments.createCharge({ customer, cents, idempotencyKey });
    } catch (err) {
      if (attempt >= 2) throw err;
    }
  }
}

A better version would also retry only errors that can be transient, with a backoff between attempts. A declined card won't succeed on a second try.

What Auditor reported

We ran the eval set on Oct 11, 2026 at both review depths, with Claude Sonnet 5.5 finding and Claude Opus 5.5 verifying. Both depths put a High finding on the createCharge line.

The Thorough review titled it "Charge retries dropped the idempotency key". Its passes can read the whole repository, so it quoted the contract from client.ts: a charge "is safe to retry only with the same key". Its failure scenario was the one above, a call that succeeds and then times out, charging the customer "twice (or three times) for one order". The suggested fix put the key back on the call. Three of its four passes raised it, and the verifier confirmed it.

The Standard review, which reads the diff and the changed files, found the same bug without seeing client.ts, and noted that orderId was now unused. It also posted a second, Medium finding: the loop retries every error, including declines, with no backoff. Our answer key counted that as a false positive, because it isn't the seeded bug. It's the same point as the last paragraph of the fix, though, so we'd read it rather than dismiss it.

On this case, Standard cost $0.031 and Thorough $0.069.

The caveats

The bug was planted by us, in a two-file repository, and the answer key is ours. The eval measures the deep review only, not the rules pass or static analysis. It's also small: 22 cases, of which 14 are small TypeScript repositories with one seeded bug each, three are built from real Shopify theme files with eight seeded bugs each, and five are clean changes that should get no finding. Across all of them, that run's Standard review found 34 of 38 seeded bugs with one false positive (the one above), and Thorough found all 38 with none. A real repository is bigger and messier, and these numbers don't promise the same there.

Frequently asked questions

What is an idempotency key?

A value sent with a request so the server can recognize a repeat of it. Sent with the same key, a retried charge returns the first charge instead of creating a second one.

Why not just remove the retry?

Retrying a timed-out charge is reasonable, as long as every attempt carries the same key. Without retries, a lost response leaves the order unpaid in your system even though the charge went through.

What's the difference between Standard and Thorough?

Standard runs three passes over the diff and the changed files. Thorough, the default, runs four specialized passes that can read and search the whole repository, and costs more per review. Both have a verifier check each finding before it is posted.

One bill for your apps, your models and your reviews.

Host apps and functions next to your model traffic. Routing picks each turn's model by your quality floor, and Auditor reviews every pull request against your own rules.