Retry · endpoint outages · resending

If your endpoint does not answer 2xx, we resend on the fixed schedule below. The numbers on this page are the only ones in the system — there is no other set.

Retry schedule

attempt	delay since previous	cumulative since event creation
1	immediate	00:00:00
2	10 seconds	00:00:10
3	1 minute	00:01:10
4	5 minutes	00:06:10
5	30 minutes	00:36:10
6	2 hours	02:36:10
7	6 hours	08:36:10
8	12 hours	20:36:10
9	24 hours	44:36:10
10	24 hours	68:36:10
11	24 hours	92:36:10

• 11 attempts in total · a window of roughly 92 hours 36 minutes (3 days 20 hours)
• ±10% jitter per attempt ⇒ this table is the target. The real time for a given delivery is in next_retry_at in the dashboard.
• The X-PG-Attempt header tells you which attempt this is. Delivery metadata is not part of the event body.

Why the inbox is retained 31 days although retries last only ~92 hours

Manual resend remains available until day 30 after event creation. Retain pg_webhook_inbox for at least 31 days so the final-day resend still hits its dedupe row.

Attempts 9–11 are a day apart

If your endpoint is down for 2 hours and then recovers, the backlog does not arrive immediately — it waits for the next scheduled attempt.

Do not just wait. Use bulk replay or reconcile with GET /v1/pay/events to pull the backlog yourself.

What counts as success

criterion	value
Success	HTTP 200 201 202 204 only
connection timeout	5 seconds
timeout to the first response byte	10 seconds
total request timeout	15 seconds (aim to answer within 3 seconds)
your response body	We read up to 8 KB into our logs and close the connection · content has no effect on success.
our request body	≤ 256 KB (guaranteed)
3xx	🔴 We do not follow redirects = failure (blocks SSRF and keeps the signature off other hosts).
410 Gone	Permanent failure → retries stop immediately + the endpoint_gone flag is set.
429	Failure → retried on the schedule · 🔴 we do not read your Retry-After. The schedule is fixed so it stays predictable.
Other 4xx/5xx · TLS/DNS/connection errors	Failure → retried on the schedule.

Retries stop when we get a 2xx · all 11 attempts are used · we get a 410 · you delete or disable the endpoint (pending deliveries are canceled) · the system disables the endpoint automatically.

A delivery has exactly four possible states: pending · delivered · failed · canceled.

Answer in 3 seconds, not 15

15 seconds is the deadline where we cut you off, not a budget to spend.

A handler that takes 12 seconds starts timing out the moment traffic ticks up, and then the loop begins: timeout → retry → duplicate work → slower → timeout again.

The correct shape: verify → one transaction inserts/dedupes the inbox row together with an outbox row → commit → answer 200. The outbox relay continues later.

Do not answer 500 when signature verification fails

500 = failure = we retry all 11 times. Multiply that by every event that fails verification and you have DDoSed yourself.

Bad signature → 401 · timestamp outside the window → 400 · duplicate event → 🔴 200.

Sustained endpoint outages

6 h of continuous failure    → warning email + a banner in the dashboard (retries continue as normal)
12 h of continuous failure   → second warning email + a notice to the provider that manages your account
24 h of continuous failure   → 🔴 the endpoint is disabled automatically

Conditions for automatic disabling (both must hold): in the last 24 hours, successes = 0 and failed deliveries ≥ 50.

What happens when it is disabled

• endpoint.status = 'disabled_by_system'
• No new deliveries are created — but events are still retained for 30 days
• Pending deliveries are canceled
• webhook.endpoint.disabled is sent to your other healthy endpoint group + an email + an audit record
• 🔴 Re-enabling is a manual action in the dashboard — there is a Replay failed deliveries (last 72 h) button to press right afterwards.

A flaky endpoint is not disabled

One success in 24 hours is enough to stay enabled ⇒ an endpoint failing 90% of the time survives (though it does get warned).

The threshold is set this way because disabling a merchant's endpoint just for being slow is more dangerous than letting the retries continue.

If you have only one endpoint and it gets disabled

The events you cannot unsubscribe from (deposit.reversed · withdrawal.fill_reversed · balance.debited · webhook.key.rotating · webhook.endpoint.disabled) are sent by email to your technical contact, with a persistent red banner in the dashboard.

⇒ Set your technical contact to a mailbox a human actually reads, not a noreply@ address · and configure at least one backup endpoint group.

Resending manually

Resend a single delivery

curl -X POST \
  "https://api.example-wl.com/v1/bo/webhooks/whep_01K2A9F3TNXR7VB2HD0MKQ5CJW/deliveries/whd_01K3M8QX5DHN2VRT0BKZQ7W4YM/resend" \
  -H "Authorization: Bearer $BO_OIDC_ACCESS_TOKEN" \
  -H "X-PG-Step-Up-Token: $BO_STEP_UP_TOKEN" \
  -H "Idempotency-Key: resend-whd_01K3M8QX-1"

Resend in bulk (replay)

curl -X POST \
  "https://api.example-wl.com/v1/bo/webhooks/whep_01K2A9F3TNXR7VB2HD0MKQ5CJW/replay" \
  -H "Authorization: Bearer $BO_OIDC_ACCESS_TOKEN" \
  -H "X-PG-Step-Up-Token: $BO_STEP_UP_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: replay-2026-08-18-to-21" \
  -d '{
    "event_types": ["deposit.settled", "deposit.reversed"],
    "created_after": "2026-08-18T00:00:00.000Z",
    "created_before": "2026-08-21T00:00:00.000Z",
    "only_failed": true
  }'

Resend/replay are /v1/bo/* operations, so they use the operator's OIDC access token plus a step-up token, not sk_live_* or PG HMAC.

Rules for resends

topic	behaviour
event.id	🔴 Exactly the same ⇒ your dedupe still works — replay a whole week and nothing is credited twice, as long as you dedupe correctly.
Body bytes	Byte-for-byte identical — we stored the payload at creation time and never re-render it ⇒ an api_version change in the meantime cannot distort the payload.
Signature	Recomputed (t must be fresh) and signed with the key that is active right now.
Delivery metadata	Headers only: a new X-PG-Delivery-Id, X-PG-Attempt: 1, and X-PG-Resend: manual. The event body does not change.
Retries	A fresh set of 11 attempts.
Bulk	10,000 events per command, released at 50 deliveries/second · every command is audited.
Age	Resendable for 30 days from the event's created_at.
Rate limit	Bucket webhook_resend — 10 commands/minute per endpoint (commands, not deliveries) ⇒ one replay command covering 10,000 events counts as 1.

A 10,000-event replay means 50 deliveries/second at your endpoint

Before you press it, make sure your handler answers fast (< 3 s) and your dedupe works. Otherwise you create a new incident while recovering from the old one.

When an event really is lost

All 11 attempts exhausted without success → webhook_deliveries.status = 'failed'. The event stays in the system for 30 days and can be resent.

But after a long outage, the better move is to pull rather than wait:

# fetch every event since the last sequence you saw
curl -s "https://api.example-wl.com/v1/pay/events?since_sequence=88214&limit=100" \
  -H "Authorization: Bearer sk_live_xxx" \
  -H "PG-Signature: ..." -H "PG-Timestamp: ..." -H "PG-Nonce: ..." -H "PG-Version: 2026-08-01"

• 30 days of history
• since_sequence is the only cursor on this endpoint (it does not use the cursor style of other endpoints)
• Feed the results straight into your existing handler — dedupe and the version guard handle the duplicates

🔴 Webhooks are not the source of truth — the ledger is the truth, webhooks are notifications. A daily reconcile job is a hard requirement before go-live, not an option.

What to monitor

alert when	because
Signature verification failures > 0	A key rotation may be in progress, or someone is sending forgeries.
Handler returns non-2xx for more than 1% over 5 minutes	It is heading for the 24-hour automatic-disable threshold.
X-PG-Attempt > 3 unusually often	The endpoint is systematically slow or timing out.
A sequence gap open for more than 4 days	Genuinely lost (short-lived gaps are normal — see ordering).

Log these on every delivery: event.id + X-PG-Delivery-Id + X-PG-Attempt. Delivery headers are for observability only and must never drive business logic.

Next

read next	when
Order is not guaranteed — version guard	🔴 A 92-hour retry window means an old event can always arrive later — read this page.
Verify the Ed25519 signature	Verification still fails and the retries are burying you.
Testing before production	You want to take the endpoint down deliberately and watch what happens.
Webhook overview	You need the endpoint requirements and the envelope again.
Go-live checklist	Production is close.