Webhook Notification Delivery Delays and Missing Notifications

A postmortem has been published for the webhook notification delivery delays and missing notifications incident affecting the API.

The issue affecting webhook notification delivery has been resolved; all queued notifications were processed and delivered, and service is operating normally.

Webhook processing remains active, queue volumes reduced by 85%, but intermittent pauses are still under investigation.

Webhook processing remains active, queue volumes reduced by 70%, but intermittent pauses are still under investigation.

Webhook processing has resumed and delayed notifications are being delivered, but engineering is investigating intermittent pauses in backlog processing.

Production fix remains deployed, but backlog processing stopped again; engineering is investigating to restore delivery while preserving queued events.

Production fix remains deployed, but backlog processing stopped; engineering is investigating to safely resume queue processing.

Production fix remains in place, but backlog processing is slowed due to database load; engineering is investigating safe acceleration.

A targeted fix has been deployed, and webhook notifications are now being processed and delivered, with the queue reducing and service operating as expected.

Investigating delayed or missing webhook notifications for some customers, affecting transactions, funding events, and bank transfers.