Designing a Retry Framework That Doesn't Make Things Worse

Every integration eventually calls something that's temporarily down. The instinct is to add a retry. The mistake is adding a different one in every flow service, tuned by whoever happened to be writing that particular integration that week.

The problem with ad-hoc retries

A retry that isn't centrally designed tends to fail in one of two directions:

• Too aggressive — tight retry loops on a struggling downstream turn a brief blip into an outage, because now every failed call is retrying immediately and stacking load on a system that was already struggling.
• Too passive — no backoff, no ceiling, so a permanently broken dependency gets hammered indefinitely instead of failing fast and alerting someone.

What the shared framework actually does

The retry framework I reuse across webMethods integrations is intentionally boring:

1. A fixed, configurable backoff curve (not a hardcoded constant per service).
2. A hard ceiling on attempts, after which the message goes to a dead-letter queue rather than looping forever.
3. Every retry attempt logged with a correlation ID, so a spike in retries shows up on a dashboard before it becomes an incident ticket.

Where this pairs with a circuit breaker

Retry alone doesn't protect the downstream system from aggregate load across many concurrent messages. That's what a circuit breaker is for — it trips after a threshold of failures and stops sending traffic altogether for a cooldown window, giving the failing system room to recover instead of getting retried into the ground by every in-flight message at once.

The two patterns are complementary, not redundant: retry handles the individual message, the circuit breaker handles the fleet.