Without idempotency, every retry is a potential duplicate order.
That’s not a theoretical concern. That’s a Monday morning support ticket from an enterprise customer who got charged twice for a €20,000 B2B order — and a customer success team that spent three hours figuring out why.
If your Go middleware has retries but no idempotency, you haven’t solved reliability. You’ve added a time bomb.
Why Retries Alone Are Not Enough
Every distributed system retries. Network timeouts happen. The payment gateway returns a 503. The ERP times out mid-sync. Your Go worker restarts mid-flight.
The question is not whether you retry. The question is whether you can retry safely.
Without idempotency:
- A timeout on a payment capture means: did it succeed before the timeout, or not?
- An ERP sync retry might create a second purchase order for the same Magento order
- A shipment notification retry might create two shipment records in the carrier system
- A pricing sync retry might double-apply a discount rule
The problem isn’t the retry. The problem is that the operation on the other end isn’t designed to recognize it has already processed this exact request.
What Idempotency Actually Means
An operation is idempotent if calling it N times produces the same result as calling it once.
HTTP GET is idempotent. Reading doesn’t change state. HTTP DELETE is typically idempotent — deleting something twice still leaves it deleted. HTTP POST is not idempotent by default — calling it twice creates two resources.
In commerce systems, most operations are write operations. Order creation. Payment capture. ERP sync. Shipment registration. None of them are idempotent by default.
To make them idempotent, you need two things:
- An idempotency key — a unique identifier for this specific operation attempt
- A dedupe store — a record of which keys have already been processed, and what the result was
The client sends a key with the request. The server checks if it has seen this key before. If yes, it returns the previous result. If no, it processes the request and stores the result against the key before returning.
Simple concept. The complexity is in the execution.
Idempotency Keys in Practice
Generating the Key
The key must be stable across retries but unique per logical operation. In a Go middleware layer, this typically means hashing a combination of:
- The Magento order ID or increment ID
- The operation type (
payment_capture,erp_sync,shipment_notify) - A version or attempt generation marker if needed
key = sha256(order_id + ":" + operation + ":" + generation)Don’t use random UUIDs unless you generate them once and persist them before the first attempt. A UUID generated fresh on each retry defeats the purpose entirely.
Storing the Result
Your dedupe store needs to hold:
- The idempotency key
- The response status and body (or enough to reconstruct the right answer)
- A timestamp for TTL and expiry
- A processing-in-progress flag — explained below
Redis is the natural choice. It’s fast, supports TTL natively, and handles distributed environments well. A PostgreSQL table with an index on (key, created_at) also works for lower-throughput scenarios where you want durability and audit history.
The In-Flight Problem
There’s a subtle race condition that trips up most first implementations. Two retry attempts arrive simultaneously before the first has finished processing. Without a lock, both see “not yet processed” and both proceed.
The fix: use an atomic compare-and-swap. When a request arrives with a key, immediately write a processing record. Only the goroutine that successfully writes that record proceeds. The second goroutine waits or returns a 409 Conflict. Once the first goroutine finishes, it updates the record to completed with the result stored.
In Go, this pattern is clean with Redis SETNX (set if not exists) or a database advisory lock. The key point: the lock must be acquired before you call the downstream system.
Real E-commerce Examples
Payment Capture
You call Stripe or Adyen to capture a payment for order #100042. The call returns a 504 timeout after 28 seconds. Did it succeed?
Without idempotency: you have to guess. Or check the payment gateway dashboard manually. Or worse — you retry the capture and charge the customer twice.
With idempotency: you retry with the same key. The payment gateway recognizes the key and returns the already-captured result. No double charge.
Most payment APIs support this natively — Stripe’s Idempotency-Key header, Adyen’s reference field. Your Go middleware should always pass one, and that key must come from your order state, not from a freshly generated UUID per attempt.
ERP Order Sync
Order #100042 is synced to SAP. The sync sends the order payload and waits for the SAP order number back. The connection drops mid-response. Did SAP create the order?
Without idempotency: you can’t safely retry. If SAP already created the order, a second attempt creates a duplicate. You now have two open orders for the same customer request. Finance will find out eventually.
With idempotency: your middleware sends the same stable key on retry. Your Go middleware checks the dedupe store — if the previous attempt completed successfully but you lost the response, return the stored SAP order number. If the previous attempt actually failed before SAP processed it, retry for real.
Shipment Notification
Your carrier API receives a notification that order #100042 is packed and ready for pickup. The API returns a 429 rate limit. You back off and retry.
Without idempotency: the carrier creates two pick-up records. The driver gets confused. The customer gets two tracking numbers. Support gets involved.
With idempotency: you retry with the same shipment reference. If the carrier API supports it, it returns the existing record. If it doesn’t, your middleware checks its own dedupe store before even sending the request.
The lesson: not all external APIs support idempotency natively. When they don’t, your middleware implements it defensively — check before you write.
Anti-Pattern: “We’ll Just Retry”
This is the most common failure mode I see in integration middleware built on Magento projects.
The team knows retries are necessary. They implement exponential backoff. They add a dead-letter queue. They feel good about the reliability story. Then they ship without the idempotency layer.
The result: a system that retries correctly and creates duplicate state every time it does.
Signs you’re living in this pattern:
- Your ERP periodically shows duplicate orders that “shouldn’t exist”
- Customer support regularly handles duplicate payment complaints
- Your DLQ is a mystery — you’re afraid to replay messages because something might double-process
- Your team says “we need to check manually” when an integration fails mid-flight
That last point is the most expensive. A dead-letter queue is only useful if you can safely replay messages from it. Without idempotency, replaying from your DLQ is risky. With idempotency, it’s a mechanical, safe recovery step — a CLI flag, not a production incident.
The Magento Side of This
Magento’s role in this pattern is clear: it’s the source of truth for orders, state transitions, and integration triggers.
What Magento should do:
- Emit reliable, stable order identifiers (increment IDs) that anchor idempotency keys
- Publish events or export payloads for Go workers to consume
- Trust that downstream systems handle deduplication — Magento doesn’t track payment provider internal state
What Magento should not do:
- Implement idempotency logic for external integrations inside PHP
- Manage dedupe state in Magento database tables (this doesn’t scale and creates coupling)
- Retry payment captures or ERP syncs via Magento cron jobs
The cron-based retry pattern is the trap. Your Magento cron retries a failed payment sync. It has no idempotency key. It has no response caching. Every retry is a fresh attempt with full duplicate risk.
Put that logic in a Go service. The Go service owns the retry loop, the idempotency store, and the response cache. Magento emits the event once and forgets it.
Decision Framework
Implement idempotency when:
- Any integration involves write operations to external systems (payments, ERP, carriers)
- Your retry strategy includes more than one attempt
- You have a DLQ and need safe replay capability
- The downstream API doesn’t natively support idempotency keys
You can simplify when:
- The operation is truly read-only (pulling order status from ERP, not pushing)
- The downstream system is idempotent by design and you control both sides
- The cost of a duplicate is zero (logging, analytics — though even here, prefer idempotency)
TTL guidance: Don’t hold idempotency keys forever. 24–72 hours covers retry windows for most commerce operations. After that, a genuine re-sync or new order creation is a different logical operation and gets a new key.
Leadership Angle
Idempotency is invisible when it works and catastrophic when it’s missing.
The business cost of duplicate orders in B2B is not just a refund. It’s an ERP reconciliation issue. It’s a legal and contractual issue if it’s a purchase order. It’s a logistics disruption if shipments get triggered twice. And it’s a trust issue with the customer — one that takes months to repair.
From a team velocity perspective: systems with proper idempotency are dramatically easier to operate. Oncall engineers can replay DLQ messages without fear. Integration failures become recoverable incidents, not data integrity incidents requiring manual intervention.
The cost of building idempotency into your Go middleware layer is one solid day of engineering work. The cost of not doing it is a production incident that escalates to the CTO at 11pm.
This is the kind of architectural decision that separates teams that operate distributed systems from teams that are operated by them.
What to Build
If you’re adding idempotency to existing Go middleware:
- Define your key schema — order ID + operation + generation, hashed to fixed length
- Choose your store — Redis with TTL for most cases; PostgreSQL if you need durability and auditability alongside the data
- Implement the atomic check-then-process pattern — write “processing” state before executing the downstream call
- Store the full response — not just “did it succeed”, but the actual result you’d return on duplicate requests
- Build replay tooling — safe DLQ replay should be a CLI command, not a manual process
The go-m2rest client library handles Magento REST API calls with auto-retry built in. Layer idempotency on top of that and you have a reliable integration foundation that you can actually trust under load.
Conclusion
Retries are not a reliability strategy on their own. They’re half a solution.
The other half is making sure that when a retry fires, the system recognizes it and doesn’t do double the work.
Idempotency is not a nice-to-have in commerce middleware. Every payment capture, every ERP sync, every carrier notification is a write operation that can be interrupted mid-flight. You need a way to recover from that interruption without creating side effects.
Build the idempotency layer. Make your DLQ safe to replay. Then go home at 5pm instead of fielding Monday morning support tickets.
