Idempotency: Stop Automation Creating Duplicate Work

An automation can fail by doing the right thing twice. A customer presses Submit again, a worker restarts, or a connection times out after the destination has already saved a record. Retrying is sensible for availability, but it can create duplicate invoices, tasks, messages, or accounting entries unless the workflow recognises repeated work.

Idempotency means that repeating the same intended operation does not create an additional business effect. It does not mean every repeated request is ignored. A genuine amendment must still be processed, and two different orders with identical amounts must remain separate.

Identify the business operation

Start by describing the action in plain language: create the accounting handoff for order 812 in company A. That is more useful than "run integration job 504." The job number may change on retry, while the business operation remains the same.

Build the operation key from stable identifiers and a documented scope. Company, source system, document identity, action type, and revision may all matter. Do not use only the customer's email or the invoice amount. Neither uniquely identifies a transaction.

Decide how long the key must remain meaningful. A short-lived cache may stop rapid double-clicks but fail when someone reimports last month's file. If repeat processing is possible over months, the operation record needs a retention policy appropriate to that workflow.

Store an outcome, not just a flag

A boolean called processed cannot explain whether a destination record exists or whether a request is still uncertain. Use explicit states and keep the destination reference when available. The following is a proposed application design, not a universal API contract.

Store a fingerprint of the approved payload. If the same key arrives with different amounts, stop and explain the conflict. Quietly returning an old success response for a changed invoice is just another form of data loss.

Handle simultaneous requests

Checking whether a record exists and then creating it in two separate steps is not sufficient when two workers can run together. Both may check before either has saved anything. Enforce uniqueness at the storage layer and make acquisition of the operation atomic.

For external calls, record ownership before sending the request. Include the destination's supported idempotency mechanism where one exists, following that provider's documentation. Your local guard and the provider's guard solve related but different problems.

Do not assume a queue guarantees exactly one execution. Design the business action to tolerate redelivery. Also make locks expire safely: an expired worker lease is a reason to inspect the operation, not automatic proof that the external action never happened.

The timeout example that catches teams

Imagine a workflow creates a delivery task in another application. The destination saves task 907, but the response is lost. Your worker records a timeout. If it immediately sends another create request, the customer may receive two dispatch calls.

In a safer design, the operation moves to Unknown. A recovery process searches the destination using the external reference or supported status endpoint. It finds task 907, verifies that the key fields agree, stores the result, and marks the operation Confirmed. This is an illustrative example rather than a reported client incident.

If the destination provides no reliable lookup, route the uncertain case to an operator. That is less convenient than automatic retry, but it avoids pretending certainty where none exists. Choose integration tools partly on their ability to support recovery, not only initial creation.

Distinguish duplication from amendment

Suppose order 812 changes its quantity after approval. Reusing the original create key with a new payload should produce a conflict. A reviewed update operation needs its own identity, reference to the previous version, and rules about what can be changed.

For accounting workflows, do not silently turn a duplicate into an alteration. The finance team must define correction behaviour, especially after reconciliation or other downstream processing. A developer should not decide whether to replace, reverse, or amend a voucher solely to make a retry succeed.

The same distinction applies to customer messages. Re-sending an identical reminder after a delivery error is different from sending an updated amount. Keep message intent and version visible so an operator can explain what the customer received.

Test business effects, not only responses

Run the same operation twice, then run it simultaneously from two workers. Kill a worker after the external request but before local confirmation. Replay an old event after a newer revision. Test the same document number in two companies and two different documents with the same value.

For each test, count the actual destination records and verify their contents. A passing unit test that checks only a local flag will miss duplicate work in another system. Keep a reconciliation report for source operations versus confirmed destination outcomes.

Measure duplicate deliveries prevented, uncertain outcomes awaiting review, and conflicts caused by changed payloads. These are useful operational signals. A high duplicate-delivery count may be harmless if the business effect is controlled; a single unexplained duplicate payment instruction is not.

Apply the pattern to one workflow first

Start with a bounded process such as lead assignment or an accounting review task. Map its identity, confirmation evidence, and recovery path. The API integration guide gives wider context, while automation systems consulting covers implementation support.

For finance integrations, product interfaces differ. Tally's integration prerequisites describe its supported connection setup; they do not remove the need to define your own duplicate and recovery controls. Send me the workflow that currently creates repeat work, including a redacted example of how the duplicate is discovered.