Sooner or later, an automation embarrasses someone. The same customer gets the same email six times. An article posts to a channel repeatedly. An invoice goes out twice.
The instinct is to blame the automation tool. It is almost never the tool. It is that the workflow has no way of knowing what it has already done.
Why this happens
A scheduled workflow is stateless by default. Every run asks the same question — "what should I process?" — and unless the answer changes, it does the same work again.
Consider a workflow that publishes new blog posts to a channel. It runs hourly and asks: give me the published posts. That query returns the same post every hour, forever. The workflow is behaving exactly as written. It simply has no concept of "already done."
The same shape causes duplicate onboarding emails, repeated Slack alerts, and reprocessed orders.
Where teams try to put the state, and why it fails
In the schedule. "It runs hourly, so only look at the last hour." This works until a run fails, or the platform is down, or a record arrives late — then that window is missed permanently. Time-window filters trade duplicates for silent data loss, which is worse because nobody notices.
In the workflow tool's memory. Some platforms offer static data or variables. This couples your correctness to one vendor, disappears when you rebuild the workflow, and cannot be inspected or corrected by anyone who is not inside the tool.
In a spreadsheet. Common, and it works until two runs overlap or someone sorts the sheet.
Where it belongs
On the record itself, in the database that already holds it.
For a publishing workflow, that is a syndicated_at timestamp on the post. The workflow asks a different question — give me published posts where `syndicated_at` is null — and stamps the column when it finishes.
That single change makes the workflow idempotent: running it twice produces the same result as running it once. The hourly schedule stops mattering. A failed run self-corrects on the next pass, because the unprocessed record is still unprocessed.
```sql alter table posts add column syndicated_at timestamptz;
create index posts_awaiting_syndication_idx on posts (published_at) where status = 'published' and syndicated_at is null; ```
The partial index matters more than it looks. The workflow only ever asks for rows awaiting processing, and that index answers exactly that question — so the query stays fast as the table grows.
The design rule underneath
State belongs where the data lives, not where the logic runs.
Workflows get rebuilt, migrated between tools, edited by people who did not write them. Anything holding correctness inside a workflow is one refactor away from breaking. A column in the database survives all of that, and can be inspected, corrected and audited by anyone with access.
It also makes the system explicable. "Why did this not send?" has an answer you can query, rather than an answer you have to reconstruct from execution logs.
Handling failure honestly
Once you have the column, you face a real decision: what happens when part of the work fails?
Say the workflow posts to two platforms. The first succeeds, the second returns an error. Do you stamp the record?
Stamp it, and a permanently-failing record does not retry forever — but a transient failure is never retried either. Failures surface in the execution log rather than self-healing.
Do not stamp it, and transient failures self-correct on the next run — but anything permanently broken retries every hour indefinitely, which is its own kind of noise.
There is no universally correct answer, and this is exactly the choice people skip. Decide deliberately, write down which you chose, and put the reasoning somewhere the next person will find it. If you need both behaviours, add an attempt counter and stop after a threshold.
The wider pattern
This generalises well beyond publishing:
- Sending onboarding emails →
welcome_sent_at - Syncing records to another system →
synced_atplus the remote identifier - Processing payments → an idempotency key on the request itself
- Enriching records →
enriched_at, so you do not pay per record repeatedly
In each case the question shifts from what exists? to what have I not handled yet? — and that shift is the difference between an automation that scales and one that eventually embarrasses you.
The check worth doing today
Take any recurring automation you run. Ask one question: if this ran twice right now, what would happen?
If the answer is "nothing, it would find nothing to do," it is idempotent and safe. If the answer is "it would do the work again," you have a column missing — and it is a fifteen-minute fix that will save you an apology later.
Common questions
- Why does my automation send duplicate emails?
- Because it is stateless. Each scheduled run asks the same question and gets the same answer, so it repeats the same work. The fix is a column on the record recording that the action was taken, and a query that excludes rows already stamped.
- What does idempotent mean for a workflow?
- Running it twice produces the same result as running it once. In practice that means the workflow asks what have I not handled yet rather than what exists, and records completion on the record itself.
- Why not store workflow state in the automation tool?
- It couples correctness to one vendor, disappears when the workflow is rebuilt, and cannot be inspected or corrected by anyone outside the tool. State belongs where the data lives so it survives refactors and tool changes.
- Should I mark a record as done if part of the work failed?
- There is no universal answer. Stamping prevents permanent failures retrying forever but means transient ones never retry. Not stamping self-heals transient failures but loops on permanent ones. Decide deliberately, document the choice, or add an attempt counter with a threshold.
- Automation
- n8n
- Data modelling
- Workflow design