Platform pain
Your nightly ETL is a 14-hour lie about inventory
Yes. If availability moves through a nightly batch, a case that sells at 10 a.m. keeps reading as sellable to every downstream system until the job runs again — a window that can stretch to 14 hours of invalid state. During it, your storefront, your wholesale portal, your delivery plan and your promotions all quote yesterday’s stock as fact. In beer route-to-market, that gap is paid off in weekend oversells and Monday reconciliation.
What exactly is the 14-hour lie?
The lie is not that the data is wrong. It is that the data is old, and nothing downstream knows it is old. A status changes at the top — a SKU sells out, a batch is quarantined, a price flips — and that change has to cascade to every system that reads it. Batch culture solves this with one large job on a schedule and calls it done. As documented in an analysis of nightly batch pipelines, the arithmetic is unforgiving:
“If a critical status changes at 10:00 AM, but the cascade ETL doesn’t run until midnight, you have 14 hours where your entire downstream ecosystem is operating on invalid state.”
Nobody schedules a 14-hour lie on purpose. It is the emergent property of a design where the truth is computed once a day and everything else trusts the copy. The copy is fine at 12:05 a.m. and rotting by lunch, and no alarm fires in between, because from the pipeline’s point of view the run succeeded.
Why does batch sync feel fine until it doesn’t?
Because most days the lie is harmless. Stock that does not move inside the window is described perfectly by a day-old snapshot, so the batch looks correct roughly ninety-something percent of the time and everyone stops questioning it. The window only bites when inventory actually moves inside it: the end of an allocation, a promotion, a hot SKU, the run-up to a weekend. RepSpark documents the canonical multichannel version — a unit sold on the storefront at 10 a.m. still reading as available to your wholesale buyers until the next batch sync.
Swap “storefront” for a depot pick and “wholesale buyers” for a field rep taking an on-trade order on a tablet, and you have the beer version exactly. The rep is not careless and the data is not corrupt. They are both reading a number that was true this morning and is a promise you can no longer keep this afternoon.
What does the batch gap actually cost?
Two bills arrive. The visible one is reconciliation labour: teams reportedly spend 6 to 15 hours a week reconciling inventory across systems, which is a polite way of saying they hand-correct the batch’s mistakes before a customer does. The invisible bill is every decision downstream systems made while the numbers were wrong, each one committed with full confidence.
| Downstream system | Decision on stale stock | Failure when the batch is hours behind |
|---|---|---|
| Wholesale / B2B portal | Accepts an order for units already sold | Oversell, then a cancellation call |
| Delivery planning | Loads a truck against phantom stock | Short-ship at the door, failed drop |
| Loyalty / promotions | Awards points on availability that is gone | Liability booked against a sale that reverses |
| Returnables / empties | Nets empties against a delivery that shrinks | Balance drift no one can reconcile |
Each row is a different team writing an apology. The loyalty row is worse than it looks, because points accrue against a sale that later unwinds — the accounting shape of that problem is the one we covered in deferred revenue in a spreadsheet. The empties row drifts the same way; if your returnables balance already lives in bespoke Z-tables, a stale delivery quantity makes it drift faster than anyone will admit at month-end.
Why is the weekend where it detonates?
Because beer demand is spiky and the batch is metronomic. An oversell you can absorb on a quiet Tuesday becomes a short-ship into Saturday’s rush, when there is no replenishment run to save you and the on-trade account remembers the empty tap longer than they remember the apology. The storefront cheerfully shows availability the depot does not physically have — the same storefront you may already be over-paying to operate, per our note on the storefront you already paid for once. The failure is not that one number was wrong. It is that four systems all acted on the same wrong number, in the same window, for the same customer.
So what actually stops the lie?
Not a faster batch. Halving the window to seven hours is still a seven-hour lie, and stock moves fastest exactly when the schedule is least helpful. The direction is to stop recomputing the truth on a clock and start propagating each change as an event the moment it happens, so availability, delivery, loyalty and returnables all read the same current state off one bus rather than four private copies of yesterday. That is the event-driven architecture we build toward, and it earns its own post’s worth of detail.
The uncomfortable part is that the nightly job will keep looking healthy right up to the quarter where it isn’t — a big allocation, a viral SKU, a heat-wave weekend — while the reconciliation hours quietly compound in the background. The brewers who move first will not do it because the batch failed a monitor. They will do it because they finally priced the 14 hours, and decided that a day-old copy of the truth is too expensive to keep believing.
Frequently asked questions
How long is the stale-inventory window with a nightly ETL?
As long as the gap between the change and the next run. If stock moves at 10 a.m. and the batch runs at midnight, downstream systems quote invalid availability for up to 14 hours — the figure depends on your schedule, not your data quality.
Isn't a more frequent batch enough to stop oversells?
It shrinks the window, not the failure mode. A four-hour batch is still a four-hour lie, and stock moves fastest exactly when it matters: allocation ends, promotions, the weekend rush. Only event-driven propagation closes the gap to seconds.
How much does batch reconciliation actually cost?
Teams reportedly spend 6 to 15 hours a week reconciling inventory across systems, plus the unmeasured cost of oversell cancellations, short-ships and loyalty points awarded on sales that later reverse. The labour is visible; the downstream decisions are not.
Stuck with exactly this?
BrewOS builds the full route-to-market stack for global brewers — order capture, stock, loyalty, returnables, delivery and analytics on one event bus, run by a team of ~20 engineers. Bring us the feature that’s been stuck the longest and we’ll show you how we’d ship it in days.
Book a 30-minute walkthrough