Coles Data Scraping Services

Two promotional mechanics running at once

Most grocers run one primary promotional mechanic. Coles runs two concurrently, and they mean different things commercially.

• Sustained reductions hold a lower price for an extended period. Duration is the signal; these are closer to repositioning than to promotion.
• Weekly specials run on the standard cycle with fixed day boundaries and are genuine short-term promotions.
• A line can be in both at once, which is where a single promotional flag breaks.

Collapsing them into one "on promotion" boolean loses the distinction between a price the retailer intends to hold and one it intends to reverse. We capture is_sustained_reduction with its start date and duration, and in_weekly_specials with the cycle identifier, as independent fields.

The combination matters: a line in a sustained reduction and a weekly special is at its deepest observable price, and that state is invisible if the two are merged. Nobody we have seen separates them.

Cycle alignment decides whether a comparison is valid

Our Woolworths page introduces both_in_cycle; this is the other half of it, and it is worth stating from this side because the failure is so easy.

The two chains run weekly specials cycles that do not always start and end on the same days. A same-day price comparison can therefore pit a promoted price at one chain against a base price at the other, and produce a competitive conclusion that reverses the following day.

• Cycle boundaries must be captured on both sides, or alignment cannot be determined at all.
• Comparisons should be filterable to aligned periods, not just computed on whatever day the data was pulled.
• Misaligned observations are still useful for single-chain analysis, so we flag rather than exclude them.

We deliver specials_cycle_id, cycle_start, cycle_end and aligned_with_competitor_cycle where we collect both chains. This is the single most common error in Australian grocery analysis, and it is avoidable with one field.

Mandated unit pricing, audited from both sides

Australia requires unit price display on grocery, which makes it possible to check a vendor's arithmetic against a published figure — the same point our Woolworths page makes, and worth having on both chains so the audit works across the comparison.

We deliver unit_price_displayed, unit_price_computed, the basis used, and unit_price_agrees.

Why it matters here specifically: a cross-chain unit price comparison inherits errors from both sides. If one chain's unit prices are computed on a different basis from the other's, the comparison is wrong even when each side is internally consistent. We normalise the basis across both chains and report where the published figures themselves use different bases for equivalent products — which happens, and which is worth knowing before you build a category index on it.