Platform specifics
Why the honest answer is not 'yes we can scrape it'
These are the reasons a Shufersal dataset needs its own handling rather than a shared retail schema.
The data is mandated, published and public
Israel's 2014 Food Act — the price transparency regulation — requires supermarket chains above a small
size threshold to publish their price, promotion and store files publicly. The requirement came into effect in 2015
and it covers roughly thirty-five chains.
The files are published on chain portals, refreshed through the day, with full per-store snapshots daily.
What that means for a request to scrape Shufersal
The prices you want are already published. Building a scraper against the storefront would produce a worse dataset
than the file the chain is legally required to put online, more slowly, and with more fragility.
We say this rather than quoting for the scraper, because quoting for it would be selling you the more expensive
version of a thing you can have more cheaply.
What the honest engagement is
If you need Israeli grocery pricing at scale, the work is ingestion and normalisation across the published
files — not extraction from web pages. That is a real piece of engineering and it is what we would scope.
Where the difficulty actually is
Mandated publication does not mean usable publication. The chains publish to the letter of the requirement and no
further, and the practical result is a set of feeds that do not agree with each other.
- Formats differ per portal. Gzipped XML, ZIP-wrapped files, and encodings including UTF-16 alongside UTF-8.
- Portal behaviour differs. Some sign download URLs with a short expiry, so a listing must be followed
immediately. Some are geo-restricted.
- Identifiers are inconsistent. Item codes are usually barcodes, but chains also publish internal codes in
the same field, so a cross-chain join needs a canonical identifier derived rather than taken.
- Hebrew text arrives as published, in several encodings, and right-to-left handling has to survive the whole
pipeline.
- Update cadence differs — incremental files through the day, full snapshots daily, and not on the same
schedule per chain.
Making thirty-five of those agree on one schema, with a derived canonical identifier and a stated freshness per
chain, is the deliverable. It is a normalisation problem, not a collection one.
Two limits of the mandated data, worth knowing before you build on it
The law covers a retailer's own sales
Where a chain operates a platform carrying other retailers, the published file contains its own sales only.
A delivery platform's file contains its own dark-store range, not the partner supermarkets listed on the same app.
That is a coverage boundary rather than a gap in the data, and it matters if you assume a platform's file
represents everything sold through it.
Kosher certification
Certification is a significant purchase driver in this market and it appears in product data at varying levels of
detail depending on the chain.
We capture certification text exactly as published, with verified_by_actowiz false. Whether a
certification is current, and which certifying authority a buyer accepts, are matters for the issuing body and the
buyer — not something extraction or normalisation can determine.
What we would not do
Interpret certification levels, rank them, or map one authority's certification onto another's. Those are religious
and commercial determinations, and a vendor making them in a data field is overreaching.