John Lewis Data Scraping Services

An included guarantee is value that never touches the price

John Lewis includes longer guarantees on categories where competitors offer a shorter standard term or sell an extension separately. That is real value, and it is invisible in a price field.

• A price comparison ignoring it systematically overstates the gap against discounters.
• A guarantee sold separately elsewhere has an observable price at that competitor, which makes a value-adjusted comparison possible rather than speculative.
• Guarantee terms vary by category, so it cannot be handled as a single retailer-level constant.

We capture guarantee_years and the terms as displayed, per product. Where a client wants a value-adjusted comparison we build it explicitly with the assumption stated, rather than folding an estimate into a price field where nobody can see it.

Own brands sit alongside third-party brands, at a different position

John Lewis operates substantial own-brand ranges across home, furniture and apparel, listed alongside the third-party brands it retails.

For a brand competing there, an own-brand line is priced and placed by the company controlling the placement, and it typically occupies a deliberate price position beneath comparable branded goods. Blending it into a category index produces a benchmark that moves with own-brand strategy rather than with market pricing.

is_own_brand is assigned by a published rule with the brand list shipped, so an index can exclude own brands deliberately rather than by someone remembering to filter.

Concession and partner listings change what the price means

Some listings are fulfilled or sold by partners and concessions rather than by John Lewis directly. Where the listing indicates this, it matters.

A John Lewis price is the retailer's own decision, and it is what a supplier negotiates against. A partner price is a third party's decision, and for a brand it is a channel question.

We capture seller_indicated and the seller name where displayed. Where the listing gives no indication, the field is recorded as unknown rather than assumed to be the retailer — assuming is how a partner's pricing quietly ends up in a retailer benchmark.

A department store is several datasets on one domain

John Lewis sells electricals, furniture, fashion, beauty and home on the same site. These do not share a data shape, and treating them as one catalogue produces a dataset that is adequate for none of them.

• Electricals need model number, guarantee term and delivery tier.
• Fashion needs size-level availability, or the stock field is close to meaningless.
• Furniture needs configuration options and lead time, where the price on the page is a starting point.
• Beauty needs shade-level availability and non-price promotion capture.

We scope collection per category tree rather than applying one field set across the site. Where a client only needs one department the schema is cut to it; where several are needed, department-specific extensions sit alongside a common core so categories stay comparable on price and lifecycle without being forced into an identical shape.

Made-to-order furniture is priced from, not priced at

A large part of the furniture range is configurable: fabric, finish, size, firmness, orientation. The headline figure is a from-price for the cheapest configuration, and the delivered price for a realistic specification can be substantially higher.

A dataset recording that headline number as the product price will report a sofa range as far cheaper than any shopper actually pays, and the error is systematic rather than random.

We capture is_configurable, the option groups and their price effects where displayed, and price_basis recording whether the captured figure is a from-price or a fixed one. Lead time is captured separately, because on made-to-order furniture a twelve-week lead time is a bigger competitive fact than a small price difference.

Delivery tiers are part of the proposition

Delivery charges, thresholds and premium delivery options vary by category, and on furniture and large items they are a material part of the total.

We capture the delivery options and thresholds displayed against a product rather than assuming a single site-wide policy. For large-item categories, a comparison on product price alone can be wrong by more than the price difference it is trying to measure.