Service · SERP & search data

SERP Data Scraping Services

With location, device and AI Overview citations captured, not a single global rank.

SERP data scraping is the automated collection of search engine results pages — organic positions, sponsored placements, SERP features such as featured snippets and local packs, AI Overview presence and the sources it cites — captured per keyword, location, device and language, because a search result does not exist independently of those four.

There is no such thing as your ranking. There is your position for a keyword, in a location, on a device, in a language, at a moment. Any vendor reporting one number has silently chosen the other four for you.

Get a free SERP data sample See sample data

Free pilot on your own sources, returned in 48 hours. No card, no trial clock — and you keep the sample data either way.

✓ Location and device specific✓ AI Overview citations captured✓ Free pilot sample in 48 hours

serp_results_2026-08-05.jsonl LIVE FEED

{"keyword":"best cordless vacuum", "engine":"google","country":"GB", "location":"Manchester, GB", "device":"mobile","language":"en-GB", "observed_at":"2026-08-05T06:11:42Z", "features_present":["ai_overview","shopping_pack", "people_also_ask","video_carousel"], "ai_overview":{"present":true, "cited_domains":["which.co.uk","techradar.com", "example-brand.com"],"citation_count":5, "position_on_page":1}, "organic":[{"pos":1, "domain":"which.co.uk","pixel_depth":1840}, {"pos":2,"domain":"example-brand.com"}], "paid_slots_top":4, "organic_above_fold":false} {"keyword":"cordless vacuum near me", "device":"mobile", "features_present":["local_pack"], "local_pack":[{"pos":1,"name":"Example Retail Manchester"}]}

2 of 12,408,900 keyword-location-device rows · run 2026-08-05T06:00Zfeature detection 97.1% · schema v5.2

Our Data Powers

B2C Marketplace

amazon

D2C + Marketplace

NYKAA

D2C + Marketplace

Walmart

FMCG Marketplace

udaan

Food Delivery

Uber Eats

Quick Commerce

blinkit

Taxi Aggregator

Uber

E-Commerce

Tmall

Written and reviewed by the Actowiz Solutions Data Engineering team Published 5 August 2026 Last updated 5 August 2026 Reading time 11 min

Key facts at a glance

What it is: Managed collection of search results pages with full feature detection, per keyword, location, device and language
Dimensions: Keyword, engine, country, location, device and language are all first-class fields
SERP features: AI Overviews, featured snippets, local packs, shopping packs, video carousels, People Also Ask and more
AI Overview citations: Which domains an AI Overview cites, and how many, per keyword and location
Pixel depth: How far down the page a result actually sits, which rank position alone does not tell you
Paid placement: Sponsored slot counts above and below organic, so real visibility is measurable
Refresh: Daily standard; sub-daily on volatile keyword sets or during algorithm updates
Who it's for: SEO and content teams, agencies, brands tracking AI search visibility, and researchers

4 dimensionson every recordkeyword, location, device, language

AI Overviewcited domains capturedthe new visibility layer

Pixel depthnot just position numberabove-fold reality

97.1%SERP feature detectionmeasured monthly

Key takeaways

What it is: Managed collection of search results pages with full feature detection, per keyword, location, device and language
Dimensions: Keyword, engine, country, location, device and language are all first-class fields
SERP features: AI Overviews, featured snippets, local packs, shopping packs, video carousels, People Also Ask and more
AI Overview citations: Which domains an AI Overview cites, and how many, per keyword and location
Pixel depth: How far down the page a result actually sits, which rank position alone does not tell you
Paid placement: Sponsored slot counts above and below organic, so real visibility is measurable

Last verified 5 August 2026 by the Actowiz Solutions Data Engineering team.

Definition

What is SERP data scraping, and why is a single rank number meaningless?

SERP data scraping is the automated collection of search engine results pages: the organic results and their order, sponsored placements, and the growing set of SERP features that occupy the page — AI Overviews, featured snippets, local packs, shopping carousels, video blocks and question boxes.

The central point about this category is that a search result is a function of four variables, and collapsing them produces a number that describes nobody's experience.

The four dimensions

Keyword. Obvious, and the only one most tools handle properly.
Location. Results differ by city and sometimes by neighbourhood, especially where local intent exists. A national rank is an average of many different pages.
Device. Mobile and desktop SERPs differ in feature composition and in how much of the page organic results occupy.
Language. Distinct from country. Multilingual markets return substantially different results by interface language.

Why position number is no longer sufficient

Being position one means less than it used to, because the page above position one has grown. On many commercial queries a mobile SERP now carries an AI Overview, four sponsored slots and a shopping pack before the first organic result appears.

So we capture pixel depth — how far down the page a result actually sits — alongside position, plus an organic_above_fold flag. A position-one result 1,800 pixels down the page is not a position-one experience, and no amount of rank reporting will show that.

AI Overview citations: the new visibility question

Where an AI-generated overview appears, the commercially important question has shifted from "do I rank" to "am I cited". We capture AI Overview presence, the domains it cites, citation count and its position on the page — per keyword and location, since AI Overview presence itself varies by both.

What we do not provide

Search volume, click data or clickstream. Volume figures come from search engine tools and licensed data providers; click behaviour is not observable from a SERP. Anyone offering scraped search volume is passing off a model or a licensed source.

What we collect

Six categories of SERP data

Feature detection and AI Overview citation tracking are the fastest-growing use cases.

Organic results

Positions with the context that makes them meaningful.

Position, URL, domain and title
Pixel depth on the rendered page
Above-fold flag by device
Displayed snippet text
Sitelink and rich result presence

AI Overviews & answer boxes

The new visibility layer.

AI Overview presence per keyword and location
Cited domains and citation count
Position of the overview on the page
Featured snippet presence and source
Overview presence change over time

Paid placement

How much of the page is bought.

Sponsored slot counts above and below organic
Advertiser domains where displayed
Shopping pack presence and participants
Paid share of above-fold space
Sponsored placement change over time

Local & map results

Where local intent is served.

Local pack presence and members
Local pack position and ratings shown
Map result composition
Location sensitivity of the whole SERP
Local pack change by location

SERP feature inventory

The full composition of the page.

Every feature present, classified
Feature ordering down the page
People Also Ask questions
Video, image and news carousels
Feature appearance and disappearance events

Volatility & change

Handled honestly.

Position change over time windows
Volatility measurement per keyword set
Feature composition change events
Repeat-sample variance where measured
Algorithm update period flagging

Service scope

What the ecommerce data scraping service includes

A managed engagement, not a tool licence. We own the pipeline and everything that breaks in it.

✓ Included in every engagement

Keyword, location, device and language mandatory on every record
Pixel depth and above-fold flags alongside ordinal position
AI Overview presence with cited domains and citation counts
Configurable repeat sampling with observed variance delivered
Precise observed_at timestamps, since SERPs change within the day
Source discovery, scoping and a written collection plan
Free pilot on your own sources before any commitment
Full pipeline build, hosting and proxy infrastructure
Schema design, validation and sampled human QA on every run
Ongoing maintenance when source layouts change — our cost, not yours
Delivery to your warehouse, bucket, SFTP or API endpoint
Documented methodology and compliance notes for your legal review

× Not included — stated upfront

Search volume, click data or clickstream of any kind
Search engine advertising accounts, keyword planner or any authenticated surface
Claims about why a domain is cited in an AI Overview
Single-number rank reporting that hides location, device and variance
Anything behind a login, paywall or credentialed session
Personal data beyond a documented lawful basis
Licensed third-party datasets we do not hold rights to
Guarantees about fields a source simply does not publish

Schema

SERP data fields you receive

Every engagement delivers a documented schema. These are the core fields; the full dictionary runs to 120+ and is agreed during scoping.

Deliverable schema — v5.2 core fields (full dictionary: 120+ fields)
Field	Type	What it captures	Refresh
`keyword / engine`	string / enum	Query and search engine, since results differ substantially between engines	Every run
`country / location / device / language`	string / enum	The four dimensions that define a result, all mandatory	Every run
`observed_at`	timestamp	Exact capture time, essential because SERPs change within the day	Every run
`features_present`	array	Every SERP feature detected, classified into a standard taxonomy	Every run
`ai_overview`	object	Presence, cited domains, citation count and position on page	Every run
`organic`	array	Positions with URL, domain, title, snippet and pixel depth	Every run
`pixel_depth`	int	Vertical position on the rendered page, which position number alone does not convey	Every run
`organic_above_fold`	boolean	Whether any organic result appears above the fold on that device	Every run
`paid_slots_top / paid_slots_bottom`	int	Sponsored placement counts, so bought share of the page is measurable	Every run
`local_pack`	array	Local pack members with position and displayed rating where present	Every run
`sample_variance`	decimal	Where repeat sampling is configured, observed variance for the same query	Per configured runs

SERPs are not deterministic. Two requests seconds apart can differ, and we do not pretend otherwise. Where accuracy matters, repeat sampling is configurable and observed variance is delivered rather than hidden behind a single reading.

Coverage

Engines, locations and surfaces we collect from

Location granularity is the main cost lever. City-level is usually sufficient; sub-city is available where local intent is strong.

Google SearchGoogle Shopping surfaceGoogle Maps resultsBingDuckDuckGoYandexBaiduNaverYahoo JapanAmazon searchYouTube searchApp store searchCountry-level collection in 40+ marketsCity-level location targetingSub-city targeting where local intent is strongMobile and desktopMultiple interface languages

We collect what is publicly returned to an ordinary search request. We do not use search engine advertising accounts, keyword planner data or any authenticated surface, and we do not provide search volume. Request a source we don't list →

Markets served

Countries and markets where this service is in highest demand

We deliver into 40+ countries. These are the markets where this particular service is requested most, and the reason demand concentrates there.

Highest-demand markets for this service, and why demand concentrates there
Market	Why demand concentrates here
United States & United Kingdom	Where AI Overviews rolled out earliest and most broadly, making citation tracking most urgent for brands here.
Germany, France & Spain	Multilingual SERPs where language and location interact, so single-dimension rank tracking fails most visibly.
India & GCC	High-growth search markets with heavy mobile skew, where device-specific feature composition differs sharply from desktop.
Australia & Canada	Strong local intent patterns making sub-city location targeting particularly valuable for multi-location businesses.

North America

United StatesCanadaMexico

United Kingdom & Ireland

United KingdomIreland

Western Europe

GermanyFranceNetherlandsBelgiumSpainItalySwitzerlandAustria

Nordics

SwedenNorwayDenmarkFinland

Middle East

United Arab EmiratesSaudi ArabiaQatarKuwaitIsrael

Asia Pacific

SingaporeAustraliaNew ZealandJapanSouth KoreaMalaysiaIndonesiaThailandVietnamPhilippines

South Asia

IndiaBangladeshSri LankaPakistan

LATAM

BrazilArgentinaChileColombia

Africa

Studying search result composition and information sources requires reproducible, documented SERP capture.

What we deliver

Documented SERP capture with all four dimensions recorded and sampling variance reported.

Metric that moves

Reproducibility

Use cases

How SERP data gets used in practice

Four patterns, with the outcome each is judged on.

AI Overview citation tracking

AI Overview presence and cited domains are captured per keyword and location, so citation share can be measured over time and against competitors, including where an overview replaces organic clicks entirely.

Outcome: AI search visibility measured rather than assumed, with citation gaps identified by keyword cluster.

Real visibility measurement with pixel depth

Positions are captured with pixel depth and above-fold flags per device, so a position-one result sitting below an AI Overview, four ads and a shopping pack is correctly identified as low visibility.

Outcome: Visibility reporting that tracks traffic reality instead of ordinal rank.

Multi-location local pack monitoring

SERPs are collected per targeted location including sub-city where local intent is strong, capturing local pack membership and position per location.

Outcome: Local visibility managed per catchment rather than as a national average.

Competitive page-share analysis

Full feature inventory plus paid slot counts and organic positions are combined into a share-of-page view by keyword, showing who occupies what and how it changes.

Outcome: Competitive share measured across the whole page including paid and features, not organic alone.

Engagement examples

Two engagements, anonymised

Clients rarely permit naming. These are real engagement shapes with identifying detail removed, so you can judge whether the work resembles your situation.

Retail brand · UK

Rank improved while organic traffic fell

Situation

Reporting showed positions rising across a commercial keyword set, but organic sessions declined, and nobody could reconcile the two.

What we ran

Collection with pixel depth, above-fold flags and full feature inventory including AI Overview presence, per device and location.

Result

Position-one results were sitting below an AI Overview, four ads and a shopping pack on mobile, which explained the divergence.

Publisher · EU

AI Overview citation share was completely unknown

Situation

The publisher suspected AI-generated answers were reducing clicks but had no measurement of whether it was being cited or bypassed.

What we ran

AI Overview presence and cited domain capture across the topic keyword set, tracked by location and over time.

Result

Citation share became measurable by topic cluster, redirecting content effort to where citations were being lost.

Examples are anonymised at client request. Named references are available on request under NDA. See published case studies →

The 48-hour sample — run on your sources, not ours

Before you commit to anything, we run this service against your own sources and send you the output. If the coverage isn't there, the sample will show you that too — which is the point. We would rather lose the deal at the pilot than at month three.

Real extraction from your actual sources
Returned inside two business days
Coverage and QA note included
You keep the data either way
No card, no trial clock
Named engineer on the call

Get my free sample Book a 20-min scoping call Reply within one business day. Reference calls available under NDA.

How we engage

Three ways to engage us for this work

Same collection pipeline and same QA underneath. The difference is who holds the schedule and how the data reaches you.

Managed service (most common)

We own the collection, the QA and the delivery. You receive clean data on a schedule and never touch a scraper.

Dedicated engineer assigned to your account
Site changes fixed by us, not reported to you
Scheduled delivery to your warehouse or S3
Named contact on Slack or email

Best fit: Teams who need the data, not the infrastructure.

API access

The same collection pipeline exposed as an authenticated REST endpoint your systems query directly.

On-demand and scheduled endpoints
Rate limits agreed to your load profile
Sandbox keys for integration testing
Versioned schema with deprecation notice

Best fit: Product and engineering teams building on live data.

One-time or project extraction

A defined pull for a specific question — market sizing, diligence, a pitch, a one-off audit.

Fixed scope agreed in writing upfront
Single delivery with full QA report
Methodology documented for your records
Converts to managed if you want continuity

Best fit: Research, strategy and diligence work with a deadline.

Pricing

Every engagement is quoted individually, because the honest answer depends on your scope: how many sources, how many records, how often, and how the data reaches you. We scope it with you, run a free pilot on your own sources, and then quote a fixed monthly figure — no per-request metering and no overage billing when volumes move. Request a quote and you will have a number after one call.

Build vs buy

Should you build SERP collection in-house or hire it as a service?

Feature detection changes constantly as search engines change layouts, which makes this permanent maintenance rather than a build.

In-house build vs self-serve tool vs Actowiz managed service
Consideration	In-house scraping team	Generic proxy / DIY tool	Actowiz managed feed
Time to first usable data	6–12 weeks of engineering before anything is trustworthy	Days, but output needs manual cleanup before use	Free pilot in 48 hours, production in 5–10 business days
Who fixes it when a source changes	Your engineers, at the cost of their roadmap	You do — tools report failures, they don't resolve them	We do, same business day, inside the retainer
Data quality assurance	Whatever your team has time to build	None beyond HTTP success	Schema validation plus sampled human QA on every run
Compliance documentation	Rarely produced, then requested urgently by legal	Not provided; terms risk sits with you	Sources, method and lawful basis documented for review
Accountability	Distributed across a team with other priorities	A support ticket queue	A named engineer and an account owner
True annual cost	Engineer salaries, proxies, hosting, ongoing maintenance	Low licence fee plus significant hidden analyst time	One fixed monthly retainer, quoted after scoping

Why we report sampling variance instead of pretending SERPs are deterministic

Rank tracking tools present a single number per keyword per day. This implies determinism that does not exist, and the implication causes real analytical damage.

Why the same query returns different results

Ongoing experimentation. Search engines continuously test layouts and rankings, so different requests can land in different test buckets.
Infrastructure variation. Requests are served from different data centres with slightly different index states.
Fine-grained location inference. Even at the same nominal location, inferred precision can vary and shift local results.
Feature triggering thresholds. Features like AI Overviews appear inconsistently on borderline queries.
Time of day. Freshness-sensitive queries genuinely change through the day.

What we do about it

Where accuracy matters, repeat sampling is configurable: the same query is collected multiple times within a window, and we deliver observed variance in a sample_variance field alongside the reading. That turns a false certainty into a measured confidence interval.

Every record also carries a precise observed_at timestamp, so a position can be tied to a moment rather than to a day. For keyword sets tracked through an algorithm update, this is the difference between seeing a change and guessing at one.

We would rather deliver a range with variance than a single number that looks authoritative and is not reproducible. Teams making content investment decisions on rank movements of one or two positions are frequently reacting to noise, and only variance data reveals that.

AI Overviews changed the question from ranking to being cited

For a growing share of informational and commercial queries, an AI-generated overview now sits above the organic results and answers the query directly. This changes what SERP data needs to capture.

What matters now

Whether an overview appears at all. It varies by keyword, location, device and over time, and its appearance materially reduces organic click-through below it.
Which domains are cited. Citation is the new visibility. A page can rank fourth and be cited, or rank first and not be.
How many sources are cited. Fewer citations means a narrower set of beneficiaries.
Where the overview sits. Above or below other features changes how much page it consumes.
How citation sets change. Citation stability over time indicates whether a position is defensible.

How we capture it

AI Overview presence, cited domains, citation count and page position are captured per keyword, location and device on every run. Because presence itself is variable, repeat sampling is particularly valuable here — a single reading can miss an overview that appears on most requests.

What we do not do is claim to explain why a domain is cited. Citation selection is not documented by search engines, and vendors offering optimisation formulas for it are guessing. What we provide is accurate observation of who is cited, for which queries, where, and how that changes — which is the factual basis your content strategy needs. Our news data service is often paired with this for tracking which publishers gain citation share.

How it works

How a SERP data engagement goes live in 5 to 10 business days

Keyword set, location granularity, devices and sampling configuration are scoped first, since those four multiply volume.

Scope the sources and fields

You send us target sites, regions, SKUs or keywords. We return a field-level schema proposal, coverage estimate and refresh recommendation — usually within two working days.

Pilot sample, free

We extract a real sample from your actual targets so you can inspect field fill rates, edge cases and match quality before any commitment.

Production build and QA harness

Our engineers build extractors, then wire validation rules: type checks, range checks, duplicate detection and golden-record comparison against a manually verified subset.

Scheduled delivery into your stack

Feeds run at your chosen cadence and land in the warehouse or bucket you already use. Schema changes are versioned and announced before they ship.

Ongoing monitoring and SLA support

We watch coverage drift, fill rates and source changes daily. A named engineer owns your account, and layout breaks are fixed by us — not queued for you.

Formats & destinations

JSON, JSONL, CSV, Parquet or XLSX, delivered to Amazon S3, Google Cloud Storage, Azure Blob, SFTP, Snowflake, BigQuery, Databricks or a REST/GraphQL endpoint. Webhooks fire on completion, and every batch ships with a manifest containing row counts, schema version and QA results so your pipeline can fail loudly instead of silently ingesting a bad file.

Compliance & data ethics

We collect what is publicly returned to an ordinary search request, at controlled request rates. We do not use search engine advertising accounts, keyword planner data, or any authenticated surface, and we do not provide search volume or click data. Search engine terms restrict automated querying and we state that plainly; methodology is documented per engine.

Service commitments

What we commit to, in writing

These are contractual, not marketing copy. They appear in the engagement document.

Service level commitments written into every managed engagement
Commitment	What we hold ourselves to
Pilot turnaround	A real sample from your own sources within 48 hours of scoping, at no cost.
Go-live	Production collection running within 5–10 business days of sign-off.
Delivery punctuality	99.5% on-schedule delivery, measured monthly and reported to you.
Breakage response	Source layout changes triaged same business day; critical sources inside 4 hours.
Data quality	Schema validation on every run plus sampled human QA before any delivery leaves us.
Escalation	A named engineer and an account owner, not a shared ticket queue.
Change requests	Field additions and source changes handled inside the retainer, not re-quoted.
Exit	Your historical data exported in full on request. No lock-in, no export fee.

Why teams pick Actowiz for this work

Engineers, not a dashboard. You get people who fix breakages, not a self-serve tool you maintain yourself.
We tell you what we can't do. Scope limits and coverage gaps are stated before you sign, not discovered in month three.
QA is part of the service. Schema validation and sampled human review run before delivery, every run.
Compliance is documented. Sources, method and lawful basis written down so your legal team can review them.
Fixed monthly cost. No per-request metering, no surprise overage on a month when a competitor adds SKUs.
Six years, 40+ countries. Long-running production pipelines across retail, travel, mobility and finance.

Definitions

Terms used on this page

Plain definitions of the terms used on this page, so procurement and legal reviewers are working from the same vocabulary as your data team.

Pixel depth: How far down a rendered results page a result actually appears. Position one can sit well below the fold once an AI Overview, sponsored slots and a shopping pack occupy the space above it.
AI Overview citation: A source referenced by an AI-generated answer at the top of a results page. Where overviews appear, citation matters more than ordinal rank because the answer resolves the query without a click.
Sampling variance: The observed difference between repeated collections of the same query. SERPs are not deterministic, so a single reading carries uncertainty that only repeat sampling can quantify.

FAQ

SERP data scraping: frequently asked questions

What SEO, content and agency teams ask during evaluation.

Yes — presence, cited domains, citation count and position on the page, per keyword, location and device. This is the fastest-growing reason clients come to us.

What we will not do is claim to explain why a domain gets cited. Citation selection is not documented by search engines, and formulas for it are guesswork. We provide accurate observation of who is cited for which queries and how it changes, which is the factual basis a content strategy needs.

Because a search result does not exist independently of them. Results differ by city and sometimes neighbourhood, and mobile and desktop SERPs differ in feature composition and in how much page organic results occupy.

A tool reporting one rank number has silently chosen a location and device for you. We make all four dimensions — keyword, location, device, language — mandatory fields, so you always know what the number describes.

How far down the rendered page a result actually sits, in pixels. Position one used to mean top of page; on many mobile commercial queries it now sits below an AI Overview, four sponsored slots and a shopping pack.

We deliver pixel depth and an organic_above_fold flag alongside position, because a position-one result 1,800 pixels down is not a position-one experience. This is usually the field that explains why improving rank stopped improving traffic.

No. Search volume is not published in search results — it comes from search engine advertising tools and licensed data providers. Anyone offering scraped search volume is passing off a model or a licensed source.

We collect what is observable on the page. For volume, use the engine's own tools or a licensed provider, and join it to our data on the keyword field.

Not perfectly, and we do not pretend otherwise. The same query seconds apart can differ due to ongoing experimentation, data centre index variation, location inference and feature triggering thresholds.

Where accuracy matters, repeat sampling is configurable and we deliver observed variance in a sample_variance field. That converts false certainty into a measured confidence range — and it often reveals that a team reacting to a one-position movement is reacting to noise.

Country, city, and sub-city where local intent is strong. Sub-city granularity matters for local pack work, since pack composition can differ between neighbourhoods in the same city.

Location count is one of the main cost multipliers, so we scope it deliberately. For most national SEO work, a handful of representative cities answers the question; for multi-location businesses, per-location collection is the point.

Google as the primary, plus Bing, DuckDuckGo, Yandex, Baidu, Naver and Yahoo Japan where relevant to your markets. We also collect Amazon, YouTube and app store search results, which behave as their own search ecosystems.

Feature detection is most complete on Google because it has the richest feature set; on other engines we detect what exists. We state per engine what feature coverage looks like during scoping.

Search engine terms restrict automated querying, and we say that rather than glossing over it. Our practice is to collect what is publicly returned to an ordinary request, at controlled rates, without advertising accounts or any authenticated surface.

You receive a written methodology document per engine and a DPA before signature. Notably, rank tracking is an established industry practice used by essentially every SEO team and tool, which is useful context for that conversation — though it is not a legal conclusion, and your counsel should form their own.

We quote individually, and here the quote is driven by four multipliers: keywords, locations, devices and refresh frequency — plus repeat sampling if configured, which multiplies again.

A focused keyword set across a few cities on both devices at daily refresh sits at the lighter end. Large keyword sets across many locations with repeat sampling sits considerably higher. One scoping call, a free pilot on your own keywords within 48 hours, then a fixed monthly quote. Request a quote.

See real SERP data for your own keywords

Send us keywords and target locations. We return full SERPs with feature detection, AI Overview citations and pixel depth within 48 hours.

Get a free SERP data sample Book a 20-min scoping call

Free pilot, no card, no obligation. We'll show you sampling variance on your own keyword set.

Enterprise Data Extraction

AI Training Data

Analytics Dashboard

AI Dynamic Pricing

Social Commerce

GCC Quick Commerce

KitchenIntel

UK Grocery Price Tracker

Ready-to-Use Data