Public Catalogue Data Pipelines Across Shopee, TikTok Shop, Pinduoduo & Meituan

The Client

A market-intelligence business serving brands and investors with visibility into Asian commerce — the fastest-growing, least-transparent retail region in the world. Their clients needed to understand what was happening on the platforms that actually move volume across Southeast and East Asia: Shopee across SEA, TikTok Shop in its fastest-growing markets, Pinduoduo in China's value segment, and Meituan in local services and instant retail. Their problem was straightforward to state and brutal to solve: these are among the hardest public commerce surfaces on the internet to collect from reliably, and their existing approach — a patchwork of scripts and manual checks — broke constantly.

They came to Actowiz Solutions for reliable, recurring, clean public catalogue data across all four platforms.

The Challenge

Asian marketplace data is a different order of difficulty from Western e-commerce, and it compounds across four axes at once:

• These are app-first, heavily dynamic, defended surfaces. Shopee, TikTok Shop, Pinduoduo, and Meituan are built mobile-first with dense dynamic rendering and robust automated-access defenses that evolve constantly. Naive collection doesn't degrade here — it fails outright. Reliable recurring extraction demands genuinely resilient, self-healing infrastructure and deep per-platform experience, which is precisely the depth that separates a specialist from a general scraper.
• Language and script complexity is structural, not cosmetic. Product titles, specifications, variants, and merchant information arrive in Simplified Chinese, Thai, Vietnamese, Bahasa Indonesia, and more — frequently mixed. Every downstream use depends on correct encoding, segmentation, and meaning-preserving translation: a mistranslated variant or specification is worse than an untranslated one. This is a different processing stack from any Latin-script pipeline, and it draws on the multilingual depth from our regional-language work.
• Variant density is extreme. A single listing on these platforms routinely carries dozens of variants — sizes, colours, bundle configurations, quantity tiers — each with its own price and availability. Collecting the "listing price" and stopping captures one cell of a large matrix and mislabels it as the matrix.
• Discount layering is aggressive and platform-specific. Shop coupons, platform vouchers, flash sales, full-store discounts, bundle deals, and live-commerce pricing stack in mechanics that differ per platform and per market. The sticker price is frequently not the paid price, and computing effective price per variant is essential rather than optional — the lesson central to all our pricing work.
• Merchant fragmentation and authenticity signals. These are seller-marketplace ecosystems with enormous long tails, where official brand stores, authorised resellers, and gray-market sellers coexist. For brand-protection use cases, capturing store-type, rating, and positioning signals matters as much as the product data itself.

The Actowiz Solution

1. Public-surface scope, defined explicitly.

The engagement's boundary was set clearly at the outset and it is worth stating plainly, because it is the foundation of everything else: collection covers publicly accessible catalogue, price, availability, and merchant-listing data only. No authentication circumvention, no login-gated surfaces, no credentialed access, no personal data. This isn't a limitation we work around — it's the scope that makes the data defensible in enterprise procurement, where clients inherit their vendors' collection risk. Buyers who ask about this boundary first are the buyers worth having.

2. Per-platform extraction on self-healing infrastructure.

Distinct extraction tuned to each platform's structure and behaviour, running on our self-healing stack — output-quality watchdogs, schema-conformity checks, sudden-anomaly detection, and automatic re-mapping when a platform ships changes. This is the difference between a pipeline that survives these environments and one that needs a human every week: across the first quarter, the overwhelming majority of platform changes were absorbed without intervention.

3. Multilingual processing pipeline.

Correct encoding end to end; language and script identification per record; meaning-preserving translation of titles, variants, and key specifications into the client's working language, with original source text always retained alongside so nothing is lost and any translation can be verified. That retention is a small design choice that carries a lot of trust with clients working across five languages.

4. Full variant expansion.

Every listing's variant matrix expanded into individual records — variant identity, defining attributes, variant-specific price, availability, and imagery reference — making the variant the unit of analysis, as it must be in these catalogues.

5. Effective-price resolution per platform.

Base price, shop coupons, platform vouchers, flash-sale pricing, bundle mechanics, and full-store discounts captured and resolved into a computed effective price per variant, with the promotional context retained so genuine markdowns are distinguishable from permanent changes.

6. Merchant and authenticity signals.

Store type, rating, listing patterns, and price positioning captured per merchant — supporting the brand-protection and gray-market monitoring use cases that drive much of the demand for Asian marketplace data among Western brands.

7. Unified cross-platform schema.

Four very different platforms normalised into one schema — common fields unified for cross-platform analysis, platform-specific richness preserved in typed extensions — with per-record lineage, capture timestamps, and history retained so trends accrue.

8. Compliance documentation.

Public data only, respectful request pacing per our ethical-load standards, no personal data collected, per-record lineage, and documentation mapped for the client's own enterprise diligence — the provenance-pack standard from our compliance framework.

The Outcome

The client replaced a brittle patchwork with a dependable recurring feed across four of Asia's hardest public commerce surfaces — variant-level, effective-priced, translated with originals retained, and normalised into one schema their analysts could query across platforms and markets. The reliability was the headline: a pipeline that survives these platforms' change velocity without weekly firefighting turned Asian marketplace coverage from an ongoing engineering drain into infrastructure they could build products on.

Two things surprised them in the delivery. First, effective-price resolution changed their analysis materially — across these platforms' aggressive discount layering, sticker-price data had been systematically misrepresenting competitive positioning, and correcting it altered conclusions their clients were making about Asian pricing. Second, the original-language retention became a trust feature: their analysts working across five languages could verify any translated field against source, which mattered more than either side anticipated in a region where translation errors silently corrupt analysis.

The explicitly public-surface scope also proved commercially useful rather than limiting. When their own enterprise clients ran vendor diligence, a documented "public catalogue data, no authentication circumvention, no personal data" posture with per-record lineage cleared review — whereas a gray-area collection story would have stalled it. In 2026, that documentation is what allows the products built on the data to be sold at all.

The engagement continues with market coverage expanding across the same schema.

Why This Pattern Repeats

Western brands, investors, and intelligence businesses increasingly need visibility into Asian commerce, and Shopee, TikTok Shop, Pinduoduo, and Meituan are where that commerce happens. The transferable design: explicitly public-surface scope, per-platform extraction on self-healing infrastructure, a genuine multilingual pipeline with original-text retention, variant-level expansion, effective-price resolution per platform's discount mechanics, merchant and authenticity signals, and a unified cross-platform schema with documented provenance. Difficulty of access is exactly what makes the data valuable — and doing it within a clean, documented boundary is what makes it sellable.

Frequently Asked Questions

Can data be collected reliably from Shopee, Pinduoduo, and Meituan?

Yes, from their public surfaces, with specialist self-healing infrastructure — these are app-first, dynamic, aggressively defended environments where general-purpose scraping fails, but purpose-built extraction delivers clean recurring data.

What exactly is in scope?

Publicly accessible catalogue, price, availability, and merchant-listing data. No authentication circumvention, no login-gated surfaces, no credentialed access, and no personal data — a boundary that keeps the resulting datasets defensible in enterprise diligence.

How is the multilingual content handled?

With correct encoding, per-record language identification, and meaning-preserving translation of product fields — with original source text always retained alongside, so any translation can be verified.

Why does effective-price resolution matter so much on these platforms?

Because shop coupons, platform vouchers, flash sales, group-buy, and full-store discounts stack aggressively — sticker-price data systematically misrepresents competitive position across Asian marketplaces.

Can this support brand-protection use cases?

Yes — merchant store-type, rating, and positioning signals help distinguish official-brand from reseller and gray-market presence. Contact Actowiz Solutions to scope Asian marketplace coverage.