Curated Medical Web Datasets for a Healthcare Small Language Model (SLM)

About the Client

A healthtech company building a Small Language Model for patient-facing health information and provider-facing drug reference — a deliberately compact model designed to run cheaply, answer within a bounded medical domain, and be auditable in a way frontier general models are not. Their thesis matched the industry's 2026 direction: medical teams increasingly deploy specialized SLMs, and these models don't need trillions of generic tokens — they need hyper-specific, quality-controlled vertical data.

The Challenge

• Quality is a safety issue. In retail, a noisy record wastes a token; in health, low-quality content trains a model that gives dangerous answers. The corpus had to be built from authoritative, credentialed sources — and provably so.
• The web is polluted with health misinformation and AI-generated filler. SEO-farm "health articles" now dominate raw crawls of medical queries. Filtering them out wasn't a curation nicety; it was the core of the job.
• PII risk is severe. Patient stories, forum posts, and review content are dense with health information about identifiable individuals — the most sensitive personal-data category under GDPR, DPDP, and US health-privacy frameworks. The client's counsel required that such data never enter the pipeline at all.
• Structure matters more than volume. Drug references, interaction data, dosage formats, and condition taxonomies needed to arrive as structured records mapped to standard vocabularies — not prose soup.

The brief: a curated, provenance-tracked medical web corpus — public authoritative content plus structured drug/condition reference data — sized for SLM training, with a compliance pack that would survive both vendor diligence and a health-regulator conversation.

The Actowiz Solution

1. Source whitelisting, not open crawling

With the client's medical advisors, we built a whitelist-first source universe: government health portals, public regulatory drug databases, medical societies' public patient-education content, public formulary and pricing pages, and peer-reviewed open-access repositories. Open-web health content was excluded by default — inverted from the usual crawl-then-filter approach.

2. Credibility scoring

Every source and record carried a credibility tier (regulatory > society/institutional > publisher), letting the training team weight the corpus explicitly rather than trusting a blend.

3. Aggressive AI-content and misinformation filtering

Classifier-based AI-text detection plus structural signals (template farms, publication-burst patterns) ran on all publisher-tier content; flagged material was excluded, with the exclusion log delivered for audit.

4. PII exclusion by design

Forum content, patient reviews, and testimonial pages were out of scope entirely — not collected-and-masked, but never collected. Residual PII scanning ran on everything else as a safety net, with edge-level masking as the standing control.

5. Structuring to standard vocabularies

Drug records normalized to standard identifiers, conditions mapped to standard clinical terminologies, dosage strings parsed to typed fields — so the SLM's reference behavior could be grounded in structured lookups, not just prose patterns.

6. Refresh where medicine moves

Drug pricing, availability, and regulatory-status content refreshed on schedule; stable educational content archived point-in-time. Two cadences, one corpus.

The Outcome

The client's SLM shipped to its first provider-side pilot with a data story their compliance officers could defend line by line: whitelisted sources, credibility tiers, exclusion logs, zero patient-derived content. Their team reported that the structured drug-reference layer became the model's differentiator — grounding dosage and interaction answers in typed records rather than paraphrased prose — and the engagement expanded to a second corpus: multilingual public health-education content for a regional-language rollout.

The pattern generalizes beyond health: legal, finance, and medical SLMs all live or die on vertical data quality — the thesis we laid out in our broader AI training data work now proven in the hardest vertical first.

Frequently Asked Questions

Can medical data be scraped compliantly for AI training?

Public, authoritative, non-personal medical content — regulatory databases, institutional patient education, public formularies — can be collected with whitelist-first sourcing and strict PII exclusion. Patient-derived content is the red line; in this engagement it was never collected at all.

Why an SLM instead of a large general model for healthcare?

Compact vertical models are cheaper to run, easier to audit, and — when trained on curated domain data — more reliable within their bounded domain, which is exactly what clinical settings demand.

How is health misinformation kept out of the corpus?

By inverting the pipeline: whitelist-first sourcing, credibility tiers, and AI-content/misinformation classifiers on publisher content, with exclusion logs delivered for audit.

Can the same approach work for legal or finance SLMs?

Yes — whitelisted authoritative sources, structured reference layers, and provenance-first delivery transfer directly. Contact Actowiz Solutions to scope a vertical corpus pilot.