How to build a competitor catalog monitor that survives website changes

Competitor monitoring fails when it begins with a vague request to track everything. Begin with decisions: respond to price cuts, identify new assortment, investigate a stockout, or track promotions before a campaign. Each decision determines an alert and the evidence a reviewer needs.

A price field alone is rarely enough. Capture the regular price, sale price, currency, availability, product URL, product or variant identifier, and capture time. Add shipping threshold or promotional copy only when it changes the commercial decision.

The matching rule is the heart of a monitor. A product URL may be enough for a small catalogue, but a SKU, GTIN, or manufacturer part number is stronger when URLs change or similar products have nearly identical names. Preserve the raw title too; it is useful when a match needs review.

Decide whether each variant is a separate row. Size, color, pack count, and subscription option can have different prices and availability. If the client buys or compares at variant level, the monitor should too.

HTML changes constantly: tracking tags, badges, review counts, and layout adjustments can all create noise. Compare normalized fields instead. Convert currencies consistently when that is part of the brief, retain both regular and sale price, and record the raw value for auditability.

Keep the previous and current values beside the source URL. A reviewer should be able to open the relevant product page and understand why the alert exists without searching through an old spreadsheet.

Not every disappearance means a product was discontinued. It may have moved category, become temporarily unavailable, changed URL, or failed to load. Use an unknown or review-needed state before declaring a commercial conclusion.

Run a small quality sample after a site redesign or a new template appears. Check totals, duplicate products, the last page of a collection, and a handful of variants. A monitor that reports uncertainty clearly is more valuable than one that confidently reports the wrong change.

Group changes by priority. A weekly report might lead with meaningful price movements, new competitors in a category, and stock changes that affect a campaign; lower-priority cosmetic changes can stay in the dataset.

The deliverable becomes actionable when it includes a recommendation: match a price, update a promotion, review a new bundle, or simply watch a developing trend. The monitor supplies the evidence; the commercial team supplies the judgment.