IP Intelligence playbook
Detect commercial proxies hammering your pricing pages
Pricing pages, inventory feeds, and product catalogs are the most-scraped surfaces on e-commerce sites. The traffic almost never converts and frequently feeds a competitor pricing engine that re-undercuts within minutes. Robots.txt does nothing here. Login walls break the user experience. The right answer is to recognise the traffic at the IP layer and serve it a non-live snapshot.
This playbook combines three IP Intelligence fields: is_datacenter, is_proxy, and is_residential_proxy. Datacenter and proxy hits get a static cached page. Residential-proxy hits get a tighter rate limit per /24 because that is how commercial pools rotate. Legitimate search crawlers (service.kind === "search-crawler") stay on the fast path.
Try it now
Free tier returns the same shape. Sign in for a key, replace the placeholder, watch the response.
curl -H "Authorization: Bearer dk_live_..." \ "https://dnschkr.com/api/v1/ip?address=$REQ_IP" # Branch on the response: # data.is_datacenter || data.is_proxy # -> serve a static, non-live snapshot (no real-time pricing) # data.is_residential_proxy # -> stricter per-/24 rate limit; CAPTCHA gate # data.service.kind === "search-crawler" # -> allow (Googlebot, Bingbot, etc.)
What this catches
The cleanest signal for scraping volume is hosting-ASN traffic on consumer-product URLs. Real shoppers do not browse from datacenter egress IPs. Public proxy lists are added on top. Commercial residential-proxy networks are the trickiest tier but DNS Checker tracks the major ones (Bright Data, Smartproxy, Oxylabs) where the egress ASN is publicly identifiable.
Service identification matters here. Googlebot, Bingbot, AhrefsBot, and other legitimate crawlers should reach live content. The service object in the response identifies them by reverse DNS plus IP-range membership against the published ranges, so the rule is not just an ASN match.