free tools

diagnose a site
before you scrape it.

Lightweight scanners for URL access, WAF signals, structured data, robots.txt policy, and sitemap discovery. Built for teams deciding whether direct requests, browser rendering, or monitoring workflows make sense.

run scrapability testview api docs
toolbox

public diagnostics for messy web data

diagnosticshttp + browser fit

Scrapability test

Diagnose whether a URL is reachable with direct HTTP, needs browser rendering, or shows anti-bot friction.

open tool ->
access signalscdn / waf hints

WAF detector

Check headers, cookies, status codes, and page signals for common WAF and CDN fingerprints.

open tool ->
extractionschema + metadata

Structured data extractor

Preview JSON-LD, OpenGraph, canonical tags, and product-like fields before writing an extractor.

open tool ->
compliancecrawl policy

Robots.txt analyzer

Parse robots.txt directives, sitemap links, crawl-delay hints, and user-agent groups for a domain.

open tool ->
discoveryurl inventory

Sitemap extractor

Find sitemap URLs, nested indexes, lastmod values, and sample URLs for catalog discovery.

open tool ->