Content Protection

Scrapers and AI crawlers fetch your pages all day, and the worst of them never run JavaScript — so client-only analytics and most bot tools never see them. You end up guessing which traffic is real and which is quietly draining your content into someone else’s index or training set.

Content Protection pairs a lightweight beacon with an edge model that fingerprints visitors and names the scrapers and AI crawlers hitting your content — including the ones that never run a single line of JavaScript. Every hit becomes a scored observation with a decision and a list of reasons you read in the dashboard Logs.

When to use it

Reach for Content Protection when you want to know who is fetching your pages, not just how many requests came in.

The beacon and server reporting share one publishable key and feed the same observation stream. Crawler-heavy content sites run both: the beacon scores real humans, server reporting catches everything that skips JavaScript.

How it works

1. Drop the beacon on your pages

   Add one async <script> tag pointing at https://api.formshield.dev/js/formshield.js with your publishable key and data-fs-mode="pageload". It auto-initializes from the data-fs-* attributes — no extra code.

   <script
     async
     src="https://api.formshield.dev/js/formshield.js"
     data-fs-project-key="fs_pub_live_…"
     data-fs-action="pageview"
     data-fs-mode="pageload"
   ></script>

   On load it performs a signed handshake (POST /v1/handshake), then posts browser fingerprint and automation signals to POST /v1/collect on each pageview.

2. Catch the non-JS crawlers server-side

   Pure crawlers fetch your HTML without running scripts, so the beacon never sees them. Report each request from your origin worker or backend with POST /v1/report, passing the visitor’s UA and IP.

   Fire it with ctx.waitUntil (fire-and-forget) so it adds zero latency and your page never depends on FormShield being up. See server reporting for the complete Worker example.

3. Read named, verified verdicts in your logs

   The edge model scores every hit, classifies the user agent, and checks IP reputation. It names the bot (bot_id like gptbot or googlebot, plus the operating company) and, for operators that publish IP ranges — Google, Microsoft, OpenAI, DuckDuckGo — verifies the request really came from them.

   A forged Googlebot from the wrong IP is flagged bot:spoofed. View the score, decision, and reasons per observation in the dashboard Logs.

Quickstart

Once the beacon is on your pages, add server reporting to catch the crawlers it can’t see. Report each origin request with the visitor’s UA and IP.

curl -X POST https://api.formshield.dev/v1/report \
  -H "Authorization: Bearer fs_pub_live_…" \
  -H "Content-Type: application/json" \
  -d '{
    "ua": "Mozilla/5.0 (compatible; GPTBot/1.1; +https://openai.com/gptbot)",
    "ip": "203.0.113.42",
    "hostname": "example.com",
    "path": "/pricing",
    "action": "pageview"
  }'

Response:

{ "ok": true, "request_id": "rpt_a1b2c3d4e5f6" }

/v1/report returns an acknowledgement, not a verdict. Scoring happens server-side and the score, decision, and reasons land on the observation — read them in Logs. Never gate your response on this call.

Endpoints

The beacon and server reporting use one publishable key (fs_pub_live_…), safe to expose in the browser.

Signals

Each observation is scored from these signals. They combine user-agent classification with self-hosted IP intelligence.

Key reasons

Common questions

Next steps