Token-aware web scraping for AI agents: get only the content your LLM needs

Answer first

Raw HTML wastes tokens. For agents, return structured JSON (SERP, Maps, Amazon, YouTube) or cleaned page text instead of the full DOM, and fail fast on slow sites instead of losing the run.

Patterns

1. Prefer a structured endpoint over scrape when one exists (e.g. /x402/v1/amazon instead of scraping the product page).
2. Use /x402/v1/scrape for everything else; pass only the fields you need to the model.
3. Set a client timeout and retry once; pay-per-call means a retry costs cents, not a plan.
4. Over MCP (https://api.fastaf.co/mcp), let the agent pick the tool by name.

Why FastAF

Pay per call from $0.003 via x402, no subscription; or use 500 free credits every month after email verification. Start at https://fastaf.co.

Technical reference, with parameters, credits and examples: token-efficient web data for AI agents.

Related: Web scraping API · Google SERP API · Playwright MCP token cost vs Markdown (context window measurements) · Glossary

How do you make a real request?

This example calls Web Scraping API using its documented input. Save the API key from the signup email, confirm your email and set FASTAF_API_KEY in your server environment.

curl --fail-with-body -H "Authorization: Bearer $FASTAF_API_KEY" 'https://api.fastaf.co/v1/scrape?url=https%3A%2F%2Fexample.com%2F&formats=markdown'

# pip install fastaf-sdk
import os
from fastaf import FastAF

with FastAF(os.environ["FASTAF_API_KEY"]) as client:
    data = client.request("scrape", {"url":"https://example.com/","formats":"markdown"})
    print(data)

Markdown response text for formats=markdown, not a JSON wrapper. HTML is the default; links/images formats return JSON and binary targets return bytes.

The example costs 1 credits after the trial. Alternatively, this endpoint costs $0.005 per successful x402 call at the default configuration; FastAF x402 calls start at $0.003. These are separate billing modes.

For x402, request the same parameters under /x402/v1/scrape without an API key, validate the live PAYMENT-REQUIRED challenge, then sign and retry with a payment-aware client. A bare curl request returns HTTP 402 and does not pay; unsupported higher-cost options require API-key billing.

Frequently asked questions