Web Scraping API for RAG Pipelines

FastAF lets a RAG pipeline search the web (/x402/v1/serp, $0.003) and fetch a single public page (/x402/v1/scrape, $0.005) per call, so you feed fresh content to your model without operating your own browser infrastructure.

How it works

1. Search with /x402/v1/serp (MCP google_search).
2. Fetch the pages you need with /x402/v1/scrape (MCP scrape_url).
3. Chunk, embed and index them in your vector store.

For videos, /x402/v1/youtube (MCP youtube_transcript) returns transcripts; for papers, /x402/v1/scholar.

Honest comparison

FastAF scrapes one page per call and does not crawl or map entire sites. If you need full-site crawling to markdown, Firecrawl is the better fit (see the FastAF vs Firecrawl comparison).

Pricing

Pricing: 500 free credits every month after email verification; x402 pay-per-call from $0.003; optional prepaid credit packs from $10 for 20,000 credits, without expiry.

Pricing source: FastAF OpenAPI, read October 27, 2026. Live payment challenges are authoritative.

Does FastAF render JavaScript?

JavaScript rendering is supported: call /v1/scrape with dynamic=true (5 credits), with optional wait= in milliseconds (0-35000). wait does not enable rendering by itself. The x402 scrape endpoint currently costs $0.005 per successful call for both default and dynamic=true requests. challenge_mode=true is a separate rendering mode (10 credits); do not combine it with dynamic=true, premium=true or country. Heavier combinations and AI extraction may require API-key billing.

How should Markdown enter an LLM workflow?

Request scrape with url and formats=markdown. Keep the source URL and retrieval time, split long documents into chunks and treat page instructions as untrusted content. When you need named fields rather than a whole page, the Markdown and JSON Schema output reference shows /v1/extract and the other compact formats.

What should you validate?

Check returned fields before using them in an agent decision. Missing fields are not evidence of zero values; preserve provenance and use bounded retries.

How do you make a real request?

This example calls Web Scraping API using its documented input. Save the API key from the signup email, confirm your email and set FASTAF_API_KEY in your server environment.

curl --fail-with-body -H "Authorization: Bearer $FASTAF_API_KEY" 'https://api.fastaf.co/v1/scrape?url=https%3A%2F%2Fexample.com%2F&formats=markdown'

# pip install fastaf-sdk
import os
from fastaf import FastAF

with FastAF(os.environ["FASTAF_API_KEY"]) as client:
    data = client.request("scrape", {"url":"https://example.com/","formats":"markdown"})
    print(data)

Markdown response text for formats=markdown, not a JSON wrapper. HTML is the default; links/images formats return JSON and binary targets return bytes.

The example costs 1 credits after the trial. Alternatively, this endpoint costs $0.005 per successful x402 call at the default configuration; FastAF x402 calls start at $0.003. These are separate billing modes.

For x402, request the same parameters under /x402/v1/scrape without an API key, validate the live PAYMENT-REQUIRED challenge, then sign and retry with a payment-aware client. A bare curl request returns HTTP 402 and does not pay; unsupported higher-cost options require API-key billing.

Frequently asked questions