JavaScript rendering in web scraping: when you need it and what it costs

A plain HTTP fetch gets you the HTML the server sent. Increasingly, that isn’t the page a visitor sees. The gap between the two is JavaScript rendering, and it’s the single biggest reason a scraper that works on one site fails silently on the next.

How do you tell if a page needs rendering?

Two checks, both quick, before you write any scraping code:

Compare view-source to the DOM. In a browser, view-source (Ctrl+U or Cmd+Option+U) shows exactly what a plain fetch would receive: the raw HTML from the server, before any script runs. Now open dev tools and look at the Elements panel, which shows the DOM after JavaScript has run. If the content you want, prices, listings, reviews, is present in Elements but missing or replaced by a loading skeleton in view-source, the page is building that content client-side.

Fetch it yourself without a browser. The same check from code, no dev tools needed:

import requests

resp = requests.get("https://example.com/products", timeout=30)
print(len(resp.text))
print("add-to-cart" in resp.text)  # swap for a string you know is in the real content

If a marker string you can see on the live page is absent from resp.text, a plain fetch isn’t getting you the content, and you need something that executes JavaScript before reading the page.

A quick reason this happens more than it used to: modern frameworks (React, Vue, Svelte and friends) ship a near-empty HTML shell and build the actual page in the browser. That’s a deliberate architecture choice for the site, not an anti-scraping measure, but it produces the same result for you either way.

Why does rendering cost more, everywhere?

A plain fetch is a socket, a request, a response. Rendering means standing up something closer to a full browser: loading the DOM, executing scripts, waiting for network calls to settle, sometimes scrolling to trigger lazy-loaded content. That’s real CPU and memory per page, held open for the duration of the render, not the microseconds a static fetch takes.

This cost structure isn’t specific to any one vendor. Every provider that offers both a static fetch and a rendered fetch prices the rendered one higher, because it’s genuinely more expensive infrastructure to run at scale: a pool of real browser processes instead of a connection pool. If a quote for rendered scraping looks the same price as static, look closely at what “rendered” means in their docs.

What rendering costs on FastAF

/scrape prices in four tiers, and the multiplier tracks the cost of what’s actually running behind the request:

Mode	Parameter	Credits
Static fetch	(default)	1
JS rendering	dynamic=true	5
Premium proxies, no rendering	premium=true	10
JS rendering + premium proxies	dynamic=true&premium=true	25

Static is a 5x jump to add rendering, and a 25x jump to add both rendering and premium proxy routing on top. That’s the real cost of the extra infrastructure, not an arbitrary markup, which is why it’s worth checking whether you need rendering at all before defaulting to it.

The strategy: try static first, escalate only on failure

Most scraping jobs mix pages that need rendering with pages that don’t, even within the same site. Paying for dynamic=true on every request when only some pages need it is the most common way to overspend on this endpoint:

import requests

API_KEY = "faf_live_..."  # your FastAF key

def scrape(url, dynamic=False, premium=False, wait=None):
    params = {"url": url}
    if dynamic:
        params["dynamic"] = "true"
    if premium:
        params["premium"] = "true"
    if wait:
        params["wait"] = wait
    resp = requests.get(
        "https://api.fastaf.co/v1/scrape",
        params=params,
        headers={"Authorization": f"Bearer {API_KEY}"},
        timeout=90,
    )
    resp.raise_for_status()
    return resp.text

def scrape_smart(url, marker):
    html = scrape(url)
    if marker in html:
        return html  # static was enough, 1 credit spent

    html = scrape(url, dynamic=True)  # escalate, 5 credits spent
    return html

marker is a string you know appears in the real content, a product title, a price format, anything that would be missing from a shell page. If the static fetch already has it, you saved 4 credits on that URL. If it doesn’t, you escalate once, not on a guess.

Reach for premium=true separately: it’s about proxy quality and getting past IP-based blocking, not about JavaScript. A site can need one, both, or neither. wait (milliseconds) is worth setting explicitly on dynamic=true requests for pages that load content after an API call finishes, rather than assuming the default wait is enough.

The short version

Check view-source against the rendered DOM, or fetch the page yourself, before assuming you need a browser. Static is 1 credit, rendering is 5, premium proxies are 10, both together are 25: the multipliers exist because real browser infrastructure genuinely costs more to run. Try static first and escalate to dynamic=true only on the URLs where the marker content is actually missing, and you’ll pay the rendering premium only where it’s earned.