Pulling papers, authors and citation counts from Google Scholar with Python

You’re building a literature review, or a dashboard that tracks a research topic over time, and you need the papers: title, authors, year, how many times each one has been cited. The default way to get that is a browser tab on Google Scholar and a spreadsheet you fill in by hand, one page of results at a time. Google Scholar has no official public API, so most tooling around it is exactly that: a person copying rows.

The request

A call to /v1/scholar takes the same query you’d type into the search box, plus a few optional parameters: language, page, results, and as_ylo / as_yhi if you want Scholar itself to restrict the response to a year range before anything comes back to you.

It runs close to eight seconds, so don’t wait on it

In measurements we ran on August 27, 2026, /v1/scholar had a median response time around 7.9 seconds, the slowest endpoint we sell. That’s not a bug to route around, it’s what a large literature index costs to query. Design for it: fetch a batch of topics in a background job or a queue, write the JSON to your own store, and have your dashboard or export read from that store. Don’t put this endpoint in a request path where a person is watching a spinner.

Papers, years and citations from the same JSON

Here’s a request for “web scraping api” that filters to papers from 2020 on and sorts what’s left by citation count:

import re
import requests

API_KEY = "faf_live_..."  # your FastAF key

resp = requests.get(
    "https://api.fastaf.co/v1/scholar",
    params={"query": "web scraping api"},
    headers={"Authorization": f"Bearer {API_KEY}"},
    timeout=60,
)
resp.raise_for_status()
papers = resp.json()["scholar_results"]

def citation_count(paper):
    total = paper.get("inline_links", {}).get("cited_by", {}).get("total", "")
    match = re.search(r"\d+", total)
    return int(match.group()) if match else 0

def pub_year(paper):
    match = re.search(r"\b(19|20)\d{2}\b", paper.get("displayed_link", ""))
    return int(match.group()) if match else None

recent = [p for p in papers if (pub_year(p) or 0) >= 2020]
recent.sort(key=citation_count, reverse=True)

for p in recent:
    authors = ", ".join(a["name"] for a in p.get("authors", []))
    print(f'{citation_count(p):>5}  {pub_year(p)}  {p["title"]}')

We ran exactly this while writing the post (August 2026). Ten papers came back for 5 credits; six were from 2020 or later, and sorting by citations put a 2021 survey on web scraping techniques on top with 651 citations, ahead of a 2024 paper at 14 and a 2025 paper that had no citation count at all, which the code above correctly scores as zero rather than crashing on it.

What’s solid, and what you have to compute yourself

Each entry in scholar_results gives you title, title_link, a snippet, and an authors list of {name, link, author_id} when Scholar attaches named authors to the result at all (books and a few of the results in our sample came back with an empty list). Two things people expect aren’t there as clean fields. There’s no standalone year: it’s embedded in the free-text displayed_link string along with the venue, which is why the code above pulls it out with a regex instead of reading a key. And there’s no plain integer citation count either, just inline_links.cited_by.total, a string that reads “Cited by 418” for most results but read “Related articles” for one entry in our sample, meaning that result has no citation number to show at all. Treat both as best-effort text pulled from a citation string, not guaranteed structured fields, and write the fallback for “missing” before you write the happy path. When a result does have a PDF available, it shows up under resources.

The same query does not return the same number of papers

This one will bite you if you assume it away. On 27 August 2026 we sent the identical query, graph neural networks, six times in a row through /v1/scholar. The result counts came back 5, 10, 10, 8, 0 and 10. One call in six returned nothing at all, and it was charged like any other call, because the request itself succeeded: Scholar simply had nothing to give at that moment.

So do not treat an empty list as an answer, and do not treat the count as stable. Retry an empty result once after a short pause before recording “no papers found”, and if you are diffing today’s results against last week’s, compare by paper rather than by count, or you will generate alerts about papers appearing and disappearing that were never really gone.

import time


def search_with_retry(query, attempts=2):
    for attempt in range(attempts):
        papers = search(query)          # the function from the previous section
        if papers:
            return papers
        time.sleep(2)
    return []

Budget for the retries: at 5 credits a call, a one-in-six empty rate adds roughly 17 percent to the cost of a run that retries every blank.

What it costs

/v1/scholar is 5 credits per call, the same regardless of how many papers come back. The free tier’s 500 one-time credits cover 100 calls, enough to check field coverage against your own topics. On the Scale pack ($99 for 300,000 credits) that’s 60,000 calls, $1.65 per 1,000.

The short version

Treat /v1/scholar as a batch job, not a live lookup: queue the topics, cache the JSON on your side, and let your own UI do the instant filtering and sorting that the endpoint itself is too slow to do on demand. Get 500 free credits, no card and run it against a topic you actually track.