Best Web Scraping APIs for Indeed: July 2026 Benchmark
Benchmarks run by the Scrapeway team Β·
Last updated: August 14, 2026 Β·
How we benchmark
Scrapfly is the best web scraping API for Indeed, with a
97% success rate across 8 web scraping APIs
benchmarked against live Indeed pages in July 2026.
1 of the 8 cleared Indeed reliably enough to recommend.
Indeed is protected by Cloudflare plus its own
proprietary layer of scraping defenses, so most scraping APIs either fail it or pay for it in speed and cost.
The benchmark is refreshed twice a month, with no affiliate links and no sponsors.
Ranking history: web scraping APIs for Indeed over time
Indeed target ranking history
The 7 web scraping APIs for Indeed, reviewed
1. Scrapfly: 97% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
97%
8.9s
$8.95
#1 of 7
$30/mo
Indeed pairs Cloudflare with its own layer of scraping defenses in front of its search and job pages, and the
job data sits in an embedded JSON object (the window.mosaic / _initialData state)
rather than the visible HTML. Scrapfly cleared 97% here by
generating a genuine browser fingerprint and clearing the Cloudflare challenge, so it reaches the rendered
page and the embedded JSON instead of a challenge screen. Because it only bills for successful requests, the
block and challenge pages Indeed returns don't quietly run up the cost.
At $8.95 per 1,000 successful requests and
8.9s average response time, the figures are strong for an API capable of
clearing Cloudflare. The 97% success rate is the headline, and the ranking history
above shows how that rate has held across previous runs.
Pros:
Highest success rate in the test, clearing Cloudflare's challenge and Indeed's own layer
Only charges for successful scrapes, so challenge pages cost nothing
One asp flag plus residential proxies handles Indeed with little tuning
First class SDKs for Python, TypeScript, Go, and Rust, plus a Scrapy extension
Cons:
Credit cost per request rises once ASP, JavaScript rendering, or residential proxies are enabled
The entry (Discovery) plan caps concurrency at 5, so large search crawls need a higher tier
The free tier is a single batch of 1,000 credits, enough to prototype but not to benchmark at volume
2. Firecrawl: 79% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
79%
6.8s
$7.16
#2 of 7
$16/mo
Firecrawl's draw on Indeed is its output format. It returns structured markdown rather than raw HTML, so a
pipeline feeding an LLM or RAG system skips the parsing step on Indeed's job markup. Rendering a full browser
on every page and generating that markdown carries a cost and latency premium over lighter, HTTP based
options, so that saving has to be worth the premium for your use case. On Indeed it cleared 79%
this run.
Pros:
Returns markdown built for LLM pipelines, saving a parsing step for AI and RAG pipelines
Runs a real browser, which covers pages that need JavaScript execution
Cons:
Cost is the top user complaint in reviews
Renders a full browser on every page, which adds latency
3. Scraperapi: 75% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
75%
5.9s
$4.90
#3 of 7
$49/mo
Scraperapi is built around a fast request path. When it clears a request it tends to return quickly, which
suits latency sensitive lookups that can absorb retries. Indeed's Cloudflare layer and its own checks have to
be cleared before the embedded job JSON is reachable, and the scorecard carries how it did on that this run.
On Indeed it cleared 75% this run.
Pros:
Fast when it clears, with a simple integration
Broad language SDK support
Cons:
Login required flows and form filling are off limits
Geotargeting is gated by plan (US and EU only until the Business tier)
4. WebScrapingAPI: 72% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
72%
46.9s
$2.71
#4 of 7
$19/mo
WebScrapingAPI covers a broad range of language SDKs behind a simple REST interface, so it drops into most
stacks without a client library, and it supports async and batch submission for queued jobs. The scorecard
carries this run's speed and cost. On Indeed it cleared 72% this run.
Pros:
Broad language SDK range behind a simple REST interface
Async and batch submission for queued jobs
Cons:
Cloudflare's block pages are easy to keep paying for unless you check content
Support tickets often go unanswered for days, per user reviews
5. Zenrows: 43% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
43%
6.7s
$6.90
#5 of 7
$69/mo
Zenrows is a general purpose scraping API with JavaScript rendering and session support, positioned as a
generalist rather than a specialist on any single axis. It renders a real browser and supports sessions, and
on Indeed it cleared 43% this run. It fits when you want one general purpose tool for Indeed plus
other targets rather than optimizing hard for cost, speed, or maximum reliability.
Pros:
Real browser rendering and session support for targets that need JavaScript execution
One general purpose tool spans Indeed and other targets
Cons:
Cost climbs on heavy or large scale usage, the recurring user complaint
Premium proxy geographic coverage is unclear
6. Scrapingbee: 41% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
41%
1.9s
$3.25
#6 of 7
$49/mo
Scrapingbee is fast and cheap per request, with JavaScript rendering for lighter targets. Indeed is a hard
case for it, because Cloudflare's challenge has to be cleared in a real browser and Indeed's own layer scores
the session, and every request that gets blocked partway through still costs credits. On Indeed it cleared
41% this run.
Pros:
Fast response times
Low sticker cost per request
Cons:
Cloudflare's challenge and Indeed's own layer penalize an HTTP first approach
Credits burn quickly once JavaScript rendering or premium proxies are enabled
No plan tier between the small and large options
7. Scrapingant: 2% success on Indeed
On Indeed
Speed
Cost/1k
Overall
From
2%
55.3s
$1.90
#7 of 7
$19/mo
Scrapingant bundles JavaScript rendering and session support at a low entry price, with a smaller feature
surface than the larger providers. On Indeed it cleared 2% this run.
Pros:
Low sticker price
JavaScript rendering included
Cons:
Smaller feature surface and fewer integration options than the larger providers
Billing covers requests that return block or challenge pages, so failures still cost credits
Small provider with a thin public track record
COMPARISON
About scraping Indeed
Indeed is the largest job board on the web, and the data people scrape from it is mostly recruitment detail. That
means job titles, hiring companies, locations, salary ranges where listed, posting dates, full job descriptions,
apply links, and company ratings and reviews. Most of this lives on search results pages (the
/jobs?q=...&l=... URLs) and on individual job detail pages.
Indeed renders its pages on the server but keeps the structured job data in an embedded JSON object rather than
the visible markup. The reliable place to read it is the inline state (the window.mosaic /
_initialData variable), which holds the search results and job records in one object, so parsing that
JSON is cleaner than scraping individual DOM nodes. Search results paginate through the start URL
parameter.
HTTP ANALYSIS
Indeed is protected by Cloudflare plus its own proprietary layer of scraping defenses. See the
Cloudflare benchmark page for how Cloudflare detects
bots. The practical detail for scraping Indeed is that its blocks and challenge pages can return without a hard
error status, so success has to be measured on response content, not status codes.
indeed_scraper.py
importjsonfromparselimportSelector# install using `pip install scrapfly-sdk`fromscrapflyimportScrapflyClient,ScrapeConfig,ScrapeApiResponse# create an API client instanceclient=ScrapflyClient(key="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.scrape(ScrapeConfig(url=url,headers=headers,))returnapi_result.selector# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install scrapfly-sdk`
from scrapfly import ScrapflyClient, ScrapeConfig, ScrapeApiResponse
# create an API client instance
client = ScrapflyClient(key="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.scrape(ScrapeConfig(
url=url,
headers=headers,
))
return api_result.selector
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# install using `pip install firecrawl-py`fromfirecrawlimportFirecrawl# create an API client instanceclient=Firecrawl(api_key="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.scrape(url=url,headers=headersor{},formats=['rawHtml'],only_main_content=False,skip_tls_verification=False,timeout=150000,store_in_cache=False,max_age=900,)assertapi_result.raw_html,"firecrawl returned no html for this page"returnSelector(api_result.raw_html)# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install firecrawl-py`
from firecrawl import Firecrawl
# create an API client instance
client = Firecrawl(api_key="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.scrape(
url=url,
headers=headers or {},
formats=['rawHtml'],
only_main_content=False,
skip_tls_verification=False,
timeout=150000,
store_in_cache=False,
max_age=900,
)
assert api_result.raw_html, "firecrawl returned no html for this page"
return Selector(api_result.raw_html)
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# install using `pip install scraperapi-sdk`fromscraperapi_sdkimportScraperAPIClient# create an API client instanceclient=ScraperAPIClient(api_key="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.make_request(url=url,headers=headers,params={"render":False,},method='GET',)assertapi_result.ok,api_result.textreturnSelector(api_result.text)# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install scraperapi-sdk`
from scraperapi_sdk import ScraperAPIClient
# create an API client instance
client = ScraperAPIClient(api_key="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.make_request(
url=url,
headers=headers,
params={
"render": False,
},
method='GET',
)
assert api_result.ok, api_result.text
return Selector(api_result.text)
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# webscrapingapi has a Python SDK but it's not great, use httpx instead:# `pip install httpx`importhttpx# create an API client instanceclient=httpx.Client(timeout=180)# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.get("https://api.webscrapingapi.com/v2",# the target url is passed as a parameter belowheaders=headers,params={"url":url,"api_key":"YOUR API KEY",# NOTE: add your API KEY here!"timeout":60_000,"render_js":render_jsorFalse,"method":'GET',},)assertapi_result.status_code==200,api_result.reason_phrasereturnSelector(api_result.text)# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# webscrapingapi has a Python SDK but it's not great, use httpx instead:
# `pip install httpx`
import httpx
# create an API client instance
client = httpx.Client(timeout=180)
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.get(
"https://api.webscrapingapi.com/v2", # the target url is passed as a parameter below
headers=headers,
params={
"url": url,
"api_key": "YOUR API KEY", # NOTE: add your API KEY here!
"timeout": 60_000,
"render_js": render_js or False,
"method": 'GET',
},
)
assert api_result.status_code == 200, api_result.reason_phrase
return Selector(api_result.text)
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# install using `pip install zenrows`fromzenrowsimportZenRowsClient# create an API client instanceclient=ZenRowsClient(apikey="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.get(url,headers=headers,params={})# zenrows answers with the page HTML; "json_response" is only accepted with js_render onassertapi_result.ok,api_result.textreturnSelector(api_result.text)# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install zenrows`
from zenrows import ZenRowsClient
# create an API client instance
client = ZenRowsClient(apikey="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.get(
url,
headers=headers,
params={
}
)
# zenrows answers with the page HTML; "json_response" is only accepted with js_render on
assert api_result.ok, api_result.text
return Selector(api_result.text)
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# install using `pip install scrapingbee`fromscrapingbeeimportScrapingBeeClient# create an API client instanceclient=ScrapingBeeClient(api_key="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.get(url,headers=headers,params={"json_response":True,"transparent_status_code":True,})assertapi_result.ok,api_result.textdata=api_result.json()returnSelector(data['body'])# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install scrapingbee`
from scrapingbee import ScrapingBeeClient
# create an API client instance
client = ScrapingBeeClient(api_key="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.get(
url,
headers=headers,
params={
"json_response": True,
"transparent_status_code": True,
}
)
assert api_result.ok, api_result.text
data = api_result.json()
return Selector(data['body'])
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# install using `pip install scrapingant-client`fromscrapingant_clientimportScrapingAntClient# create an API client instanceclient=ScrapingAntClient(token="YOUR API KEY")# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:api_result=client.general_request(url,)# the scrapingant client returns its own Response object: content holds the pageassertapi_result.status_code==200,api_result.textreturnSelector(api_result.content)# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# install using `pip install scrapingant-client`
from scrapingant_client import ScrapingAntClient
# create an API client instance
client = ScrapingAntClient(token="YOUR API KEY")
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
api_result = client.general_request(
url,
)
# the scrapingant client returns its own Response object: content holds the page
assert api_result.status_code == 200, api_result.text
return Selector(api_result.content)
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
importjsonfromparselimportSelector# scrapingdog has no integration but we can use httpx# install using `pip install httpx`importhttpx# create an API client instanceclient=httpx.Client(timeout=180)# create scrape function that returns HTML parser for a given URLdefscrape(url:str,country:str="",render_js=False,headers:dict=None)->Selector:params={"api_key":"YOUR API KEY","url":url,}api_result=client.get('https://api.scrapingdog.com/scrape',params=params,)assertapi_result.status_code==200,api_result.textreturnSelector(api_result.text,type="html")# example search page url:url="https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"selector=scrape(url,country="US")# Indeed jobs can be found in Javascript variable as an array of job objects:data=selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')data=json.loads(data[0])jobs=data["metaData"]["mosaicProviderJobCardsModel"]["results"]print(len(jobs))15frompprintimportpprintpprint(jobs[0])
Output$ python indeed_scraper.py
{
'applyCount': 0,
'company': 'Pythonwise',
'companyRating': 0,
'companyReviewCount': 0,
'createDate': 1568635928000,
'jobLocationCity': 'Seattle',
'jobLocationState': 'WA',
'normTitle': 'Python developer',
'organicApplyStartCount': 1493,
'pubDate': 1568610000000,
'rankingScoresModel': {'bid': 0, 'eApply': 0.015428938, 'eQualified': 0},
'salarySnippet': {'currency': '', 'salaryTextFormatted': False},
'snippet': '
<ul style="list-style-type:circle;
margin-top: 0px;
margin-bottom: ' '0px;
padding-left:20px">
\n'
'
<li>
We are looking for a <b>Python</b> Web Developer responsible '
'for developing, enhancing, modifying, maintaining applications '
'and managing the interchange of dataβ¦
</li>
\n'
'
</ul>
',
'sourceId': 14854320,
'sponsored': False,
'viewJobLink': '/viewjob?jk=1a20a1c56fb7df73&from=vjs&tk=1hr8aknntirln85c&viewtype=embedded&xkcb=SoCQ67M3CoddJxwBkx0LbzkdCdPP&continueUrl=%2Fjobs%3Fq%3Dpython%26l%3DSeattle%252C%2BWA',
# ... and much more
}
import json
from parsel import Selector
# scrapingdog has no integration but we can use httpx
# install using `pip install httpx`
import httpx
# create an API client instance
client = httpx.Client(timeout=180)
# create scrape function that returns HTML parser for a given URL
def scrape(url: str, country: str="", render_js=False, headers: dict=None) -> Selector:
params = {
"api_key": "YOUR API KEY",
"url": url,
}
api_result = client.get(
'https://api.scrapingdog.com/scrape',
params=params,
)
assert api_result.status_code == 200, api_result.text
return Selector(api_result.text, type="html")
# example search page url:
url = "https://www.indeed.com/jobs?q=python&l=Seattle%2C%20WA"
selector = scrape(url, country="US")
# Indeed jobs can be found in Javascript variable as an array of job objects:
data = selector.re(r'window.mosaic.providerData\["mosaic-provider-jobcards"\]=(\{.+?\});')
data = json.loads(data[0])
jobs = data["metaData"]["mosaicProviderJobCardsModel"]["results"]
print(len(jobs))
15
from pprint import pprint
pprint(jobs[0])
How to choose a web scraping API for Indeed
Because Indeed pairs Cloudflare with its own layer of scraping defenses, start by narrowing to the providers still
clearing it this run (the top of the ranked table), then choose within that set based on your job. Low volume job
lookups give you more room on cost. Higher volume work puts more weight on reliability and concurrency. Our
benchmark covers job listing pages, and search result pages are more heavily rate limited, so results there may
differ.
Reliability first.Scrapfly leads the current Indeed ranking, which makes it the
default starting point for production job page crawls. The ranking history shows its track record.
Value. Among the APIs still clearing Indeed, sort by cost per successful request. The cheapest
sticker price is rarely the cheapest per usable Indeed page.
Speed. For latency sensitive job lookups rather than bulk crawls, pick the fastest option that
still clears Indeed reliably.
The key principle is to judge on cost per successful request, not sticker price. Cloudflare and Indeed's own layer
return block and challenge pages that can look like a normal response, so a cheap API can burn through requests
without returning usable data, which makes its real cost per usable result far higher than the rate card suggests.
PRICING
How we benchmark web scraping APIs for Indeed
We independently benchmark 8 web scraping APIs against live Indeed pages, 1,000+ requests per
service, twice a month. Every API is tested against the same Indeed URLs at the same time, and cost is measured
per 1,000 successful requests on entry plan pricing. We pay for the plans ourselves. No affiliate links. No
sponsors. Just data.
8 APIs Β· 1,000+ requests each Β· twice a month Β· no affiliate links, no sponsors
Benchmarking Indeed has one wrinkle worth knowing about. Success is measured on response content, not HTTP status
codes. Cloudflare's challenge pages and Indeed's own block responses don't always come with a hard error status,
so a test that only checked the status code would overstate results. We verify that responses contain the expected
job data before counting them as successful. Every provider is tested against the same URLs in the same run, so
the numbers stay comparable.
Latest data: Jul 31 to Aug 14, 2026.
QUALITY TESTING
Frequently asked questions about scraping Indeed
Is it legal to scrape Indeed?
Scraping publicly available Indeed data such as job titles and descriptions is generally treated as lower risk
than scraping data behind a login, but Indeed's Terms of Service prohibit automated access, and any personal data
in reviews or profiles can fall under privacy laws. Legality depends on what you collect, where you operate, and
how you use the data, so treat this as general information rather than legal advice and check your own situation.
What's the cheapest API that works on Indeed?
Sort the ranked table by cost per successful request and read down to the first provider still clearing Indeed
this run. That's the cheapest option that actually delivers. Lower priced APIs further down often fail too many
requests for their sticker price to be meaningful, and Indeed's blocks hide those failures unless you check
response content.
Do I need a headless browser to scrape Indeed?
Not always. Indeed's pages are mostly server rendered and the job data arrives in the embedded
window.mosaic / _initialData state, so a request that clears the antibot layer often
returns it. Our own Indeed benchmark runs without JavaScript rendering. What matters is clearing Cloudflare and
Indeed's own checks, which the APIs at the top of the ranking handle for you.
Why do some APIs score 0% on Indeed?
Because Cloudflare and Indeed's own checks both have to be cleared before the job page is returned. A client that
doesn't clear them gets block or challenge pages instead, which we score on response content as failures rather
than successes.
How often is this benchmark updated?
Twice a month against the same live Indeed targets, 1,000+ requests per API each run. We publish after validating
the run and checking failures for configuration or detection errors.
Conclusion
Indeed pairs Cloudflare with its own layer of scraping defenses and serves its job data through an embedded JSON
object, so the right web scraping API is one that clears the challenge reliably and returns the full page HTML.
For production job page crawls, start with Scrapfly, the highest success rate in the current run. For
cost or speed on lighter job page lookups, choose among the providers still clearing Indeed this run.
Whatever you pick, verify results on response content rather than status codes, because Indeed's block and
challenge pages don't always carry a hard error status. The benchmark refreshes twice a month, so check the live
Indeed results before committing.