Twitter / X Scraper API

Extract profile and search as structured JSON from public Twitter / X targets. Queue a request, then retrieve the result with its responseId.

Get your API key Read the documentation1,000 free credits per month, no card required.

Decide before you build

Endpoint: POST /scrape/data-scraper
Scrapers: twitterProfile twitterSearch
Cost: 30 credits / call
Free plan: 1,000 credits / month
Call pattern: async, POST then GET
On failure: credits refunded
Response format: JSON
Concurrency: by plan tier

Launching a Social Media scrape costs 30 credits. Retrieving a pending result costs nothing; failed scrapes are refunded.

Try the Twitter / X scraper on your own targetRun a request against a public target and inspect the JSON before you integrate.

Open the live test

What you get back

twitterProfile

Profile

Use twitterProfile to retrieve profile data from a public Twitter / X target.

Required input: url
Optional input: number_of_tweets
Structured JSON once the result is ready
Public data only; no account login is used

twitterSearch

Search results

Use twitterSearch to retrieve search results data from a public Twitter / X target.

Required input: search
Optional input: posts_max_count
Structured JSON once the result is ready
Public data only; no account login is used

Call the Twitter / X scraper

Send the scraper name with the input required by that scraper. The POST returns a responseId to poll for the final JSON.

Step 1Submit a twitterProfile request and keep the responseId returned by the API.

bash

# POST - queue a public social media scrape
curl -X POST "https://api.scraping-bot.io/scrape/data-scraper" \
  -u "yourUsername:yourApiKey" \
  -H "Content-Type: application/json" \
  -d '{
  "scraper": "twitterProfile",
  "url": "https://example.com/public-page",
  "number_of_tweets": 30,
  "search": "web scraping",
  "posts_max_count": 100
}'

Step 2Wait a few seconds, then retrieve the twitterProfile result with the responseId. Repeat the GET while the status is pending.

bash

# GET - retrieve the result after the POST returns a responseId
curl -G "https://api.scraping-bot.io/scrape/data-scraper-response" \
  -u "yourUsername:yourApiKey" \
  --data-urlencode "responseId=RESPONSE_ID" \
  --data-urlencode "scraper=twitterProfile"

Code example

Authenticate with your ScrapingBot username and API key, then poll until the queued result is ready.

const auth = "Basic " + Buffer.from(
  "yourUsername:yourApiKey",
).toString("base64");

const queued = await fetch("https://api.scraping-bot.io/scrape/data-scraper", {
  method: "POST",
  headers: { "Content-Type": "application/json", Authorization: auth },
  body: JSON.stringify({
  "scraper": "twitterProfile",
  "url": "https://example.com/public-page",
  "number_of_tweets": 30,
  "search": "web scraping",
  "posts_max_count": 100
}),
});

const { responseId } = await queued.json();
let result;
do {
  await new Promise((resolve) => setTimeout(resolve, 5000));
  result = await fetch(
    "https://api.scraping-bot.io/scrape/data-scraper-response?" +
      new URLSearchParams({ responseId, scraper: "twitterProfile" }),
    { headers: { Authorization: auth } },
  ).then((response) => response.json());
} while (result.status === "pending");

Run this call in two minutesCreate an account, copy your API key, then use the example above.

Get your API key

Request parameters

Parameter	Type	Default	Description
`scraper`	string	—	One of the scraper names supported by this endpoint.
`url`	string	—	Public page URL, including https://.
`number_of_tweets`	int	30	Optional for this scraper.
`search`	string	—	Search term for the Twitter / X search scraper.
`posts_max_count`	int	100	Optional for this scraper.

Response example

Example returned by the legacy Twitter / X scraper documentation.

json

[
  {
    "profile_name": "Elon Musk",
    "isVerified": false,
    "bio": "nothing",
    "location": "A Shortfall of Gravitas",
    "following": 235,
    "followers": 135942980,
    "website_url": "",
    "posts": 24929,
    "media_count": 1511,
    "profile_background_image_url": "https://pbs.twimg.com/profile_banners/44196397/1576183471",
    "profile_image_url": "https://pbs.twimg.com/profile_images/1590968738358079488/IY9Gx6Ok_normal.jpg",
    "created": "Tue Jun 02 20:12:29 +0000 2009",
    "handle": "elonmusk",
    "collected_number_of_posts": 30,
    "posts_info": [
      {
        "text": "Great song 
https://t.co/TpFcobGrch",
        "time": "Fri Apr 21 03:49:33 +0000 2023",
        "id": "1649259048892497922",
        "replies": 3533
      }
    ]
  }
]

Credits, failures and limits

Case	Credits	Notes
Launch a scrape	30 credits	Consumed when a result is ready.
Result not ready	Not consumed	The API returns a pending status; retry after a few seconds.
Failed, blocked or timed-out scrape	Refunded	Credits are returned for an unsuccessful request.

Twitter / X API or Raw HTML scraper?

	Twitter / X API	Raw HTML scraper
Endpoint	/scrape/data-scraper	/scrape/raw-html
Returns	Profile and search JSON	Rendered page HTML
Flow	Asynchronous: POST then GET	One synchronous request
Starting cost	30 credits	1 credit
Use it when	You need parsed Twitter / X data	You need your own extraction logic

Questions about the Twitter / X API

Which Twitter / X scraper names can I use?: This page supports twitterProfile and twitterSearch. Send the exact name in the scraper field.
Why does the API return a responseId?: Social Media requests are asynchronous. The responseId lets you retrieve the result without holding the initial request open.
What if the result is still pending?: Wait a few seconds and repeat the GET request. A pending response does not consume credits.
Does this scraper access private account data?: No. It is intended for public pages and does not sign in to a social media account to access private data.

Source: ScrapingBot API documentation.

Other social media scrapers

Instagram Scraper API Instagram Profiles Scraper API Instagram Posts Scraper API TikTok Scraper API TikTok Profiles Scraper API TikTok Hashtags Scraper API

Browse the Social Media category for the complete list of guides, or visit the ScrapingBot blog for implementation guides.

4.9

★★★★★

133 Google reviews

Trusted by developers and social media teams.
Customer reviews apply to ScrapingBot's service overall, not specifically to this scraper.

Start scraping Twitter / X today

1,000 free credits every month. No card required, cancel anytime.

Get your API key Talk to us