This tutorial shows how to scrape Twitter data from X/Twitter profile, hashtag, timeline, search, or advanced-search URLs into CSV with the Twitter Data Scraper template for UScraper. You will choose the source URL, import the workflow, set the export path, run a validation batch, and check the fields before using the file.
Before you start
Prerequisites before you scrape Twitter
You need UScraper as a local desktop app, one X/Twitter URL that already loads in your browser session, a local folder for CSV output, and a clear reason for collecting the rows. Good first targets are a public profile timeline, a narrow hashtag page, a brand search URL, or an advanced-search URL with dates.
Use a small, reviewable batch first. Twitter scraping tools can fail silently when the page is virtualized, delayed, blocked, or changed by an experiment. The UScraper template is designed to make that visible: if normal tweet cards are not accessible, the export includes a diagnostic row instead of pretending the run succeeded.
A tweet being visible in a browser is a QA condition, not a full permission model. Keep a note of the URL, account state, run time, export file, and why the data was collected.
Choose route
Twitter API vs scraper vs hosted scraping tools
The best Twitter scraper tools are not interchangeable. Match the route to the job, not only to setup speed.
| Route | Best fit | Trade-off |
|---|---|---|
| UScraper template | Bounded spreadsheet exports from visible X pages | You validate browser state and keep the run supervised |
| Official X API | Sanctioned product integrations, recurring pipelines, formal endpoint behavior | Requires developer setup, access review, authentication, and API-specific limits |
| Hosted actors or scraper APIs | Remote queues, scheduled jobs, hosted datasets, webhooks | Inputs, session state, and output move through a vendor account |
| Python or browser automation | Engineering-owned extraction logic | Your team owns selectors, retries, throttling, compliance, and maintenance |
Use UScraper when the deliverable is a local CSV for research, monitoring, reporting, lead review, or a one-off social analysis task. The workflow is editable, no-code, and practical when a human can validate the first run.
Workflow
How the Twitter scraper template works
The related Twitter Data Scraper template is the download path. The export JSON is the authoritative workflow definition; the summary below explains the extraction intent.
{
"project": {
"name": "Twitter Scraper",
"description": "Best-effort X/Twitter scraper for tweet/search/profile timelines."
},
"blocks": [
{ "title": "Navigate", "config": { "url": "https://x.com/X" } },
{ "title": "Inject JavaScript", "config": { "waitForCompletion": false } },
{
"title": "Structured Export",
"config": {
"rowSelector": "#uscraper-tweet-dataset .tweet-row",
"fileName": "twitter-scraper.csv",
"includeHeaders": true,
"fileMode": "create"
}
}
]
}
The default Navigate block opens https://x.com/X. Replace it with an approved profile, hashtag, search, or advanced-search URL. The injected collector scrolls through multiple batches, stores visible tweet cards in a hidden dataset, de-duplicates rows as X virtualizes the page, and then Structured Export writes the normalized CSV.
twitter-scraper.csvColumn
page_status
ok, blocked_or_login_required, or no_visible_tweets.
Column
blocked_reason
Captured page text when normal tweet rows are not visible.
Column
profile_name
Visible profile name when the source is a profile page.
Column
profile_handle
Detected profile handle.
Column
profile_followers
Visible follower label when exposed.
Column
profile_following
Visible following label when exposed.
Column
author_name
Tweet card display name.
Column
handle
Tweet author handle.
Column
timestamp
Datetime from the tweet time element.
Column
tweet_text
Visible tweet text normalized into one cell.
Column
tweet_url
Status URL when exposed.
Column
replies_count
Visible reply count label.
Column
reposts_count
Visible repost or retweet count label.
Column
likes_count
Visible like count label.
Column
views_count
Visible view count label.
Column
bookmarks_count_or_label
Bookmark count or label when visible.
Column
media_urls
Visible media URLs joined into one cell.
Column
captured_page_url
The X URL opened for the run.
Runbook
Step-by-step: scrape tweets from X to CSV
Open the template
Go to the Twitter Data Scraper template and import the JSON workflow into UScraper.
Choose one source URL
Start with a profile, hashtag, search, or advanced-search URL that loads normally in your browser. Avoid broad searches for the first run.
Replace the Navigate URL
Edit the Navigate block so it opens your approved X URL instead of the default sample page.
Set the export folder
Confirm the Structured Export save folder and keep twitter-scraper.csv, or rename it for the campaign, client, or research topic.
Run a validation batch
Run one URL, open the CSV, and confirm that rows contain tweet text, timestamps, tweet URLs, and expected diagnostics.
Scale only after review
Add more URLs or repeat the run only after the validation file matches what the browser showed.
Validation
Validate the Twitter CSV before analysis
Open the first export before you use it in a report, spreadsheet, dashboard, or model annotation workflow. Check that each important row has a tweet URL, timestamp, handle, and text. Treat blank engagement counts as missing display data, not zero.
| Symptom | Likely cause | Fix |
|---|---|---|
| Only one diagnostic row | X showed login, CAPTCHA, rate limit, or no visible tweet cards | Stop, review the browser state, and do not bypass restrictions |
| Tweet text is blank | The card was media-only, hidden, or not fully rendered | Increase wait time only after a manual page check |
| Duplicate-looking rows | Infinite scroll reloaded similar cards or the source page changed | Deduplicate by tweet_url or timestamp plus text |
| Counts are blank | X did not expose the count label in the current session | Mark as missing display data, not zero |
| Wrong export folder | Structured Export still points to the default path | Update the save location before the next run |
FAQ
Common Twitter scraping issues
X/Twitter content can be visible in a browser, but automated collection may still be limited by platform terms, robots directives, privacy law, copyright, access controls, and local regulations. Review the current rules, use approved access where required, and do not bypass login, CAPTCHA, verification, private content, rate limits, or technical restrictions.
Next step
Download the Twitter scraper template
Start with the Twitter Data Scraper template, run one narrow validation URL, and keep the raw CSV separate from any cleaned analysis file. For adjacent workflows, browse the UScraper template library or read more tutorials in the UScraper blog.