How To Scrape Tweets With Python

Welcome back! Python is an awesome programming language with a ton of capability, if you’re new to Python, check out the link below to learn more about it:

So, let’s take a look at an awesome way to scrape tweets with Python! This specific method is by using the Twint package, here is a link to their GitHub repository:

This specific tool allows us to scrape tweets without using the Twitter API, this allows us to get as much data as we want without restrictions (most of the time). To start using this package, you can use the following pip command:

pip3 install twint

You can also clone this repository using the following command:

git clone --depth=1 https://github.com/twintproject/twint.git
cd twint
pip3 install . -r requirements.txt

At this point, you can either use the CLI or import this package into a Python script, here is an example script that scrapes the amount of tweets containing a specific keyword based off a username:

import twint

# Configure
c = twint.Config()
c.Username = "realDonaldTrump"
c.Search = "great"

# Run
twint.run.Search(c)

The output of this command would be the following:

Here is another example which brings in another username, and pulls 10 tweets from this user:

import twint

c = twint.Config()

c.Username = "noneprivacy"
c.Custom["tweet"] = ["id"]
c.Custom["user"] = ["bio"]
c.Limit = 10
c.Store_csv = True
c.Output = "none"

twint.run.Search(c)

I would highly recommend checking out their GitHub repository (linked above) to gain even more knowledge about this package, do you plan on using this package? I would love to hear your thoughts about this!

Thanks So Much!

if you have any suggestions, thoughts, or just want to connect, feel free to contact/follow me on Twitter! Also, below is a link to some of my favorite resources for learning programming:

Thanks so much for your support!