Visual Studio Code is a cross platform editor that supports multiple programming languages. Combining with Python, and its associated 3rd party packages that wrap Twitter’s API, we can easy connect to Twitter, read and use the data in just few lines of code.
Why Twitter ?
Twitter is a popular social network where users can share short SMS-like messages called tweets. Users share thoughts, links and pictures on Twitter, journalists comment on live events. The list of different ways to use Twitter could be really long, and with 500 millions of tweets per day, there’s a lot of data to play with.
Using an excellent Python library – TwitterAPI as a minimal wrapper to native Twitter API calls, we can have a tool to extract the information we need.
Getting started
At first, you will need to create a Twitter account and then a Twitter application. Once you have an account, you’ll have to go to their developer’s site, in the Application Management area and create a new application. After your application is created, you will need to get your API keys (or generate some) and also generate access tokens.
Download and install first Python 3 and then Visual Studio Code. Our example will work with minor changes also for Python 2.
Once you first open the VS Code, open the Extensions list and install Python extension.
Download from GitHub the code (click to download) and extract the archive to a local folder. Then use the option File -> Open Folder in VSCode.
To install the Twitter library, open the command prompt and run each of the next install commands. You could also run these lines within VSCode. It has integrated the command prompt, and you can easy access it by clicking on the menu item: View -> Integrated terminal.
pip install TwitterAPI pip install configparser
Connecting to Twitter
To be able to do this we will use the TwitterAPI library. With credentials from Twitter, we can access the Twitter data:
from TwitterAPI import TwitterAPI connectTwitter = TwitterAPI(consumer_key, consumer_secret, access_token_key, access_token_secret)
Then we can search and extract results in successive batches or pages. In this example searching after ‘#webapi’ string, retrieving the results in English, in batches of 10 results.
twitterPage = TwitterRestPager(connectTwitter, 'search/tweets', \
{'q':'#webapi', 'count':10, 'lang': 'en'})
And in the end we can iterate through search results
for item in twitterPage.get_iterator():
print('Tweet: ', item['text'], ' from ', item['user']['name'])
To summarize, here is the full code allowing to connect, and retrieve the details from Twitter:
from TwitterAPI import TwitterAPI
from TwitterAPI import TwitterRestPager
configValues = configparser.RawConfigParser()
configValues.read(r'.\config.real.ini')
# connecting to Twitter - replace parameters with values
connectTwitter = TwitterAPI(consumer_key, consumer_secret,
access_token_key, access_token_secret)
# search tweets
twitterPage = TwitterRestPager(connectTwitter, \
'search/tweets', \
{'q':'#webapi', 'count':10, 'lang': 'en'})
responseValues = []
# if connected successfully, retrieve paginated search results
for item in twitterPage.get_iterator():
if 'user' in item and 'text' in item:
responseValues.append({'username': item['user']['name'],
'screen_name': '@{}'.format(item['user']['screen_name']),
'profile_image': item['user']['profile_image_url_https'],
'profile_description': item['user']['description'],
'text': item['text']})
print('Tweet ', len(responseValues), ' from ', item['user']['name'])
elif 'message' in item and item['code'] == 88:
print('SUSPEND, RATE LIMIT EXCEEDED: %s\n' % item['message'])
break
# stop after first 50 tweets, to not exceed the limit too fast
if len(responseValues) > 50:
break
Save the results to a json file
Saving in json format brings many way to use effectively the data. This could be done using a simple command from json package.
import json
with open(r'.\twitter.txt', 'w') as outfile:
json.dump(responseValues, outfile, indent=4)
Load the credentials from a configuration file
Loading the connection credentials from a configuration file, could be easily done in Python. Here is a sample configuration sample:
[TwitterSettings] consumer_key = aaa consumer_secret = bbb access_token_key = ccc access_token_secret = ddd
To read any of the associated configuration keys we run next commands:
import configparser
configValues = configparser.RawConfigParser()
configValues.read(r'.\config.ini')
print configValues.get('TwitterSettings', 'consumer_key')
Calling the TwitterAPI constructor using the settings from a configuration files becomes:
connectTwitter = TwitterAPI(configValues.get('TwitterSettings', 'consumer_key'),
configValues.get('TwitterSettings', 'consumer_secret'),
configValues.get('TwitterSettings', 'access_token_key'),
configValues.get('TwitterSettings', 'access_token_secret'))
Debug the code with VS Code
We can run the above code and easily debug in VS Code. To be easier, dowload the code and open the VS Code debugger, selecting Python as language.
Python plugin for VSCode comes with linting support, which is source code, bug and quality checker, following the style recommended by Python style guide (PEP-8). Pylint quickly highlights the errors and it shows recommendations on how to write the code.
Or run directly
The other quick way to run the code, without debug, would be to use the combination CTRL-B. This uses the configuration file task.json, saved in folder .vscode and run automatically the code file using Python.
{
"command": "python",
"showOutput": "always",
"windows": {
"command": "python.exe"
},
"args": ["${file}"]
}
Final results
Using the above code, we could retrieve tweets like the ones displayed below
[
{
"profile_image": "https://pbs.twimg.com/profile_images/802976662187114496/gif5G5oY_normal.jpg",
"text": "RT @techjunkiejh: Chart Widgets With Server Side Data In MVC Using #AngularJS And #WebAPI https://t.co/MfW9zi431f #javascript https://t.co/\u2026",
"screen_name": "@Phunick",
"profile_description": "Software engineer | Consulting | Architect | Developer Expert: MCP, MCSD, MSSQL,BigData, AngularJS,TypeScript,Nodejs,Reactjs.Telerik&...",
"username": "Nick"
}
]
Project available in GitHub
This code could be access also from GitHub – https://github.com/fpetru/python-twitter-vscode
