{"id":273,"date":"2016-12-13T09:00:08","date_gmt":"2016-12-13T07:00:08","guid":{"rendered":"http:\/\/qappdesign.com\/?p=273"},"modified":"2016-12-13T10:18:38","modified_gmt":"2016-12-13T08:18:38","slug":"vscode-twitter-python-json","status":"publish","type":"post","link":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/","title":{"rendered":"Visual Studio Code – Connect to Twitter with Python"},"content":{"rendered":"

Visual Studio Code<\/a> is a cross platform editor that supports multiple programming languages. Combining with Python, and its associated 3rd party packages that wrap Twitter\u2019s API, we can easy connect to Twitter, read and use the data in just few lines of code. <\/p>\n

Why Twitter ?<\/h4>\n

Twitter<\/a> is a popular social network where users can share short SMS-like messages called tweets. Users share thoughts, links and pictures on Twitter, journalists comment on live events. The list of different ways to use Twitter could be really long, and with 500 millions of tweets per day, there\u2019s a lot of data to play with.<\/p>\n

Using an excellent Python library – TwitterAPI<\/a> as a minimal wrapper to native Twitter API calls, we can have a tool to extract the information we need.<\/p>\n

Getting started<\/h4>\n

At first, you will need to create a Twitter account and then a Twitter application. Once you have an account, you\u2019ll have to go to their developer\u2019s site, in the Application Management<\/a> area and create a new application. After your application is created, you will need to get your API keys (or generate some) and also generate access tokens.<\/p>\n

Download and install first Python 3<\/a> and then Visual Studio Code<\/a>. Our example will work with minor changes also for Python 2. <\/p>\n

Once you first open the VS Code, open the Extensions list and install Python<\/em> extension.<\/p>\n

\"Visual

Download from GitHub the code (click to download<\/a>) and extract the archive to a local folder. Then use the option File -> Open Folder in VSCode.<\/p>\n

\"VSCode

To install the Twitter library, open the command prompt and run each of the next install commands. You could also run these lines within VSCode. It has integrated the command prompt, and you can easy access it by clicking on the menu item: View -> Integrated terminal.<\/p>\n

\r\npip install TwitterAPI\r\npip install configparser\r\n<\/pre>\n
Connecting to Twitter<\/h4>\n
To be able to do this we will use the TwitterAPI<\/a> library. With credentials from Twitter, we can access the Twitter data: <\/p>\n
\r\nfrom TwitterAPI import TwitterAPI\r\nconnectTwitter = TwitterAPI(consumer_key, consumer_secret, access_token_key, access_token_secret)\r\n<\/pre>\n
Then we can search and extract results in successive batches or pages. In this example searching after ‘#webapi’ string, retrieving the results in English, in batches of 10 results.<\/p>\n
\r\ntwitterPage = TwitterRestPager(connectTwitter, 'search\/tweets', \\\r\n                               {'q':'#webapi', 'count':10, 'lang': 'en'})\r\n<\/pre>\n
And in the end we can iterate through search results<\/p>\n
\r\nfor item in twitterPage.get_iterator():\r\n   print('Tweet: ', item['text'], ' from ', item['user']['name'])\r\n<\/pre>\n
To summarize, here is the full code allowing to connect, and retrieve the details from Twitter:<\/p>\n
\r\nfrom TwitterAPI import TwitterAPI\r\nfrom TwitterAPI import TwitterRestPager\r\n\r\nconfigValues = configparser.RawConfigParser()\r\nconfigValues.read(r'.\\config.real.ini')\r\n\r\n# connecting to Twitter - replace parameters with values\r\nconnectTwitter = TwitterAPI(consumer_key, consumer_secret,  \r\n                            access_token_key, access_token_secret)\r\n\r\n# search tweets\r\ntwitterPage = TwitterRestPager(connectTwitter, \\\r\n                               'search\/tweets', \\\r\n                               {'q':'#webapi', 'count':10, 'lang': 'en'})\r\n\r\nresponseValues = []\r\n\r\n# if connected successfully, retrieve paginated search results\r\nfor item in twitterPage.get_iterator():\r\n    if 'user' in item and 'text' in item:\r\n        responseValues.append({'username': item['user']['name'],\r\n                               'screen_name': '@{}'.format(item['user']['screen_name']),\r\n                               'profile_image': item['user']['profile_image_url_https'],\r\n                               'profile_description': item['user']['description'],\r\n                               'text': item['text']})\r\n        print('Tweet ', len(responseValues), ' from  ', item['user']['name'])\r\n    elif 'message' in item and item['code'] == 88:\r\n        print('SUSPEND, RATE LIMIT EXCEEDED: %s\\n' % item['message'])\r\n        break\r\n\r\n    # stop after first 50 tweets, to not exceed the limit too fast\r\n    if len(responseValues) > 50:\r\n        break\r\n<\/pre>\n
Save the results to a json file<\/h4>\n
Saving in json format brings many way to use effectively the data. This could be done using a simple command from json package.<\/p>\n
\r\nimport json\r\nwith open(r'.\\twitter.txt', 'w') as outfile:\r\n    json.dump(responseValues, outfile, indent=4)\r\n<\/pre>\n
Load the credentials from a configuration file<\/h4>\n
Loading the connection credentials from a configuration file, could be easily done in Python. Here is a sample configuration sample:<\/p>\n
\r\n[TwitterSettings]\r\nconsumer_key = aaa\r\nconsumer_secret = bbb\r\naccess_token_key = ccc\r\naccess_token_secret = ddd\r\n<\/pre>\n
To read any of the associated configuration keys we run next commands:<\/p>\n
\r\nimport configparser\r\n\r\nconfigValues = configparser.RawConfigParser()\r\nconfigValues.read(r'.\\config.ini')\r\n\r\nprint configValues.get('TwitterSettings', 'consumer_key')\r\n<\/pre>\n
Calling the TwitterAPI constructor using the settings from a configuration files becomes:<\/p>\n
\r\nconnectTwitter = TwitterAPI(configValues.get('TwitterSettings', 'consumer_key'),\r\n                            configValues.get('TwitterSettings', 'consumer_secret'),\r\n                            configValues.get('TwitterSettings', 'access_token_key'),\r\n                            configValues.get('TwitterSettings', 'access_token_secret'))\r\n<\/pre>\n
Debug the code with VS Code<\/h4>\n
We can run the above code and easily debug in VS Code. To be easier, dowload the code<\/a> and open the VS Code debugger, selecting Python as language.  <\/p>\n
\"VSCode
Python plugin for VSCode comes with linting<\/em> support, which is source code, bug and quality checker, following the style recommended by Python style guide (PEP-8). Pylint<\/a> quickly highlights the errors and it shows recommendations on how to write the code.<\/p>\n
Or run directly<\/h4>\n
The other quick way to run the code, without debug, would be to use the combination CTRL-B. This uses the configuration file task.json, saved in folder .vscode and run automatically the code file using Python.<\/p>\n
\r\n{\r\n    \"command\": \"python\",\r\n    \"showOutput\": \"always\",\r\n    \"windows\": {\r\n        \"command\": \"python.exe\" \r\n    },\r\n    \"args\": [\"${file}\"]\r\n}\r\n<\/pre>\n
Final results<\/h4>\n
Using the above code, we could retrieve tweets like the ones displayed below<\/p>\n
\r\n[\r\n    {\r\n        \"profile_image\": \"https:\/\/pbs.twimg.com\/profile_images\/802976662187114496\/gif5G5oY_normal.jpg\",\r\n        \"text\": \"RT @techjunkiejh: Chart Widgets With Server Side Data In MVC Using #AngularJS And #WebAPI https:\/\/t.co\/MfW9zi431f #javascript https:\/\/t.co\/\\u2026\",\r\n        \"screen_name\": \"@Phunick\",\r\n        \"profile_description\": \"Software engineer | Consulting | Architect | Developer Expert: MCP, MCSD, MSSQL,BigData, AngularJS,TypeScript,Nodejs,Reactjs.Telerik&...\",\r\n        \"username\": \"Nick\"\r\n    }\r\n]\r\n<\/pre>\n
Project available in GitHub<\/h4>\n
This code could be access also from GitHub – https:\/\/github.com\/fpetru\/python-twitter-vscode<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
Visual Studio Code is a cross platform editor that supports multiple programming languages. Combining with Python, and its associated 3rd party packages that wrap Twitter\u2019s API, we can easy connect to Twitter, read and use the data in just few lines of code. Why Twitter ? Twitter is a popular social network where users can share short SMS-like messages called tweets. Users share thoughts, links and pictures on Twitter, journalists comment on live events. The list of different ways to use Twitter could be really long, and with 500 millions of tweets per day, there\u2019s a lot of data to play with. Using an excellent Python library – TwitterAPI as a minimal wrapper to native Twitter API calls, we can have a tool to extract the information we need. Getting started At first, you will need to create a Twitter account and then a Twitter application. Once you have an account, you\u2019ll have to go to their developer\u2019s site, in the Application Management area and create a new application. After your application is created, you will need to get your API keys (or generate some) and also generate access tokens. Download and install first Python 3 and then Visual Studio […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[7,14,8],"tags":[15,16],"yoast_head":"\nVisual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Visual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations\" \/>\n<meta property=\"og:description\" content=\"Visual Studio Code is a cross platform editor that supports multiple programming languages. Combining with Python, and its associated 3rd party packages that wrap Twitter\u2019s API, we can easy connect to Twitter, read and use the data in just few lines of code. Why Twitter ? Twitter is a popular social network where users can share short SMS-like messages called tweets. Users share thoughts, links and pictures on Twitter, journalists comment on live events. The list of different ways to use Twitter could be really long, and with 500 millions of tweets per day, there\u2019s a lot of data to play with. Using an excellent Python library – TwitterAPI as a minimal wrapper to native Twitter API calls, we can have a tool to extract the information we need. Getting started At first, you will need to create a Twitter account and then a Twitter application. Once you have an account, you\u2019ll have to go to their developer\u2019s site, in the Application Management area and create a new application. After your application is created, you will need to get your API keys (or generate some) and also generate access tokens. Download and install first Python 3 and then Visual Studio […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/\" \/>\n<meta property=\"og:site_name\" content=\"Cloud, Data and Integrations\" \/>\n<meta property=\"article:published_time\" content=\"2016-12-13T07:00:08+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2016-12-13T08:18:38+00:00\" \/>\n<meta property=\"og:image\" content=\"http:\/\/.\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary\" \/>\n<meta name=\"twitter:creator\" content=\"@qappdesign\" \/>\n<meta name=\"twitter:site\" content=\"@QAppDesign\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Petru Faurescu\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"5 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebSite\",\"@id\":\"https:\/\/qappdesign.com\/code\/#website\",\"url\":\"https:\/\/qappdesign.com\/code\/\",\"name\":\"QualityAppDesign\",\"description\":\"with Petru Faurescu\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/qappdesign.com\/code\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#primaryimage\",\"inLanguage\":\"en-US\",\"url\":\".\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg\",\"contentUrl\":\".\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#webpage\",\"url\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/\",\"name\":\"Visual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations\",\"isPartOf\":{\"@id\":\"https:\/\/qappdesign.com\/code\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#primaryimage\"},\"datePublished\":\"2016-12-13T07:00:08+00:00\",\"dateModified\":\"2016-12-13T08:18:38+00:00\",\"author\":{\"@id\":\"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a\"},\"breadcrumb\":{\"@id\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/qappdesign.com\/code\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Visual Studio Code – Connect to Twitter with Python\"}]},{\"@type\":\"Person\",\"@id\":\"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a\",\"name\":\"Petru Faurescu\",\"image\":{\"@type\":\"ImageObject\",\"@id\":\"https:\/\/qappdesign.com\/code\/#personlogo\",\"inLanguage\":\"en-US\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g\",\"caption\":\"Petru Faurescu\"},\"description\":\"Product lead, software developer & architect\",\"sameAs\":[\"https:\/\/qappdesign.com\/code\",\"https:\/\/www.linkedin.com\/in\/petrufaurescu\/\",\"https:\/\/twitter.com\/@qappdesign\"]}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Visual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/","og_locale":"en_US","og_type":"article","og_title":"Visual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations","og_description":"Visual Studio Code is a cross platform editor that supports multiple programming languages. Combining with Python, and its associated 3rd party packages that wrap Twitter\u2019s API, we can easy connect to Twitter, read and use the data in just few lines of code. Why Twitter ? Twitter is a popular social network where users can share short SMS-like messages called tweets. Users share thoughts, links and pictures on Twitter, journalists comment on live events. The list of different ways to use Twitter could be really long, and with 500 millions of tweets per day, there\u2019s a lot of data to play with. Using an excellent Python library – TwitterAPI as a minimal wrapper to native Twitter API calls, we can have a tool to extract the information we need. Getting started At first, you will need to create a Twitter account and then a Twitter application. Once you have an account, you\u2019ll have to go to their developer\u2019s site, in the Application Management area and create a new application. After your application is created, you will need to get your API keys (or generate some) and also generate access tokens. Download and install first Python 3 and then Visual Studio […]","og_url":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/","og_site_name":"Cloud, Data and Integrations","article_published_time":"2016-12-13T07:00:08+00:00","article_modified_time":"2016-12-13T08:18:38+00:00","og_image":[{"url":".\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg"}],"twitter_card":"summary","twitter_creator":"@qappdesign","twitter_site":"@QAppDesign","twitter_misc":{"Written by":"Petru Faurescu","Est. reading time":"5 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebSite","@id":"https:\/\/qappdesign.com\/code\/#website","url":"https:\/\/qappdesign.com\/code\/","name":"QualityAppDesign","description":"with Petru Faurescu","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/qappdesign.com\/code\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"ImageObject","@id":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#primaryimage","inLanguage":"en-US","url":".\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg","contentUrl":".\/wp-content\/uploads\/2016\/12\/VSCode-Python.jpg"},{"@type":"WebPage","@id":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#webpage","url":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/","name":"Visual Studio Code - Connect to Twitter with Python - Cloud, Data and Integrations","isPartOf":{"@id":"https:\/\/qappdesign.com\/code\/#website"},"primaryImageOfPage":{"@id":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#primaryimage"},"datePublished":"2016-12-13T07:00:08+00:00","dateModified":"2016-12-13T08:18:38+00:00","author":{"@id":"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a"},"breadcrumb":{"@id":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/qappdesign.com\/code\/vscode-twitter-python-json\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/qappdesign.com\/code\/"},{"@type":"ListItem","position":2,"name":"Visual Studio Code – Connect to Twitter with Python"}]},{"@type":"Person","@id":"https:\/\/qappdesign.com\/code\/#\/schema\/person\/54db90dc6fe846cfd4c5a9544d93b75a","name":"Petru Faurescu","image":{"@type":"ImageObject","@id":"https:\/\/qappdesign.com\/code\/#personlogo","inLanguage":"en-US","url":"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/84fb359a4e3d583dbea5a34bd5566956?s=96&d=mm&r=g","caption":"Petru Faurescu"},"description":"Product lead, software developer & architect","sameAs":["https:\/\/qappdesign.com\/code","https:\/\/www.linkedin.com\/in\/petrufaurescu\/","https:\/\/twitter.com\/@qappdesign"]}]}},"jetpack_featured_media_url":"","jetpack_publicize_connections":[],"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/p82xvM-4p","_links":{"self":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/273"}],"collection":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/comments?post=273"}],"version-history":[{"count":34,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/273\/revisions"}],"predecessor-version":[{"id":312,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/posts\/273\/revisions\/312"}],"wp:attachment":[{"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/media?parent=273"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/categories?post=273"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/qappdesign.com\/code\/wp-json\/wp\/v2\/tags?post=273"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}