TweePy – Twitter of Python

I had an opportunity to attend the NASSCOM Technology and Leadership Forum in Mumbai from 20th to 22nd Feb 2019. It was an amazing experience to listen, gain knowledge and insights from the industry’s best of the bests. CxO’s keynote speeches were focused on technologies like AI, ML, Blockchains et al

My personal favorite out of all the keynote was from Vala Afshar, Chief Digital Evangelist @ Salesforce. It was a privilege to be there and watch him explaining about AI, ML, and the importance of data. Truly inspiring and mesmerized by his depth of knowledge in the IT industry.

As mentioned by Vala Afshar, “Data is the oil of 21st century but oil is just useless thick goop until you refine it into fuel. AI is your refinery“. In those 3 days, a lot of critical information was shared and scattered across all the social media. The reason to write this blog is to share my idea to save those GEM of information which I can keep munching time and again to get inspired and motivated.

Twitter, the most popular social media platform is one of my favorites and wanted to save all those tweets which had hashtag #NASSCOM_TLF (official hashtag of the event). After quick research using ‘DuckDuckGo’, I had decided to use TweePy Twitter for Python module which is developed to use Twitter API to connect, read, write, retweet and send direct messages right from Python.

Tweepy requires twitter app to be created to use Twitter’s API to exchange information between Twitter and Python. I followed this link to set up and run my twitter app.

Here is the code which I wrote to download all the tweets having hashtag #NASSCOM_TLF and save it to an excel file!

# Download tweepy using pip install tweepy
import tweepy
# Pandas dataframe used to get tweets in tabular format and export it excel
import pandas as pd

# Replace consumer key, consumer secret
consumer_key = 'REPLACE'
consumer_secret = 'REPALCE'

# Replace access token key and secret
access_token = 'REPLACE-REPLACE'
access_token_secret = 'REPLACE'

# Authenticate to Twitter API
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

# Create dataframe with columns names 
df = pd.DataFrame(columns=['text','timeline', 'username', 'user_id'])
# Initialize lists to store messages
msgs = []
msg =[]

# search twitter with hashtag #NASSCOM_TLF and exclude retweets
for tweet in tweepy.Cursor(api.search, q='#NASSCOM_TLF -filter:retweets',tweet_mode='extended', rpp=100).items():
    msg = [tweet.full_text, tweet.created_at, tweet.user.name, tweet.user.screen_name] 
    msg = tuple(msg)                    
    msgs.append(msg)

# Append tweets stored in lists to dataframe
df = pd.DataFrame(msgs)
# Column header columns having full tweet messages, time, user name and ID
df.columns = ['Tweet Text', 'Tweet Date Time (GMT)', 'Username', 'User ID']
# Check the first 5 tweets to see any errors
print(df.head())
# Create a file 
output = "tweets_ntlf.xlsx"
# Export tweets from dataframe to excel
try:
    df.to_excel(output, index=False)
except Exception as Error:
    print("Unable to get NASSCOM Tweets", Error)
Screenshot of the excel file

NOTE: I’ve excluded retweets to avoid duplication of information.

Please continue to stop by this blog and share your comments below.

Up Next – Manage Alerts of VMAX array using REST API

Advertisements

Docker’ize’ Python

What is Docker?

Docker is a computer program that performs operating-system-level virtualization, also known as “containerization”. It was first released in 2013 and is developed by Docker, Inc. source: Wikipedia

How Docker works?

Docker containers wrap up software and its dependencies into a standardized unit for software development that includes everything it needs to run: code, runtime, system tools, and libraries. This guarantees that your application will always run the same and makes collaboration as simple as sharing a container image.
source: www.docker.com

Why Docker?

Docker unlocks the potential of your organization by giving developers and IT the freedom to build, manage and secure business-critical applications without the fear of technology or infrastructure lock-in. source: www.docker.com

Pycodestyle @ Sublime Text

Is your script/code looks dull and not in order? If yes, then its time to use pycodestyle!!

Pycodestyle (formerly known as PEP8) is a syntax and style checker tool for Python language. In this blog, I’ve documented step by step procedure to install this with Sublime Text.

Before we start, let us briefly understand more about linter. Lint or linter is a tool that analyzes source code to flag programming errors, bugs, stylistic errors. To know more please check out the wiki link

Pycodestyle is one of the linter plugins available for Sublime Text. To install this plugin, first, we need to install Sublime Linter plugin.

Prerequisites:

  1. Python3.x
  2. Sublime Text
  3. PIP

***This step by step procedure to apply to Debian and Debian based Linux Operation Systems.***

Steps to install pycodestyle:

  1. Install system package using pip or apt
$pip3 install pycodestyle
OR
$sudo apt install python3-pycodestyle
$which pycodestyle # shows the location where pycode style is

2. Install pycodestyle plugin in sublime text

a. Open package control (CTRL+SHIFT+P), type 'install package' and hit 'ENTER' # It takes some time to load repositories. 
b. Type 'sublimelinter-pycodestyle' and Click on the first option as shown below. After installation, a new tab window will open showing installed plugin information
C. To configure sublime-linter settings, goto Preferences > Package Settings > SublimeLinter > Settings
d. In the Sublime Linter settings, make sure @disable: false is set s shown in below screenshot. 
e. To test pycodestyle, create a new file and save it as test.py.
Start writing the code, Pycodestyle will automatically start showing the errors while you are typing in the console message.

That’s it!! Your code looks more disciplined of course after fixing the errors 🙂

In addition to pycodestyle, try pylint, pyflakes and pydocstyle plugins as shown above. These plugins improve Code Quality by following Python Coding Standards. Following best practices will make you a good scripter/developer and remember Quality Matters!!

References:

  1. https://pypi.org/project/pycodestyle/
  2. https://github.com/ergdev/SublimeLinter-pycodestyle
  3. http://pycodestyle.pycqa.org/en/latest/
  4. http://www.sublimelinter.com/en/stable/settings.html

So go ahead and use pycodestyle. Use the above references to deep dive and learn more about this plugin which is a ‘Must Have’ for the Sublime Text. Happy Pythonic way of coding!!