Install and Configure Go/Golang on Raspberry Pi

Go/Golang is one of the hot programming languages while I’m typing this today.

Go is a programming language created at Google in 2009 by Robert Griesemer, Rob Pike and Ken Thompson. Go is a statically typed compiled, procedural language similar to C, with memory safety, garbage collection, structural typing, concurrency and other great features are bundled to make it better compared to other languages in the marketplace.

Docker, Kubernetes, Graphana, Hugo are some of the best apps written in Go. It has a robust set of libraries and app performance is better compared to other languages.

Today I’m starting my journey to learn Go/Golang and Google will be my mentor to install and setup Golang on my Raspbian OS/Raspberry Pi 3. To get the latest version use below steps instead of native package management tool such as apt.

Installation Steps:

  • Download the current stable version of Go available on the google’s official website. At the time of writing this tutorial, 1.12.4 is the stable version. Check the latest version here
cd ~ && curl -O https://dl.google.com/go/go1.12.4.linux-armv6l.tar.gz

Above command would change directory to your ‘Home’ directory and download Go compressed tar file using ‘Curl’

  • Extract compressed tar file and place it inside /usr/local directory. Please note root level access or sudo access required to perform this step.
sudo tar -C /usr/local -xzvf go1.12.4.linux-armv6l.tar.gz
  • Set Path variables to avoid typing complete path and in order to access binaries or libraries of Go by the Raspbian OS. Open ~/.profile, a hidden file located in your ‘Home’ directory. Use nano or vi or subl to edit the file (subl ~/.profile). Add below lines at the end of the file.
export GOPATH=$HOME/go
export PATH=$PATH:/usr/local/go/bin:$GOPATH/bin
  • To take effect of the above changes made to the file ‘profile’, run the below command. The source command can be used to load any functions file into the current shell, script or a command prompt.
source ~/.profile
  • Create directory called ‘go’ in the ‘Home’ directory. All my codes are placed in the folder. Change the directory name as per choice but do change the GOPATH accordingly as mentioned above.
 mkdir $HOME/go 
  • Validate Go is working as expected or not by running below command
vb@pi:~ $  go version
go version go1.12.4 linux/arm
vb@pi:~ $

My first code in Go

Create a directory first_code in ‘go’ and write following content in the file and save it as first_code.go

mkdir -p $HOME/go/src/first_code 
{-p is used to create directory and its sub-directories at once}
package main

import "fmt"

func main() {
    fmt.Printf("My first code in Go Language!!!\n")
}

Now build and run the code. Change directory
to the first_code cd ~/go/src/first_code and run below commands

vb@pi:~/go/src/first_code $ go build
vb@pi:~/go/src/first_code $

vb@pi:~/go/src/first_code $ ./first_code
My first code in Go Language!!!
vb@pi:~/go/src/first_code $

Note:

Above steps are applicable to any Linux distributions just by changing the first step of downloading the compressed tar file. Change the architecture from arm6l to amd64 or as applicable to your hardware.

I am able to successfully setup GO on my Pi. Hope you’d also do the same and happy learning Go. If you have any issues or questions please mention them in the below comment section.

Advertisements

Python Pandas Pivot Table

This blog is going to be mostly helpful to folks who work with excel and databases. But, if you work with Excel and find yourself needing to deal with automation of pivot tables, you might also appreciate the technique.

When working with MySQL database, it was not an easy way for me to fetch the data and reshape it to a Pivot Table. I did duckduckgo, found ample of examples but most of them were hard to understand or not yielding the required output.

Python’s Pandas Pivot Table was the savior!! I followed below steps to fetch data from MySQL DB and generate a simple Pivot Table. Later this pivot table exported as an excel file and shared with stakeholders using slack.

  • Create connection string to MySQL DB using PyMySQL
  • Create a SQLAlchemy Engine (ORM)
  • Write a Query statement to fetch data from DB
  • Create a Pandas Dataframe directly from the query using SQLAlchemy
  • Create a Pandas Pivot Table with required Index, Columns and Values
  • Export data to excel or csv file using to_excel and to_csv respectively
  • Share it to stakeholders via slack

Please find below Python code screenshots

I hope this blog triggered a thought to automate the mundane task of generating pivot tables from spreadsheets or databases. Please reach out to me if you want to check out the full code.

Thank you for stopping by this blog and please share your suggestions below under the Comment section

Up Next – Manage Alerts of VMAX3 array using REST API

Image Courtesy:

Matt Whitt
https://ttorial.com/Images/python-automation-automate-mandane-task-python.jpg

TweePy – Twitter of Python

I had an opportunity to attend the NASSCOM Technology and Leadership Forum in Mumbai from 20th to 22nd Feb 2019. It was an amazing experience to listen, gain knowledge and insights from the industry’s best of the bests. CxO’s keynote speeches were focused on technologies like AI, ML, Blockchains et al

My personal favorite out of all the keynote was from Vala Afshar, Chief Digital Evangelist @ Salesforce. It was a privilege to be there and watch him explaining about AI, ML, and the importance of data. Truly inspiring and mesmerized by his depth of knowledge in the IT industry.

As mentioned by Vala Afshar, “Data is the oil of 21st century but oil is just useless thick goop until you refine it into fuel. AI is your refinery“. In those 3 days, a lot of critical information was shared and scattered across all the social media. The reason to write this blog is to share my idea to save those GEM of information which I can keep munching time and again to get inspired and motivated.

Twitter, the most popular social media platform is one of my favorites and wanted to save all those tweets which had hashtag #NASSCOM_TLF (official hashtag of the event). After quick research using ‘DuckDuckGo’, I had decided to use TweePy Twitter for Python module which is developed to use Twitter API to connect, read, write, retweet and send direct messages right from Python.

Tweepy requires twitter app to be created to use Twitter’s API to exchange information between Twitter and Python. I followed this link to set up and run my twitter app.

Here is the code which I wrote to download all the tweets having hashtag #NASSCOM_TLF and save it to an excel file!

# Download tweepy using pip install tweepy
import tweepy
# Pandas dataframe used to get tweets in tabular format and export it excel
import pandas as pd

# Replace consumer key, consumer secret
consumer_key = 'REPLACE'
consumer_secret = 'REPALCE'

# Replace access token key and secret
access_token = 'REPLACE-REPLACE'
access_token_secret = 'REPLACE'

# Authenticate to Twitter API
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth)

# Create dataframe with columns names 
df = pd.DataFrame(columns=['text','timeline', 'username', 'user_id'])
# Initialize lists to store messages
msgs = []
msg =[]

# search twitter with hashtag #NASSCOM_TLF and exclude retweets
for tweet in tweepy.Cursor(api.search, q='#NASSCOM_TLF -filter:retweets',tweet_mode='extended', rpp=100).items():
    msg = [tweet.full_text, tweet.created_at, tweet.user.name, tweet.user.screen_name] 
    msg = tuple(msg)                    
    msgs.append(msg)

# Append tweets stored in lists to dataframe
df = pd.DataFrame(msgs)
# Column header columns having full tweet messages, time, user name and ID
df.columns = ['Tweet Text', 'Tweet Date Time (GMT)', 'Username', 'User ID']
# Check the first 5 tweets to see any errors
print(df.head())
# Create a file 
output = "tweets_ntlf.xlsx"
# Export tweets from dataframe to excel
try:
    df.to_excel(output, index=False)
except Exception as Error:
    print("Unable to get NASSCOM Tweets", Error)
Screenshot of the excel file

NOTE: I’ve excluded retweets to avoid duplication of information.

Please continue to stop by this blog and share your comments below.

Up Next – Manage Alerts of VMAX array using REST API

Docker’ize’ Python

What is Docker?

Docker is a computer program that performs operating-system-level virtualization, also known as “containerization”. It was first released in 2013 and is developed by Docker, Inc. source: Wikipedia

How Docker works?

Docker containers wrap up software and its dependencies into a standardized unit for software development that includes everything it needs to run: code, runtime, system tools, and libraries. This guarantees that your application will always run the same and makes collaboration as simple as sharing a container image.
source: www.docker.com

Why Docker?

Docker unlocks the potential of your organization by giving developers and IT the freedom to build, manage and secure business-critical applications without the fear of technology or infrastructure lock-in. source: www.docker.com

Pycodestyle @ Sublime Text

Is your script/code looks dull and not in order? If yes, then its time to use pycodestyle!!

Pycodestyle (formerly known as PEP8) is a syntax and style checker tool for Python language. In this blog, I’ve documented step by step procedure to install this with Sublime Text.

Before we start, let us briefly understand more about linter. Lint or linter is a tool that analyzes source code to flag programming errors, bugs, stylistic errors. To know more please check out the wiki link

Pycodestyle is one of the linter plugins available for Sublime Text. To install this plugin, first, we need to install Sublime Linter plugin.

Prerequisites:

  1. Python3.x
  2. Sublime Text
  3. PIP

***This step by step procedure to apply to Debian and Debian based Linux Operation Systems.***

Steps to install pycodestyle:

  1. Install system package using pip or apt
$pip3 install pycodestyle
OR
$sudo apt install python3-pycodestyle
$which pycodestyle # shows the location where pycode style is

2. Install pycodestyle plugin in sublime text

a. Open package control (CTRL+SHIFT+P), type 'install package' and hit 'ENTER' # It takes some time to load repositories. 
b. Type 'sublimelinter-pycodestyle' and Click on the first option as shown below. After installation, a new tab window will open showing installed plugin information
C. To configure sublime-linter settings, goto Preferences > Package Settings > SublimeLinter > Settings
d. In the Sublime Linter settings, make sure @disable: false is set s shown in below screenshot. 
e. To test pycodestyle, create a new file and save it as test.py.
Start writing the code, Pycodestyle will automatically start showing the errors while you are typing in the console message.

That’s it!! Your code looks more disciplined of course after fixing the errors 🙂

In addition to pycodestyle, try pylint, pyflakes and pydocstyle plugins as shown above. These plugins improve Code Quality by following Python Coding Standards. Following best practices will make you a good scripter/developer and remember Quality Matters!!

References:

  1. https://pypi.org/project/pycodestyle/
  2. https://github.com/ergdev/SublimeLinter-pycodestyle
  3. http://pycodestyle.pycqa.org/en/latest/
  4. http://www.sublimelinter.com/en/stable/settings.html

So go ahead and use pycodestyle. Use the above references to deep dive and learn more about this plugin which is a ‘Must Have’ for the Sublime Text. Happy Pythonic way of coding!!

phpoffice/phpspreadsheet

phpoffice/phpspreadsheet is a successor of phpExcel. We recently developed an automated solution to pull all the backup success rate reports from Dell EMC Data Protection Advisor (DPA). The portal which displays the report was good but not enough to use that information further like sharing the reports, pivoting etc..

After trying several methods, found this library to be very useful. There aren’t many examples available on the internet to retrieve data from MySQL and write to XLSX file. Hope this blog might be helpful who would like to use this library for creating XLSX files. Fear not, it doesn’t throw errors while opening this file in MS Office Excel