Python Pandas Pivot Table

This blog is going to be mostly helpful to folks who work with excel and databases. But, if you work with Excel and find yourself needing to deal with automation of pivot tables, you might also appreciate the technique.

When working with MySQL database, it was not an easy way for me to fetch the data and reshape it to a Pivot Table. I did duckduckgo, found ample of examples but most of them were hard to understand or not yielding the required output.

Python’s Pandas Pivot Table was the savior!! I followed below steps to fetch data from MySQL DB and generate a simple Pivot Table. Later this pivot table exported as an excel file and shared with stakeholders using slack.

  • Create connection string to MySQL DB using PyMySQL
  • Create a SQLAlchemy Engine (ORM)
  • Write a Query statement to fetch data from DB
  • Create a Pandas Dataframe directly from the query using SQLAlchemy
  • Create a Pandas Pivot Table with required Index, Columns and Values
  • Export data to excel or csv file using to_excel and to_csv respectively
  • Share it to stakeholders via slack

Please find below Python code screenshots

I hope this blog triggered a thought to automate the mundane task of generating pivot tables from spreadsheets or databases. Please reach out to me if you want to check out the full code.

Thank you for stopping by this blog and please share your suggestions below under the Comment section

Up Next – Manage Alerts of VMAX3 array using REST API

Image Courtesy:

Matt Whitt
https://ttorial.com/Images/python-automation-automate-mandane-task-python.jpg

Advertisements

phpoffice/phpspreadsheet

phpoffice/phpspreadsheet is a successor of phpExcel. We recently developed an automated solution to pull all the backup success rate reports from Dell EMC Data Protection Advisor (DPA). The portal which displays the report was good but not enough to use that information further like sharing the reports, pivoting etc..

After trying several methods, found this library to be very useful. There aren’t many examples available on the internet to retrieve data from MySQL and write to XLSX file. Hope this blog might be helpful who would like to use this library for creating XLSX files. Fear not, it doesn’t throw errors while opening this file in MS Office Excel

VMAX SRP Utilization Report & Upload to MySQL DB using Python

I had an opportunity to attend Python Basics Training imparted by Mr. Ashish Gulati. Ashish, a technology coach; was really good in explaining basics of language, data types and Pro & Cons of this language. The way he imparted knowledge was an unique experience. He was flexible to explain some of the real time use cases in data analytics, JSON, SSH connectivity etc.. and explained in detail about the various module’s functionality.

Thank much Ashish for the session, it was informative, very simple topics but effective learning experience, hands on coding was a big plus point. I have learned much that will assist me in my workplace. As an outcome, I have already started migrating from Perl to Python.  This blog is about my first attempt to write Python scripts @ work.

Scripts written in Python would run from several VMAX3 Enterprise Storage Management Servers located at various data centers.

Python_vmax_cap