Python’s sh Library

Wish you all Happy, Prosperous & Pythonic New Year 2019!!

I thank everyone for stopping by my blog for 15K times in 2018 which is the highest so far! 

“Looking back at my life’s voyage, I can only say that it has been a good trip.” ~ Ginger Rogers

Similar to the above quote, I’d started my scripting voyage by writing shell, batch and Perl scripts in good old days. I miss some of the awesome, builtin & yet simple to use features available in shell. I felt using those features or builtins would make my life easier while writing scripts in Python…

Besides that, one main reason to look back at the past was while imparting the Python Workshop at my office. One of the participants raised concerns over Python’s builtin modules like OS and Subprocess. They mentioned we can accomplish with very simple steps in ‘Shell’ instead of using OS or Subprocess module. I had to accept the fact that sometimes Python is not so easy as we think!

After googling for a while I finally found this… Say hola to ‘sh‘ Module!

The ‘sh’ library provides simple and intuitive alternative to OS/System/Subprocess modules.

‘sh’ module simplifies Python’s ability to interact with native OS by calling shell commands. This would greatly helps in automation of routine tasks and running sequence of commands, parsing output as per requirement.

Python is a powerful language powered by those great developers who are continuously contributing to enable new features every day in day out. I would like to thank Andrew Moffat for the ‘sh’ library. This library is one of my favourites.

To install this library using PIP, run below command

pip3 install sh 

Please find sample scripts output. 

>>> import sh

>>> sh.uname()

Darwin

>>> sh.uptime()

 7:13  up 11 days, 23:29, 4 users, load averages: 1.51 1.65 1.88

>>> 

>>> sh.echo(‘Hello, This is shell\’s echo running from Python’)

Hello, This is shell's echo running from Python

>>> sh.who()

vinay    console  Dec 21 07:45 

vinay    ttys000  Dec 21 07:46 

vinay    ttys001  Dec 21 07:46 

vinay    ttys003  Jan  2 07:02 

>>>sh.df()

Filesystem    512-blocks     Used Available Capacity iused               ifree %iused  Mounted on

/dev/disk1s1   236568496 96124200 131754736    43% 1042911 9223372036853732896    0%   /

devfs                376      376         0   100%     653                   0  100%   /dev

/dev/disk1s4   236568496  6291496 131754736     5%       3 9223372036854775804    0%   /private/var/vm

map -hosts             0        0         0   100%       0                   0  100%   /net

map auto_home          0        0         0   100%       0                   0  100%   /home

>>> sh.ifconfig()

lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384

options=1203<RXCSUM,TXCSUM,TXSTATUS,SW_TIMESTAMP>

inet 127.0.0.1 netmask 0xff000000 

inet6 ::1 prefixlen 128 

inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1 

nd6 options=201<PERFORMNUD,DAD>

gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280

stf0: flags=0<> mtu 1280

XHC20: flags=0<> mtu 0

en0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

ether b8:e8:56:36:0b:58 

inet6 fe80::149c:1b88:578c:73f8%en0 prefixlen 64 secured scopeid 0x5 

inet 192.168.1.9 netmask 0xffffff00 broadcast 192.168.1.255

nd6 options=201<PERFORMNUD,DAD>

media: autoselect

status: active

p2p0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 2304

ether 0a:e8:56:36:0b:58 

media: autoselect

status: inactive

awdl0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1484

ether 8a:0c:61:9e:64:9b 

inet6 fe80::880c:61ff:fe9e:649b%awdl0 prefixlen 64 scopeid 0x7 

nd6 options=201<PERFORMNUD,DAD>

media: autoselect

status: active

en1: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500

options=60<TSO4,TSO6>

ether 72:00:00:20:b1:90 

media: autoselect <full-duplex>

status: inactive

en2: flags=8963<UP,BROADCAST,SMART,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500

options=60<TSO4,TSO6>

ether 72:00:00:20:b1:91 

media: autoselect <full-duplex>

status: inactive

bridge0: flags=8863<UP,BROADCAST,SMART,RUNNING,SIMPLEX,MULTICAST> mtu 1500

options=63<RXCSUM,TXCSUM,TSO4,TSO6>

ether 72:00:00:20:b1:90 

Configuration:

id 0:0:0:0:0:0 priority 0 hellotime 0 fwddelay 0

maxage 0 holdcnt 0 proto stp maxaddr 100 timeout 1200

root id 0:0:0:0:0:0 priority 0 ifcost 0 port 0

ipfilter disabled flags 0x2

member: en1 flags=3<LEARNING,DISCOVER>

        ifmaxaddr 0 port 8 priority 0 path cost 0

member: en2 flags=3<LEARNING,DISCOVER>

        ifmaxaddr 0 port 9 priority 0 path cost 0

nd6 options=201<PERFORMNUD,DAD>

media: <unknown type>

status: inactive

utun0: flags=8051<UP,POINTOPOINT,RUNNING,MULTICAST> mtu 2000

inet6 fe80::bd0:9dea:48de:5645%utun0 prefixlen 64 scopeid 0xb 

nd6 options=201<PERFORMNUD,DAD>

>>> 

Thanks for stopping by, please share your comments and ideas to improve this blog. Keep watching for more Python libraries used for automation.

Advertisements

Automate Cisco switches config using NETMIKO

Netmiko is a simplified module to manage Network devices via SSH. This module is developed by Kirk Byers. I would recommend all network engineers to learn and equip this module to automate day to day configuration tasks.

netmiko_show_arp

I’ve written a demo script to disable SNMP Server’s “globalenforcepriv” on both Cisco IOS and Nexus switches. Using this script we could make the necessary changes on 100’s of switches in few minutes saving time and energies of network engineers.

Al Sweigart wrote a book called ‘Automate boring stuff using python’. This is one of the scripts which could help network engineers to automate monotonous tasks using Python and Netmiko module.

How to maintain currency of SOPs

SOP (Standard Operating Procedures) Optimization

  • Conduct a workshop to assess your SOP documentation and derive an action plan
  • Develop a risk management plan and associated procedures covering your IT-related operations
  • Implement risk-based methods in key processes and procedures, like validation planning, testing, change control
  • Streamline SOP’s pre-checks, implementation steps and post-checks. Automate wherever applicable to reduce the manual efforts and, importantly, to avoid human errors; whilst adhering to process resulting in improved turnaround time
  • Optimize selected processes using proven methods and techniques, deploy fully developed pre-filled templates, e.g. for testing or change tickets
  • Identify processes where automated tool support is highly beneficial and define business requirements for software tools. Remember to follow the approach    People >> Process >> Tools and avoid the reverse approach
  • Evaluate whether vendor support required for the chosen tools based on your business requirements. Support increases maintainence cost but there is always experts to help you out in burning situations
  • Manage the implementation and validation of selected software tools
  • Maintain the SOPs and knowledge objects in a repository which should be ease of use for end users with no frills and fancies.
  • Train your employees in applying the revised SOPs and seek feedback to make continuous improvement
  • Setup a quarterly review meeting with all stake holders including end users to make timely amendments and release SOPs by following standard approval process

Friendship Day

Once upon a time there were three best friends named Rosie, Lily and Shimmer; they were fighting. Rosie wanted to go to park but Lily wanted to go to the zoo but Shimmer wanted to go to shopping mall. Rosie got angry and went to a forest. She got trapped in a cave. After a while Rosie thought about her friends so she felt sad. Her friends missed her a lot. Rosie went deep into the cave she saw a bird she made it as her pet. Next day Rosie and her pet came out of cave. She saw a waterfall and saw her face and her friends faces. Then her friends came to her they hugged her. Suddenly Rosie fell down from her bed and said it’s just a dream then she went to play with her friends.

Moral – friends are always with us

THANK YOU

by-SAANVI RAO

(‘Byron’ – 3rdGrade, Mount Litera Zee School, Bangalore)

P.S: My 8 year old wrote this story for Friendship Day. I’m posting here without making changes 🙂

Happy Friendship Day!!

Telnet (NetCat) and Ping Script

Nowadays most of the Linux distros do not include Telnet. If needed we can install it using the usual methods.

Let’s embrace Netcat which is a best alternative to Telnet. Earlier I had written a Perl script which did Telnet and Ping checks across multiple hosts.

This script requires two input files ips.csv and ports.csv. As the name suggest all target host IP’s to be placed in ips.csv file and the ports to which to netcat is placed in ports.csv. Output is in HTML format and user receives an email with colour coded results.

Replace ips.csv & ports.csv file contents and change SMTP IP and email address.

Here is the Python script which replaces Telnet with Netcat.

Integrate Rundeck notifications with Slack

There are many plugins available for Rundeck to integrate with Slack. In this blog, I’ve explained in simple steps to configure Rundeck Job Notifications with Slack. This blog is useful for the use-cases like audit, monitor and maintain logs of Rundeck job executions.

I spent many hours of searching and fixing unexpected errors/issues in different version of Rundeck, which are all covered and documented as simple steps in this blog.

I’m using Higanworks’s plugin downloadable from GitHub

Advantages:

  • Single window to view all notifications from Rundeck {better than pile of emails}
  • Multiple users/groups can be notified by adding them to the notifications channel {no need to mess with distribution lists etc}
  • Logging and Auditing now made easier by using the powerful search options available in Slack

Requirements:

  • Rundeck 2.10.x or above {Running on CentOS 7}
  • openjdk version “1.8.0_171”
    OpenJDK Runtime Environment (build 1.8.0_171-b10)
    OpenJDK 64-Bit Server VM (build 25.171-b10, mixed mode)
  • Rundeck-slack-incoming-webhook-plugin v.0.6.dev or above
  • Working Slack user account,
  • Dedicated channel for Rundeck notifications with webhook app enabled

Downloads:

Pre-Installation Steps

  • Create a new private channel in slack {e.g: rundeck_notifications}
  • Webhook URL for the newly created channel. Refer Slack guide

Install and Configuration Steps

By now server is ready with installation of Rundeck and plugin downloaded from the Github. Make sure Rundeck server has internet access to connect and send messages to slack.

  • Copy rundeck-slack-incoming-webhook-plugin-x.y.z.jar executable file to Rundeck’s libext directory {/var/lib/rundeck/libext}

2018-06-17 11_11_23-Mint [Running] - Oracle VM VirtualBox

  • After the above file is placed in the libext directory, Rundeck automatically configure the plugin and no further user actions required. No need to restart Rundeck service. Please refer Rundeck Plugins Installation Guide for further details.
  • Below screenshot show new option ‘Slack Incoming Webhook’ available while Creating/Editing jobs in Rundeck. Paste here the webhook generated for new rundeck_notifications channel

configuration

  • Sample output for reference. Slack channel rundeck_notifications showing notifications generated by Rundeck Job executions

2018_06_17_11_24_31_Slack_Rubicon.png

References:

https://github.com/higanworks/rundeck-slack-incoming-webhook-plugin

http://rundeck.org/docs/plugins-user-guide/installing.html

https://api.slack.com/incoming-webhooks

Image Courtesy:

https://github.com/higanworks/rundeck-slack-incoming-webhook-plugin

If you are facing any problem let me know by using ‘Comments’ section below and I will try my best to help you.

Rundeck SSL Configuration

Rundeck is an OSS that automate routine operational procedures in data center or cloud environments. Here is the blog to configure SSL on Rundeck for secure transaction within intranet and internet. This blog is a reference to configure SSL for Rundeck running on Linux (CentOS/Debian)

Phase 1:

Steps to generate self-signed PKCS#12 SSL certificate and export its keys:

  • Create PKCS#12 keystore (.pfx file)
#keytool -genkeypair -keystore myKeystore.pfx -storetype PKCS12 -storepass password -alias KEYSTORE_ENTRY -keyalg RSA -keysize 2048 -validity 99999 -dname "CN=My SSL Certificate, OU=Sustaining, O=Virtustream, L=McLean, ST=VA, C=US" -ext san=dns:servername.com,dns:localhost,ip:127.0.0.1,ip:xx.xx.xx.xx

Replace servername.com with FQDN of the Rundeck server and xx with Rundeck server IP address