Docker is a computer program that performs operating-system-level virtualization, also known as “containerization”. It was first released in 2013 and is developed by Docker, Inc. source: Wikipedia
How Docker works?
Docker containers wrap up software and its dependencies into a standardized unit for software development that includes everything it needs to run: code, runtime, system tools, and libraries. This guarantees that your application will always run the same and makes collaboration as simple as sharing a container image. source: www.docker.com
Docker unlocks the potential of your organization by giving developers and IT the freedom to build, manage and secure business-critical applications without the fear of technology or infrastructure lock-in. source: www.docker.com
In recent times, we’re asked to develop scripts to capture performance metrics from EMC VMAX storage. There is a ‘symstat’ command with many attributes to capture performance metrics information from the array. But this command was not fulfilling all our requirements. While exploring various options and consultation with EMC support / community we decided to try Unisphere / RESTAPI.
So far I was using Perl as THE LANGUAGE to talk to my storage arrays. But I was forced to switch over to Python which works best with REST API / JSON. Additionally, there are lots of code out there on RESTAPI written in Python. So it is easy to ‘get inspired’ by those codes and write customized code for our requirements. So this would make me yet another ‘Pythonistas’ 🙂
This is my first ever Python (version 2.7 on GNU/Debian Linux) script to capture EMC VMAX Performance Metrics retrieved from Unisphere for VMAX (version 8.2) via RESTAPI. I’ve referred this Python script to develop custom script to suit our requirements. Many thanks to Matt Cowger (mcowger) for sharing the script in Github.
There are plenty of metrics that can be captured using this script but I’ve written a simple code for demo purpose to print few metrics in CSV format which can be either imbibed by excel for further reporting / charting or injected to MySQL DB to do many stuffs…
Here is the sample + cropped output for reference. In the below table timestamp (column B) is in epoch format which is converted to MYSQL datetime format via INSERT query
P.S: I’ve changed VMAX serial number for various factors
If interested, please reach out to me to get these Python scripts.
Storage capacity reporting has become a tedious task these days unless we don’t have tools in place. These tools are not as cheap we can think of. There are additional layers added to it to hype and increase its value in market. But if customer ask a storage admin to run the report manually then its a nightmare for the admins.
We always want to go by the easy way which means GUI / EMC Unisphere but for reporting this doesn’t helps to customize and fulfill the customer requirements . EMC SYMCLI / Solutions Enabler can be used effectively here to address this issue. We can use SYMCLI to automate these manual reports. By means of automation, we can ensure quality, timely and error-free reports are generated which can be scheduled via CRON or Tasks Scheduler to send it directly to the stakeholders. SYMCLI built-in supports XML formatted output of the commands which would become easier to parse the information using XML supported programming language.
I’ve developed PERL scripts to generate Storage Pool capacity and Disk capacity report for multiple VMAX storage arrays which are managed from the SYMCLI / Solutions Enabler server. Following are the benefits of using this script.
1. Generates Pool capacity report for all storage arrays which has columns – SYM ID,MODEL,TOTAL USABLE POOL CAPACITY TB,TOTAL USED POOL CAPACITY TB,TOTAL FREE POOL CAPACITY TB, TOTAL POOL UTILIZATION %, TOTAL POOL SUBSCRIBED %
2. Generates Disk capacity report for all storage arrays which has columns – SYM ID,MODEL,TOTAL EFD,TOTAL FC_SAS,TOTAL SATA_NLSAS,FORMATTED EFD CAPACITY GB,FORMATTED FC_SAS CAPACITY GB,FORMATTED SATA_NLSAS CAPACITY GB, TOTAL FORMATTED CAPACITY GB, UNFORMATTED EFD CAPACITY GB,UNFORMATTED FC_SAS CAPACITY GB,UNFORMATTED SATA_NLSAS CAPACITY GB,TOTAL UNCONFIGURED CAPACITY GB,TOTAL UNFORMATTED CAPACITY GB
Output generated from the scripts will be in the form of CSV format. This is information can be plugged into desired format and generate reports with Pivots, charts etc…
Please find below sample output for reference.
Pool Capacity Report:
Disk Capacity Report:
P.S: I’ve removed VMAX serial number for various factors 🙂
If interested, please reach out to me to get these PERL scripts.
I hope you guys and gals had wonderful festive days (Both Ramazon and Ganesha Chaturthi). It was long time since I wrote here. In today’s article, I will write about Symmetrix LUN allocation from A-Z. I hope this blog would help you all to understand the complete allocation steps.
If you want to learn SAN. Go ahead and start troubling him to teach you…(Don’t say that I told to do so…)
Knowledge on Basic Symmetrix Architecture
Operating systems knowledge
and a test Symmetrix, hosts to try
So, my style of writing (of-course I learnt it from Mohan Sir) will be in terms of steps. Because I believe that writing in steps will help us to remember and understand quickly. I will be using Symmetrix Command Line (SymCLI) to allocate LUN’s. Let me stop the blah blah and start the blog!
Symmetrix Allocation Steps
Step 1: Create symmetrix devices from the free space.
To create a symmetrix device, first we need to know what type of device we need to create. For example, RAID-5, RAID-1 etc… I’m going to write both the commands. To start with, we need to create a simple text file and add the below line to the file.
create dev count=xx, size=17480, emulation=FBA, config=2-way-mir, disk_group=x;
dev count=xx (replace xx with the number of devices we need to create)
emulation=FBA (FBA > Fixed Block Architecture used for Open Systems which are Solaris, HP-UC and Window$)
config=2-way-mir (Configure the devices as RAID-1, one of the oldest configuration available in all symmetrix models)
disk_group=x (disk groups are created to differentiate the tiers, performance and capacity. Based on the requirements, we can select the desired disk group number to create the new symmetrix devices)
Once you add the above line in the text file, save it and check the syntax. To make any configuration changes in the symmetrix we need to run the below mentioned commands. Ensure that the raid1.txt file in your current working directory.
symconfigure -sid xxxx -f raid1.txt preview -V
symconfigure -sid xxxx -f raid1.txt prepare -V
symconfigure -sid xxxx -f raid1.txt commit -V
symconfigure (This command used to manage major configuration changes, display capacity of symmetrix and manage dynamic (hot) spares and device reservations)
-sid (Symmetrix ID, always prefix with hyphen (-) )
-f (filename, mention the file name to which we’ll use. In this example it is raid1.txt)
preview (The preview argument verifies the syntax and correctness of each individual change defined, and then terminates the session without change execution.)
prepare (The prepare argument performs the preview checks and also verifies the appropriateness of the resulting configuration definition against the current state of the Symmetrix array)
commit (The commit argument completes all stages and executes the changes in the specified Symmetrix array.)
-V (Yes, you’re right, its verbose mode)
That’s it! After running the above the commands the new devices are created. It was to create symmetrix devices from the symcli right. Let us assume that the devices ID’s are 001 through 00A (devices created with hexadecimal numbers)
Step 2: Search for free LUN ID on the FA (Fibre Adapters)
After creating the devices, we need to map the devices to the Fibre Adapters. In legacy symmetrix, it will be SCSI Adapters (SA). IF we need to do it from ECC (EMC Control Center now called as IONIX, we need to a. Right click on the device b. Go to ‘Configure’ c. Select and Click ‘SDR Device Mapping’ and follow the wizard. Here I’ll be writing the commands to do the same.
symcfg -sid xxxx list -available -address -fa xy -p n |more
symcfg (Discovers or displays Symmetrix configuration information)
list (Lists brief or detailed information about your Symmetrix configuration.)
-available -address (Requests the next available Vbus, TID, or LUN address be appended to the output list. Used with the -address option.)
-fa (Confines the action to a Fibre Adapter (FA) director number)
xy (x is the director number eg. 8 and y is the processor number eg. a or b)
-p n (p is the port and n is the number eg. 0 or 1)
Up to DMX-4 we follow the RULE-17. So repeat the command for another FA in this e.g. 9.
Step 3: Mapping a device to the FA
Now, lets assume that the LUN ID’s 52 onwards are free to use on both the FA’s 8a and 9a. Also, the file map.txt contains the below commands. After saving the file, we have run the symconfigure commands as shown above in Step 1. The below command will map the device 0001 to the FA’s 8a:0 and 9a:0 which will have the LUN ID’s 52 on the FA’s.
map dev 0001 to dir 8a:0 target=0,lun=52;
map dev 0001 to dir 9a:0 target=0,lun=52;
map (map a device to fa)
dev (symmetrix device ID)
dir 8a (FA:port no#)
target (The SCSI target ID (hex value)
lun (Specifies the LUN addresses to be used for each device that is to be added for the host HBA.)
Step 4: LUN Masking a device to the FA
The last but one step is to do the LUN masking. It performs control and monitoring operations on a device masking environment. After running the below command, provides RW accessibility to the server having 2 port HBA’s for the device 0001.
symmask (Sets up or modifies Symmetrix device masking functionality.)
-wwn (World Wide Name of the Host Bus Adapter (HBA) zoned with the FA)
add (Adds devices to the device masking record in the database with the matching WWN)
Step 5: Update and refresh the Symmetrix database VCMDB (Volume Control Manager Database) Steps to perform from the HP-UX server
The below command looks pretty simple but very important command to append all changes to the VCMDB. This will ensure and update the DB which protects the changes made the symmetrix.
Symmask -sid xxxx refresh
refresh (updates and refreshes the VCMDB)
To confirm the symmetrix allocation done properly are not we can run the symcfg command as shown above in the Step 2. The output should show the LUN ID 52 being occupied by the Symmetrix device ID 0001.
Hurray!! This completes the symmetrix allocation steps.
Steps to perform from HP-UX server
Once we finish the task from the EMC end, we need to scan for the new LUN from the operating system. First we’ll deal with HP-UX server. The following commands needs to be executed in the same order from the server to start using the new device.
#ioscan -fnC disk
The ioscan command displays a list of system disks with identifying information, location, and size.
The insf command installs special files in the devices directory, normally /dev. If required, insf creates any subdirectories that are defined for the resulting special file. After running this command the new devices will be added to the /dev directory as special device file.
Configure logical devices as PowerPath devices. It will search for the EMC devices and adds it as a power path devices.
#powermt display dev=all
Displays configured powerpath devices. If the previous command ran successfully then we should see the new device in the output list.
Save a custom PowerPath configuration. Once we see the new device in the previous command output, then we can save the power path configuation database.
After the successful completion of the above steps we can use the devices. Further by using the volume managers we can add them to the existing volume group or the new volume group.
To Allocate LUN’s in V-Max array, following steps has to be performed. (courtesy: Sanjeev Tanwar)
1. Create a storage groups (Containing symm devices) 2. Create a port group (one or more director /port combinations) 3. Create an intitors group (one or more host wwns) 4. Creating a masking view containg the storage groups,port groups, and inititors group.
When a masking view is created,the devices are automatically masked and mapped.