Data Storage cleanup/reclaim is similar to how we manage “kinds of stuff” in our homes. We think what to keep and not to after taking the holistic decision. Time & Value will decide what is to be discarded in near future…
An All-Flash Storage array (AFA’s) gives the best performance and greater value to the business. But every GB of All-Flash Storage is at least 3 times costlier than that of mechanical drives. It is critical to perform frequent clean up of these kinds of arrays to make sure that only the mission/business critical data and deserving information should reside on All-Flash array.
From the business perspective, the concerned team needs to do the regular drills to identify then inform IT Ops to discard stale or amassed data ASAP. This requires the team to do a logical distinction of information based on value or tasks, seek approvals from respective Business Unit Owners before proceeding to do clean up.
But the other form of amassed data which resides in the array is duplicate of primary data by means of clones and/or snapshots. We create a point in time copy of primary data during the execution of critical upgrades, maintenance, testing etc… In most cases, this kind of duplicate data exhausts storage capacity silently. “Cloud” administrator need to be cognizance of the same and take actions periodically. Due to lack of resources, workload, BAU tasks, tickets and service requests; administrators infinitely push or delay the cleanup or ‘reclaim’ tasks.
A very beginning step to clean up or reclaim capacity is to identify amassed storage capacity sitting there by means of snapshots or clones. This is painful when you’ve dozens of AFA’s to manage. To solve this problem, we’ve come up with automated scripts to run through all DELL-EMC VMAX3 AFA’s and generate a spreadsheet to list down the snapshots with aging information. This would greatly help our administrators to plan for reclaim, take advantage of automation to wipe off unneeded duplicate data.
How scripts work?
1. Python script is executed via Rundeck to the respective storage management nodes
2. Script will execute SYMCLI command on the VMAX3 arrays to fetch the snapshot information
3. Calculate aging of snapshot based on the date & time stamp
4. Step 1 to 3 would be executed across all storage management nodes
5. Another Python Script collate all CSV output from individual nodes into a single spreadsheet with necessary formatting and
6. Email/Slack reports to concerned folks for proceeding with further actions
Please find below sample output for reference. Below report provides Data Center location, Array Name, Serial Number, Device ID, Snapshot Name, Date & Time of Snapshot and Snapshot Age in days.
If interested, please reach out to me to get these scripts.
Thanks for stopping by… Please leave your comments/suggestions below.
“People are still crazy about Python after twenty-five years, which I find hard to believe.” ~ Michael Palin