Corporate Data Growth: Is your data outpacing its budget?
You don’t have to venture very far to hear or read about the Big Data growth rate. While researching this post, a quick search produced more hits and info graphics than I could even share on the topic. The research firm MarketsandMarkets predicts that the global Big Data market will show nearly a 26 percent compound annual growth rate from 2013 to 2018.
What Happened to Budgeted Growth?
If you are like most IT professionals I know, you don’t need some market research analyst to confirm what you’ve been dealing with in your own environments for the past several years. Honestly, I’m still amazed by how fast some of our clients create new data. If you are a CIO or CFO, one of your first clues might be the constant addition of storage, or perhaps how your data protection fees continue to steadily rise.
In addition to the responsibility to maintain the systems that store, serve and protect this ever-growing mass of unstructured data, the IT department is often left to protect this data growth on a shrinking or stagnant budget. After doing a very informal analysis of our data protection Public Cloud clients, I commonly see growth rates of 8-12 percent per month of compressed, de-duplicated, stored data rates. In my experience, it seems like a healthy amount of unstructured data growth can be attributed to duplicated data across the organization. The only data users are good at deleting is the data they actually needed to keep.
What Can Be Done About It
Every organization varies on its ability to “socially re-engineer” its users toward a slower-paced data growth mindset. As IT professionals, we often take on a role of educator to help users realize they should possibly send links to commonly accessible files or that they don’t need a hundred revisions of their commonly updated Excel spreadsheets. This reduces the dual burden of large attachments in the Exchange Server and keeps users from saving the attachments back to the shares in a different location. The jury is still out on whether implementing a SharePoint system helps or hurts data growth, so I won’t even dip my toe into that discussion at this point.
Changing user behavior will always remain a long-term battle for IT departments. Having some simple tools at the IT department’s disposal would be a big benefit. Having a data protection suite with these types of tools built in, and that can automatically run, would be the greatest benefit.
Data Protection Tools and Tips
Some of the same tools we use during our assessment phase of onboarding a potential new client can also be utilized on an ongoing basis to help the IT department tame the Data Growth conversation inside their organizations.
- LAN Scans
Our Asigra-powered data protection solution provides a LAN scan tool which discovers all shares on your network, the number of files, and total file size per share. It also reports how many duplicate files are found and the total duplicates’ sizes. It details the top 100 largest files found and their location. It even tells the top 100 largest duplicate files. A per-user break down is reported, so you can easily target the user storing non-business related content such as personal music and movies files. The LAN Scan tool also reports on the dormancy of the data on a given share. Finally, the tool provides a bit of trending via a growth and modified file report.
- Unprotected Resources
Over time, new resources get added or shifted around to a point that they may not be included in the current backup jobs. Having a recurring job that runs on a monthly basis detailing resources that aren’t currently being protected can be a major benefit.
- Retention Policies
A backup and data recovery (BDR) suite that is capable of very detailed retention policies is worth its weight in gold. All data is not created equal, and IT departments need the capability and flexibility to customize retention rules so they can properly protect and maintain their data. This isn’t always easy for an IT department to do alone, so having buy-in from key stakeholders across the organization is always a good day. The payoffs can be huge from a data protection cost perspective.
- Archive System (i.e. Backup Lifecycle Management -BLM)
If you currently are not dealing with aging data in your organization, then you should start to look into it. As data ages, it really should be sent to a separate system for longer-term storage, just as we used to do with long-term storage of physical file folders. They were boxed and shipped to a long-term storage warehouse or a third party. Data files should be treated in a similar manner. If you are keeping backup data that is older than one year in your live backup vault, then you really should move it to an archive system capable of handling that data as it ages and expires. If you have compliance concerns, then this is a must, not a suggestion. A backup archive system is one that is capable of storing data in a format that is more easily searched while keeping track of the data’s age. The final phase in the backup lifecycle is, of course, a system capable of issuing a data destruction certificate once the expired data is properly deleted.
Finally, the big takeaway here is that we all need and hopefully have data protection solutions in place. I’ll let you decide if what you use is the right solution for your organization, but I’m of the opinion that data protection solutions should provide additional benefits in managing the very data they protect.