Creators! Is Your Data Safe?

by Kevin · January 31, 2020

Hello Fellow Readers!

It’s been a long time since I have written an article and thought it was about time to get back the grind. With a lot of various projects and personal struggles getting in the way of my creation goal I am finally back to it!

I won’t needlessly bore you with the details but I will talk to you about Computer breakdowns and data loss that are part of the computing game. If you use a computer and work with a lot of Data whether thats Video, images, audio or any other type of data that needs protection, you need to pay attention. Those of you not spending and investing in the right equipment and strategies to cope with the Inevitable Systems failures that creators face while creating will know what I am talking about.

While we as creators would love to JUST focus on the creation process and less on technology the reality is that it’s NOT possible.. or at least for most of us budget conscience creators who don’t have a whole IT team behind us. Sure there ARE people, systems and strategies and Heavy IT solutions around but you will pay and then keep paying. Not to say these solutions aren’t great, They are worth every penny in most cases, but most of us can’t afford them being single person creators.

What am I talking about here exactly?

I’m talking about those Dark hours during which your Hard drive crashes and you realize you don’t have a backup or that moment where all the data crashes on your PC and you have to reinstall windows and lose all your documents in the process.

I’m talking about Data safety.

Today we are looking at solutions and strategies to avoid this scenario and get YOU the creator thinking about Data protection, Backups and Recovery (and actually doing them)! It really is important and it needs to be available when you need it the most, but its a lot of work and the pre-planning needs to start before the crash, this article hopes to start the conversation and get you thinking about planning your data safety strategy rather than come up with any One single solution for you.

IS YOUR DATA SAFE?

The question is fairly complex but simple at the same time. It’s not a question of if you will lose data but when.

The questions you should be asking are as follows:

If I lost my hard drive today, could I restore the data easily?
If my operating system drive crashes, how long and could I get it back?
Have I recently tested my data to make sure I can restore it in the event of a failure?
Is my data all in one place and could I restore if say a natural disaster, fire or flood wiped out my home? Is my business then in fact gone?
How long would it take to recover everything and is it possible or not?
Is the data I have now in good “Shape” (Integrity) or can I not even open those files for some reason?
When is the last time I ran any kind of Drive or array check on my file structure or drives?

WHAT TO DO NEXT?

If you answered no or were unsure about any of the questions above, congratulations your data is at risk.

Most people do not even know about the physical medium or methods in which your data is actually stored let alone the strategies or methods in which to verify data integrity, so don’t feel too bad however now is the time to take your Data seriously.

Separating your “Data” from the Operating system or applications

The first thing to think about (especially in windows) is separating your Operating system or applications from your Data. The easiest way to achieve this is quite literally to store all your Data separate from your operating system. This isn’t always easy or possible which is why there are tools to “Image” your system’s boot drive as well.

Note: Image is a term that defines technology that can copy Bit for Bit your entire C-Drive as an example, efficient for backing up a whole file system, drive or partition.

But if you are like some users who store everything on the C drive in windows or in the users home drive on Mac, then its time to start thinking differently. Sure, some things can be stored in the Documents folder on windows, but I doubt you would agree the most efficient way is to store every 1080p or 4K Video files on your C-drive. For one it would fill very quickly and the Application from which you are trying to use it would struggle with reading and writing the files while also managing operating system tasks (with some exceptions). As that volume grows you will quickly run into problems and run out of disk space and if that drive crashes, you will lose everything. Single point of failure.

Note: We will cover Application and Data Performance in another article, for now, we are focused on Data integrity and safety.

Options for Separating your Data

A low-cost choice would be to add a second hard drive to your computer and have all your data on that separate from the operating system. This gives a little bit of performance increase but almost no safety in that, that one drive could still fail to cause a data loss.

Here are some options to consider for Data protection:

Buy an extra hard drive as well as an external drive and back up the data regularly using Time machine on Mac or any program like Acronis on PC. There are also numerous file backup utilities to perform this operation on PC for free. I personally enjoy using an older program called Second Copy which can be set up by profile to do EXACT file directory copies and be scheduled to run in the background.
A more robust option is to purchase a NAS device like a Drobo, QNAP or SYNOLOGY device which allows the use of RAID Array Drives (RAID stands for Redundant Array of Inexpensive Disks) which gives a lot more Storage capacity when you add more drives to the array but also safety because RAID allows failure of a drive (depending on RAID level) all while your data is still accessible until you replace a drive. Adding a good external backup of this to the mix and you can get a pretty robust backup strategy here. This is a more
an expensive option but when set up correctly can be well worth the investment.
SAN/NAS level array + Snapshot and Image level operating system backups + cloud-based backups + offisite
A San is simply an array of disks configured in a RAID array usually with a fast interface to the computer. A NAS is a SAN but with Network Operating system or interface that allows it to be accessible over your home network. QNAP, Drobo and Synology all offer devices like this. There is a free one as well called FreeNAS (www.freenas.org) which you can create SAN like features while using an existing computer but my experience has been that it is far better to buy one of these devices as the hardware and software that is implemented is superior to what is in the Free Edition for performance sake. Cloud based backups require monthly subscription charges so that you can store your backups “offsite” by sending to them to the internet fully encrypted for safe keeping. Depending on how much data you have, this can be really expensive for offsite ability however if you have no other way of doing it its still an option

Whoa! NAS/SAN, Cloud ? that’s a lot of things I don’t understand!

You are not alone, most of this technology was invented to create Data integrity and storage solutions for the ever-increasing size of storage but also to create strategies in which you won’t lose your data but also have access to it when you need it most. Being in the Information Technology field for many years, I had the fortune (and misfortune) of learning about Storage and seeing its failures from a very different light.

There are so many solutions out there today for people to choose from but every single method or technology requires some form of configuration and learning curve.

Luckily Technology is ever more increasingly becoming easier to implement than the previous generations. Software is getting better and hardware is becoming more robust, but you need to buy quality name branded solutions and have clear goals and understand what it will do for you and what it won’t.

What Questions will help me decide what Technology to implement?

How much physical Data do you need to backup?
How much does your Data change?
How Fast do you need to Access your Data?
How long can you expect to wait to recover it?
Do you have other means of accessing the Data if your primary computer goes down?
What is your Budget for Dataloss, recovery and uptime/accessibility?
How much time do you have to do manual tasks to ensure your data is getting backed up properly or do you want a fully automated solution?

Answering some of these questions will make you realize that Your data’s LIFECYCLE and Integrity are important and will take a bit of planning to create a solution that works for your needs.

WHERE TO START?

After answering those questions you will come to the conclusion (hopefully) that your data safety needs are not being met and you need to start strategizing.

Some of the fundamentals of the data safety also involve security but that’s a separate topic for another day.

Starting at the beginning with an example to help you visualize:

For example, let’s assume Creator A has about 3 TB of data (that’s terabytes which is 3,000 GB of data or roughly one 3 TB hard drive worth of data).

That 3 TB is growing monthly by let’s say 500 GB or so. The user has one workstation with Windows 10 on it and lots of other documents, files and business and adobe software lets say but their Video and photos are on a separate disk. So the user currently has a computer with 1 SSD drive (solid-state hard drive which is faster than a normal hard drive) and one 3 TB hard drive that’s full.

The initial concern

The user goes and buys an external 6 TB external USB drive, this way they can back up that drive and does so with a little space to spare.

What about the Operating system SSD drive and files?

There are two options here, first, the user can use Windows snapshot technology and save it to the 3 TB drive as a backup.. but wait he can’t because the data drive is full (but snapshot locally to the C drive is okay), so he saves it to the External drive temporarily (thus creating a separate copy of the Snapshot image). The image file that windows create will in the event of a failure allow him to restore however the user must also create a Windows 10 repair USB image so he buys a flash disk and installs the windows repair tool so that its bootable to his machine. Note: there are a lot more steps here and I am only scratching the surface here however let’s assume the user reads all of the online documentation and successfully creates a windows 10 repair USB bootable flash drive.

Next Steps

The next concern in our fictional story is that the user has no data space left on his data drive. The user must not Save anything other than a backup to the new external drive they bought as that would negate having a backup copy and any new data would then not be backed up.

What are the options in this case?

The user could buy a bigger data drive in their computer however that only solves the space problem not the data safety issue.

What happens in the event the user’s new Data drive fails before they are able to back it up? that would be very unfortunate as everything created after the last backup would be lost even though the user is backing it up.

If the data doesn’t change that often this might be an okay situation but for most of us, it isn’t.

Options:

If your PC has more drive bays and has RAID functionality on-board your computer (software RAID) you could get two more drives and add them to your data drive (backing it up first) and create a 3 drive RAID 5 array using your Systems RAID software like Intel’s Rapid Storage Technology or Intel RST. The result is a fault-tolerant array that in the event one disk fails, the software will rebuild with another drive once installed by you. There are many downsides to this config and I myself ran this for years but it is none the less an option that won’t cost much in terms of purchases, however, it does in time and reliability.
A better option at this point quite frankly is to go out and purchase a decent USB 4 bay RAID enclosure instead of setting up internal drives and bays. Get a RAID 5 capable USB 3.0 or if you have a newer PC Thunderbolt or USB C (3.1) RAID drive enclosure. Fill it with 3 or 4 drives of sufficient size for the Data you have and how much room you want to grow to. Keep in mind in a RAID 5 array you lose 1 drive to parity so as for example buying Four 6 TB drives would yield you 18 TB aprox of volume size of space, not 24 TB.. You could then just run your Data drive off your computer separate from the computer’s operating system. You could also buy yet another of these enclosures, setting it up as a Backup drive and have it removable so you can take it offsite. This setup although it requires a lot of setup, it isn’t too bad in terms of features and protection it will give your data. You may need to purchase software to back up your data drive or try some of the many free programs available. I went ahead and bought Second Copy which is a nice little program to copy my data over to the backup drives and schedule it so I don’t even have to be around when it runs but you could easily create a script to run Xcopy from windows as an example.
Every solution after this will increase performance, data reliability features sets and cost. For example, that removable USB RAID array might be okay for your needs, but what about its interface speed? What about when a drive fails, how reliable is that cheaper option to properly recognize a new drive? What about support options? Probably none available. What if you need to ever expand the array to go bigger? You see the point..
The next step up (a big step up) is a product called a DROBO which manages the data for you and makes your life more easy by allowing you to upgrade using any size of drives and it has 5 bay as a starting point. They work for Mac and PC and the software tells you how much space. They have full hardware RAID in them (a more robust hardware version of RAID) that works better than the cheap stuff so that your hard drives are managed correctly. Note: Not all hard drives are made equal and the various types can be confusing. Drives are generally inexpensive for the task they perform but getting the correct types for NAS appliances or RAID arrays can and will make your data safe experience and they do cost more but better as the products are designed to work in those devices more readily and have been “tuned” for that purpose. There are of course exceptions to this rule in every scenario but on par “NAS labeled drive” are what you should be buying. They also generally have better specs in terms of times between failure and warranty.
If you want to go for more features and data reliability, going with something like a Synology or QNAP NAS device allows your data to independent but also is a full data management computer on its own. Many of these types of devices can be run as true NAS servers with a full range of features running Linux based operating systems. Interfaces are typically built into the units for lightning-fast transfer speeds over USB 3 or 3.1 speeds. Optional interfaces like 10 Gig Ethernet allow the creation of iSCSI Volumes on these devices which allows greater volume size creation and allows the ability to connect to a volume over your home network right to your windows machine adding that Ethernet allows a form of data correction over the wire, so dropping the connection, for example, will never lose data like USB might. Connections up to 40 Gbit/per second with Thunderbolt 3 as an example can move your data really fast with no signs of slowdowns and if you have a lot of data to move around or backup, interface speed is a must. These units also have data integrity checks that can be scheduled or run on your drives and your volumes. SMART scans, Raid Parity auto rebuilds when a drive fails and also Drive scrubbing which is an automated scans of the individual bits of each drive and if the data on the physical drive’s magnetic signature reports and error (See Bit decay) the drive can use parity information and re-write that bit somewhere else on the drive or just REFRESH the bit on the drive. There are also snapshot technologies that allow super-fast backups to another volume on the array of drives so that you can backup from the snapshot while still running your critical applications at full speed from the main data volume. There are also integrated Cloud connectivity solutions for managing cloud-based backup sites. These strategies and features are all related to the Safety of your data and how redundant and secure your information is so that data loss is less likely.

That’s a lot of hardware options! How to I figure out what I actually need?

A lot of the decision is going to be what budget you have to work with and how much data safety, data time to recover and data sizes you are working with.

Budget should be the least consideration unless you are really financially strapped. Put a price on the loss of your data and then act accordingly. You do get what you pay for though however, the best and quickest solutions for backup may not be the most reliable and the most costly solutions often come with more options and features and safeguards for your data, however, there is a learning curve.

Time is also a huge factor here. Your time is also valuable and if you’re like me, you do everything yourself. Whether it be transfer time, recovery time, backup creation time, error-correcting time or just time managing the backup reliability in order to make sure it backs up as expected.

At the very least you should consult an information technology friend or service to determine your needs and get professional help for those situations you can manage on your own.

Conclusion

I hope this article will inspire all of you creators out there to take a serious hard look at what a backup solution really does and make sure it works for you when you need it. Making the proper investments in this technology are critical to all of us in the digital age.

At the very least please make and do regular backups of your data and test that you can access that data reliably because when the time comes that you absolutely need that data and you don’t have access is the wrong time to be thinking about this stuff.

I hope you enjoyed the article,

Until next time!

Keep those backups working!

Please take the time to visit these Amazon Affiliate Links below! All the equipment I recommend or have used myself.

Please note: I get a small commission from the links below but only if you check out and purchase the items 24 hours from the time you add the items to your cart. If necessary please re-visit this page and add the items back to your cart and check out if or when you are prepared to order. Thank you.

USA Links:

Canadian Links: