4K editing workstation components on a budget
What is RAID and why do you need RAID ?
RAID provides redundancy but it also provides performance in certain RAID levels. It is more expensive because it requires more storage hard drives (or SSD drives) to be purchased.
For example RAID 5 for a 16TB minimum usable configuration would be 3 x 8TB drives. RAID 10 would need (4) 8 TB for the same redundancy however RAID offers far greater performance in Write speeds than RAID 5.
RAID saves you headaches so that you lessen the risk of losing data and increase your performance at the same time.
Interfaces for your RAID Storage array
You can have numerous interfaces for your storage array. What i mean by an interface is the physical / logical connection between your storage array and your computer. That data has to travel back and forth (read and writes) fast enough so that your machine isn’t bottlenecked by it. bottlenecked means it slows down to the point of causing an issue. Just think of the highway when there’s a reduction in the amount of lanes, traffic builds up and slows everything down. Your I/O is much like that bottlenecked highway. The good news is the more money or technology you can muster the more lanes on that highway you can support.
The most commonly used Interfaces:
USB3.0 (or now 3.1) – USB is an external Serial Interface that anyone who uses a computer is familiar with. Its plug and play, simply plug it in and its ready to use (more or less). For storage this is a viable option because of the speed on the “interface”. A USB 3.0 on average depending on a lot of different factors can give you that average throughput speed of 85 MB/s to 100 MBps (Burst cached speeds are faster up to 400MBps) or more performance.
this equates to a max of 5,000 Gbps theoretical limit. Performance isn’t always guaranteed however as performance also relies on the motherboard, the drivers, the processor on the RAID or external device and the operating system you are using. It is the least favorite option of mine, but the best in terms of compatibility, convenience and common form factor out on the market for external RAID or drive arrays that you can buy. It is a good second option. Also, USB 3.1 has been released effectively doubling this speed to a theoretical max of 10 Gbps.
Internal SATA RAID (Software RAID) – This is a good option for those PCs where you have sufficient room in your PC case that you buy. A lot of premium or higher end Motherboards now have software based Intel RAID on them. This is good news for RAID arrays. Internal Onboard Storage is the fastest way to use the maximum performance of your set up due to it being native to your system. Software however is not without its downsides as it is fully dependant on the drivers and Operating system to facilitate the RAID setup and uses your CPU for some of the RAID work. Its the cheapest (you already own it) and relatively easy to set up in the BIOS. Performance is high however not as high as hardware RAID systems. Downsides are that hot swapping is typically not possible when a failure occurs. you have to shut down your PC in order to access the internal drives and swap out a new one. Because software RAID relies on the OS, the RAID system can sometimes not fail properly (although rare).
Internal Hardware RAID – Buying an Internal PCI Express Card RAID card is a dedicated RAID system that is truly independent from the Operating System and will run at the full performance it is rated for. This is the best option in terms of performance. Similar to internal software RAID, it does have some downsides. The main problem is upgrade capability, that is limited by your number of SATA ports and drive bays. If you want a small system, this isn’t the solution. The cost of True hardware RAID Internal SATA cards is not cheap so that is also a concern.
eSATA and Firewire – All competing products to USB 2.0 and USB 3.0 which are slowly disappearing. Firewire is more common in the MAC world then PC world however there are PC implementations. These standards are similar in function to USB so i won’t go into too much detail. Firewire 800 is similar but not as fast as USB 3.0. just plug and play. Daisy chain multiple devices together. Thunderbolt and USB 3.0/3.1 & iSCSI are the front-runners now.
Thunderbolt 2.0 (or higher, 3.0 is now here today) – Like USB Thunderbolt was designed by Intel and uses the PCI Express bus which was much faster than the Serial controllers. It was designed for higher bandwidth applications as well as creating single computer connections for video and data over one cable. Thunderbolt competes with USB now. Thunderbolt 2.0 was capable of 2 channels of 10 Gbps each link (effectively 20 Gbps). Thunderbolt 3.0 boosts to 40 Gbps at twice the rate of thunderbolt 2. The thunderbolt standard is also being set to royalty free by intel so that adoption of the interface will be more prevalent. Thunderbolt 4.0 rumored to hit 80 Gbps will drive 8K video to multiple displays and is the future of 8K standard interfaces. Thunderbolt today is slowly replacing Display port and HDMI (however HDMI on televisions will not go away for some time). Thunderbolt RAID and Drive arrays are still limited and they are expensive.
iSCSI –My personal setup uses this. A lot of people who haven’t worked in IT may not know what iSCSI really is. Quite simply, it is the SCSI protocol over TCP/IP protocol. This standard has been around for a long time. The benefits of this standard is that you are essentially using network technology to drive your storage needs. The downside is that high-capacity links are still relatively expensive. The links are highly error corrective because of the TCP standard. This is one of the most robust protocols around. Pull the plug accidentally for a few seconds and plug it back in and the TCP will re-transmit and correct itself (however some real-time connections would be lost, storage connections generally in theory can correct themselves). Set up is not easy however. Users have to know about Portals, Arrays and Volumes, targets and the Initiator, challenge passwords etc. All that matters is that you have a device that has the drives in it you want to configure as an array, you set that up as a portal and target, then connect to it from your new workstation with the initiator software. Once it finds the Volume you can mount that volume as a windows volume (pretty cool) and then format that like you would a hard drive on your local computer. The benefits are obvious, you have full windows permissions and have local access to volume that isn’t local. Downsides are cost for the interface if you want higher speeds. As of today 10 Gbps is expensive network card, but they are coming down in price and you can use 1 Gbps LAN card built into your new motherboard however for 4K or higher footage, we don’t recommend this. Another upside is that if you have a spare computer that has hardware good enough, you can turn that into an iSCSI RAID array for free! Yes i said Free
For more info check out FreeNAS!
www.freenas.org
Continue to Page 3
