I have somewhat of a storage dilemma at the moment, and I’m not quite sure which direction to take. Having gone through a pile of external drives over the years, I’m getting tired of individual drives and – just as importantly – wanting to have critical data mirrored wherever possible. So I’m looking for tips and suggestions. I’ve got some ideas, and I’ll run through them to give you some idea.
I’ve currently got, thanks to a recent 2TB drive failure, only about 9TB of space now on my Mac Pro, including both internal and external drives. It’s really my main storage point, and I want to consolidate as many external drives as possible into one unit.
Here’s some of the ideas I’ve had so far:
- Buy a Mac Pro RAID card and implement at least a 3 x 2TB RAID-5 unit, to give me 4TB of protected space. Problems with this option include: 3-drive RAID-5 is always inefficient, and I don’t want my boot drive included in the RAID set.; reviews on the Apple Store website universally praise the RAID card for SAS drives, but seem to be fairly divided between love and loathing on the use of it with SATA drives. That doesn’t give me a lot of confidence. Chance of me doing this – 10% or less, I’d imagine. The card alone would cost well over a grand, not to mention the drive costs.
- Buy a Drobo 4-drive unit and connect via FW800. Much cheaper option, but then limited permanently to about 70MB/s for anything I whack on that drive. It seems you can still get the 4-bay Drobo for around $600 AU, then you pay for the drives on top, of course. Thankfully with it, I can start with smaller drives and bung in bigger drives over time – almost like splitting it into CapEx and OpEx. (The RAID card is more just outright CapEx).
- Buy a Drobo 5-drive unit and connect it with eSATA. Definitely much faster, but then I’d need to also buy an eSATA card for the Mac Pro. I’ve always intended to do that, but with the 5-drive Drobo over $1000 Australian, the costs will stack up in short order. (I’m imagining a “decent” lower-end eSATA card will cost me in the order for $200 AU.) There’s the drives on top of that too. Advantages are speed, obviously.
- Buy a 4-drive NAS. I was almost starting to lean towards this, but then I saw a forum talking about say, the WD Sharestorage unit, and how it was only licensed for 3 users. Apparently the StorageTek BlackArmor is licensed for 20, but my first comment on this is: WTF? Why should I have to pay on a home storage unit to connect additional users or computers to it? That definitely played a significant factor in making me less interested in NAS. Also the prices are still fairly high – BlackArmor in AU for just 2TB to start with is still over $1200; the Western Digital is around the $900 for a 4-drive model, or if you go for the 4-drive model with only 2 bays occupied, it’s less, but you don’t get the extra trays for the additional drives. Again, WTF?
- Buy an 8-drive Drobo. OK, so this is getting to the high end of what I want to spend (and then jumping over it by quite a bit), but it’s about the highest amount I can consider. These seem to be around $2300 for the bare bones unit in Australia, which means it would cost the most – but it would also be the most flexible, offering iSCSI options as well.
- Give up in disgust and keep on using 1, 1.5 and 2TB external drives, a mix of USB and FW800. I really don’t want to do this, but all the different options are quite frankly driving me nuts. I’d also like a solution that isn’t “all or nothing” on the first purchase, since that makes it a lot more challenging that just adding to over time. The biggest reason I don’t want to do this is that I’d like to get to a point where most of my personal storage is protected from individual drive failure.
I should add – I’m a data hoarder. I have email going back nigh on 10 years, I have files from my very first PC still floating around, and I only delete when I’m absolutely, positively certain that I’m never going to need the file(s) again. And if later I do, it slows down my deletion decision making process even further. I prefer to keep my data online and searchable instead of offline and inaccessible, so some archiving solutions aren’t an option I’ll consider.
So what are your thoughts? What have you used, what would you not touch with a 10 foot barge pole, and what would you swear by?
No related posts.
Hi Preston,
I’m running a Synology CS407.
It runs a customised 2.6.x kernel and is almost infinitely extensible, it’s only limited by ones capability and the CPU performance. No stupid licensing. Nice AJAXified management interface. I’ve only got 4x512GB in a RAID5, but there are also larger and more capable units available, eg http://www.synology.com/enu/products/DS1010+/index.php
My unit is a bit dated nowadays, but current models have up to 5 hot swap bays. The CS407 runs fast enough over fast ethernet. I’m currently bound by RAID 5 performance, if I wanted more steam, I’d have to move to RAID10. But as a home user I’m willing to trade-off performance for cost.
It’s a winner for me for two reasons:
1. It’s a NAS. The data is available from all my machines. Via SMB, NFS, direct via a GUI, over http, etc.
2. There are regular updates to the firmware with bug fixes and software updates. Somewhere, I think about 12 months ago, they added the ability to migrate between different RAID types, eg change 4x512GB RAID1 into, for example, a 4x2TB RAID5.
The base unit was a bit pricey to start, and adding the disks on top made for a relatively high startup price. But I’ve got the piece of mind that tells me that I can expand it easily. And that there is good support from the vendor.
Did I mention that (some months ago) I had a dodgy SATA connection that triggered a disk labelling bug and when I reported it they cut me a custom release to fix my problem? In 2 weeks?
Cheers,
Francis
Thanks, I’ll have a look at it.
Ah – the specs show though that it runs ext3 as the filesystem? I’d never use ext3 for anything over a couple of hundred GB any more after switching to XFS for my home server…
Dude,
The time on your site is wrong. When I pressed “submit” it said the time was 10:39pm, my wall clock tells me it’s actually 11:39 pm.\
ta.
Actually it’s more that WordPress still currently requires a manual switchover of your time/date, so I just permanently leave WordPress configured for GMT+10; otherwise every time I change the time offset, stats change.
It’s a NAS appliance. Just use it.
But why xfs? Performance? The folks using a custom kernel on a buffalo nas suggest that the reliability/tools for xfs aren’t quite there. http://buffalo.nas-central.org/wiki/Using_XFS_instead_of_ext3_(network_performance_boost)
If preston’s using XFS maybe I should too. Convince me?
I can only explain with the most practical examples.
Time taken to fsck a 1TB ext3 filesystem after sudden power loss: half an hour to 2 hours.
Time taken to fsck a 1TB xfs filesystem after sudden power loss: under 30 seconds.
Time taken to delete thousands of files from an ext3 filesystem: Terribly, terribly slow.
Time taken to delete tens of thousands of files from an xfs filesystem: Practically instantaneous.
Time taken to delete a single 1.8TB file from a 3TB ext3 filesystem: Somewhere between half an hour and an hour, from memory.
Time taken to delete a single 1.8TB file from a 3TB xfs filesystem: Gone before enter key is fully back up after typing “rm ”
That to me are my compelling reasons.
ext3 is a really unpleasant filesystem in terms of meta-operations. See here for my eye opening experiences.