Large Network Storage

[impact]

Who can help me out with some details... I normally just get this stuff pre built at work and it happens...

As we all collect more and more digital media - I am seriously thinking about putting in place at home a dedicated file storage system - like a NAS or something...

So i Have found a few options to do it cheaply with some linux software or samba or others - and I dont know which we to go at present, but will work that out later....

I am thinking of Raid 5 - as not only will the storage contain TV programs (which in all honesty can be lost, but would prefer not to... ), DVD iso's - same as before, but archival stuff like video and photo's... Raid 5 would appear appropriate I suppose.

I know about the hot swap and things if a disk fails -

a) what I want to know is really things like the process - is it easy to recover, can the file server still be used while recovery is in operation...

b) If I start off with say 3 hdd (all same size) - how easy is to add the next one, and the next one, and next one... From a budget point of view - I have some hdd laying around or acquirable, but not enough to go straight out and buy 8 disks in one shot...

c) I have seen 8 channel raid controllers... suppose thats about the max - suppose I could always just add another array eh?

d) what about from a network point of view - again from a work perspective these things just run 7 * 24 - I dont want to really do that, prefer this array to go S3 or power down the drives if not in use... No big deal to wait 10 sec for the system to come alive when needed. As all this S3 stuff is relatively new to me - I notice on my current machines when referring about Lan wakeup - they talk about magic packets ??? What constitutes a magic packet - would just a request from my MCE box to access a file on this server include the magic packet or how can this be achieved...

Anything else you can advise on large external storage appreciated...

[arkay]

Impact,

Alot of the answer will largely depend on which way you go. From a unix (and this should all be available if you use linux:

a) Typically it's dead easy, remove the dead drive (software should tell you which one it is), replace the new drive and the drive contents will be rebuilt from the data on the other disks. You can even have one as a hot spare so the above takes place automatically, gives you a little more time to replace the faulty disk without worrying about the next one dying.

b) You can't add to a raid 5 array without rebuilding the entire thing(i.e. backup, rebuild, restore). If you start with a 3 disk array the best you could do is add another 3 disk array and use logical volume manager to put them together into a virtual disk that appears in linux as one large storage pool. You don't want to do this though as each array is going to use 1 drive as a parity drive. If you run 2 arrays you've lost 2 in parity drives. Better to go as big as possible up front.

c) See above.

d) Not sure on that one. I'd assume linux can do s3. It's the motherboard that does the wake from standby on reception of network traffic but again, what wakes it is another story.

Samba would also do the job nicely and using linux gives you all those niceties of unix, logical volume management, better filesystems etc etc.

Dunno how you'd go with windows and it's dodgy networking. Have the time my network drives don't remap when my MCE box comes out of hibernate.

You'd want to get a hold of a few drives and a machine and build a test unit before diving right in. You might also have to write some scripts etc to do some "post" wakeup stuff etc etc before you put in into produciton.

Cheers,
Arkay.

[monster]

all doable mate but it wont be cheap the raid card alone will cost u a fortune.

you would certainly want to use gigabit lan to reduce any bottle necks.

i would use a unix type system for that any of the BSD varieties will work well.

seting up samba is fairly easy.

anyway if you got that money to burn u better share with me

[impact]

Hahaha...

Found a decent raid card (8 Ch) for $340... Reckon I could get it a bit cheaper though... Already have plenty of motherboards / cpu / ram floating around...

I certainly was favoring towards Samba or the likes...

But need more investigation - cause it does not look like the perfect solution as yet... Want something that I can expand upon - like start with 300 Gb, easily add 300 Gb every now and again, but if raid 5 requires you to back up / reconfigure / restore each time - that just becomes to awkward...

Why Raid 5 - wanted something with some backup / protection as such - maybe I need to just use ordinary drives and find a decent backup solution and use that....

Amazing isnt it - we used to backup on floppys, then cds, now dvds are common - tape storage for the big stuff... but as soon as we start talking a terabyte - how the f*(& do you backup....

[Matt]

Impact,

Check the specs of the RAID cards because I've used ones that have allowed the addition of disks to the RAID pack which are then dynamically added into the array without the need for rebuilds. These were quite expensive cards at the time but this was 4+ years ago. AMI MegaRAID springs to mind, but I'm not 100% sure.

Cheers

Matt.

[Skorpion]

have a look at this mod

http://www.mashie.org/casemods/udat1.html

uses 5 disks, miniITX mobo..

[impact]

Originally Posted by Skorpion

have a look at this mod

http://www.mashie.org/casemods/udat1.html

uses 5 disks, miniITX mobo..

Hmmm... cute - nice I like the small footprint... Its got to be a good heater with all that compacted down !!! Bookmarked for reference later...

[Ronza]

Just pickup a server shell off eBay? I scored a Proliant 1600 for $200 w/ Dual P3-500mhz and 512MB of RAM with an inbuiult SCSI Controller Card that handles RAID5.
Guy threw in 3x 18GBs and 2x 4.3GBs in the deal along with an external storage U1 Storage array. All of this is hotswappable; I can take a SCSI drive out in two flicks, go to a mates house and plug it into his PC.

In your application; perhaps use SCSI Drives to record onto and then setup a script to back recordings that are >3 Months old onto normal IDE Drives. SCSI RAID5 is alot cheaper than IDE and SATA solutions

[impact]

Not a bad buy....

Those disk srives would fill up in a couple of hours of recording... hehehe...

Im thinking towards building 2 servers...
First one - just hanging hdd of it, combination of pata / sata - whatever I find at the time, no raid.. each just gets a new drive letter and be done with it... I'll use this for my dvd collection - where if a disk fails, I still have my original dvd's to replace it...
Second one - later - in the future - something like that lovely mashie described by scorpion... Raid 5 - to protect the digital assets that are harder to replace - photos etc..

This way - with the first server - I can just add to it easily, running low on disk space, pop a new drive in... etc.. Think I'll start off with 3 * 300Gb Sata's and see how I go...


Already have spare parts laying around - motherboards / cpu's / ram etc... Just got to find a nice case with plenty of drive space (coolmaster looks ok and relatively cheap), and find some nice (cheap) aluminium removable drive cases.... I do like the set in the mashie setup - auspcmarket have something similar - holds 4 drives in the space of 3 - but its worth $290.....

[gixasixa]

Cool thread - I was thinking of doing the same thing! Exactly the same thing! What about using those massive Lacie hdd?

[Ronza]

Originally Posted by gixasixa

Cool thread - I was thinking of doing the same thing! Exactly the same thing! What about using those massive Lacie hdd?

Waaaay overpriced. Buy your own 250GB 3.5" HDDs then a caddy or buy a couple and stick them in a P2 Machine with a PCI IDE Controller.

[impact]

Yea those Lacie devices look nice - a simple plug and play solution, but you should do a lot better / cheaper yourself particularly if you have some old hardware floating around...

Current price break appears to be 250 Gb drives going for around $180 - so 4 of those giving you around a tereabyte only costs $720 which aint too bad considering... and prices are just going to get cheaper...