Backblaze (25,000+ hard drives) which work best and last the longest?

[CanmoreOrLess]

As someone who had two hard drives fail last year (both less than a year old) I can see following this article for my next purchase. Normally I buy whatever is on sale, figuring they are all pretty much the same inside. It is difficult to test heat and noise at a store, sometimes I let aesthetics take priority. I know I am shopping in the worst possible way. My longest serving HD is from around 2004, a 750GB Lacie, it is used daily without issues. After an ugly HD failure (no back up) in early 2002, all my hard drives are backed up hourly or nightly.

http://blog.backblaze.com/2014/01/21/what-hard-drive-should-i-buy/

[Unknown303]

That's a good read with some decent data.

My longest running hard drive I have currently is probably from 2004 or 2005 currently. I've never actually had a hard drive fail on me in my computer. Although I did have my portable 3TB seagates shit the bed on me shortly after getting it for backup. So that was a waste of time.

[AE92_TreunoSC]

Thats great data, everyone talks about hard drives like cars. "I had one die in 2001 and I'll never buy one again"

I'm cheap and let my HD's spin down on inactivity so I've always bought Seagates, but this enforces their stereotype for poor quality haha.

Numbers talk bullshit walks!

[schocker]

I have not had a hard drive fail yet, so I count myself lucky. Some are older, like 2005, but I don't think I use them anymore. I always buy WD for regular HDD's now though

I think I have Seagate 120gb, 250 gb, Samsung 1tb, WD black 1tb, 2x 3tb red, Hitatchi 2tb. Samsung is now seagate for regular hdd and hitatchi is with WD.

I don't really back up but just end up having multiple copies of everything due to having many hard drives. Would be interesting to have a stat on SSD also though that would be super expensive for a place to run servers on. I have had an OCZ for a while which is unplugged now and a mushkin for my main C drive which has been great.

[msommers]

This reminds me, I need to setup a sync and start backing up my stuff regularily.

What's your guy's setup?

[revelations]

^ for the average home user, a Windows backup solutions + an external USB hdd will suffice.


Good article, pretty much goes with what I have discovered through hard knocks over the years. Hitachi got a needless bad rap years ago from one of their Deathstar (deskstar) model - which clearly has been dealt with.

[firebane]

This is a VERY biased report and I'll explain why. This is a "cloud" computing company doing backups for people and they are using desktop grade drives. Of those desktop grade drives they use Western Digital Green drives which have a HORRENDOUS failure rate due to the IDLE park they do.

The IDLE park essentially makes the drive park its head ever 5-7 seconds I think and the first generation drives allowed this to be modified but more recent versions have a different firmware that doesn't allow this so the wear and tear on the drives is ten fold.

Regarding the Seagate drives yes they have known to have a bad track record lately but put any desktop drive into a server environment and you'll compound the failure rate ten fold again.

To properly test this test they should have used Seagate ES and Western Digital Red drives.

So any company who is going to rely on desktop drives in a server environment to store clients data is just asking for trouble and a company I would steer clear of.

[Tik-Tok]

Originally posted by firebane

To properly test this test they should have used Seagate ES and Western Digital Red drives.


They have a bunch of WD Red...

Also, how is the report "biased"? They're reporting the failure rates of consumer grade products in a commercial environment... which is pretty much stress testing them all 24/7 lol. They aren't advertising, just reporting their own findings.

[firebane]

Originally posted by Tik-Tok


They have a bunch of WD Red...

Also, how is the report "biased"? They're reporting the failure rates of consumer grade products in a commercial environment... which is pretty much stress testing them all 24/7 lol. They aren't advertising, just reporting their own findings.

Because the stress and wear and tear of a consumer drive in a server environment is compounded so much that what has happened in a year could be what happens in 5, 10, 15, or 20 years of normal desktop usage. The normal desktop would never see the amount of use that it would in a server environment.

Servers run hot and have poor airflow for the most part and the drives are generally in a constant state of motion.

[CanmoreOrLess]

Originally posted by firebane
This is a VERY biased report and I'll explain why. This is a "cloud" computing company doing backups for people and they are using desktop grade drives. Of those desktop grade drives they use Western Digital Green drives which have a HORRENDOUS failure rate due to the IDLE park they do.


Not from what I read in the third to last paragraph:

"The drives we absolutely wonÕt buy are Western Digital 3TB Green drives and Seagate 2TB LP drives."

[nobb]

There isn't exactly any real hard proof that the WD Green, or any drive, having constant head parks shortens life. Note that I am talking about head parks and NOT spin downs.

I'm no IT person, but my experience with hard drives seem to be that for drives in 24/7 service...they seem to go for about 3-5 years before starting to develop bad sectors that start getting worse and worse. This has usually given me more than enough time to backup and replace the drive before they fail completely.

[syscal]

Originally posted by firebane
So any company who is going to rely on desktop drives in a server environment to store clients data is just asking for trouble and a company I would steer clear of.


Nearline SATA is cool and all, but nearly double the price. I use them in a couple of our SAN's and they fail way more than SCSI.

Commodity storage is okay for large environments that utilize file systems built for commodity hardware...in other words, if your storage can handle multiple drive failures as well as the loss of an entire storage server, you're covered and the cost savings are incredible. Think past RAID6 for this one...GlusterFS and Ceph are two good examples.

Here is how BackBlaze does it. http://www.backblaze.com/petabytes-on-a-budget-how-to-build-cheap-cloud-storage.html

[SmAcKpOo]

There are some big architectural holes in those blackblaze enclosures.

Here are a few.

-The system uses a single disk for hosting the operating system
-The system requires 2 power supplies to operate, both must be active and there is no redundancy, spare or failover unit
-The system has no hardware RAID capability
-The system only has 2 GigE network interfaces
-Any monitoring or health status reporting tools will have to be built, installed and configured by hand
-No SSD caching, this is standard in most enterprise storage solutions.

How about we not take advice from the IT guy who has never build a SAN network.

There are reasons why Nimble is gaining huge market share right now.




Think past RAID6 for this one...GlusterFS and Ceph are two good examples.

I LOL'd. Who uses RAID6 in production?

[syscal]

Originally posted by SmAcKpOo
There are some big architectural holes in those blackblaze enclosures.

Here are a few.

-The system uses a single disk for hosting the operating system
-The system requires 2 power supplies to operate, both must be active and there is no redundancy, spare or failover unit
-The system has no hardware RAID capability
-The system only has 2 GigE network interfaces
-Any monitoring or health status reporting tools will have to be built, installed and configured by hand
-No SSD caching, this is standard in most enterprise storage solutions.

How about we not take advice from the IT guy who has never build a SAN network.

There are reasons why Nimble is gaining huge market share right now.





I LOL'd. Who uses RAID6 in production?

Netapp...? I haven't used them but they are a big player http://www.netapp.com/us/products/platform-os/raid-dp.aspx - regardless, I was using RAID6 as an example of a system that allows multiple drive failures.

Also, if you read the article, you'd see the backblaze architecture doesn't require the reliability of an enterprise SAN. Any of your other points don't make a lot of sense if you read the article. Monitoring isn't an issue...?

Were you saying I have no SAN experience? funny guy.

edit :: to elaborate, the backblaze storage pods work for them because of the way they distribute their data. I was merely pointing out, in defense of some statements on this thread, how they can use cheap storage and still provide a reliable service.

[rlim891]

yes, as mentioned above. Different drives now are made to do different things. For nas - WD Red drives..this drives are meant to be on and working all the time. Red drives warranty are much better and different then Black drives.

I do recommend WD over Seagate. After selling both at an IT Distributor job...I find RMAs on WD are less then Seagate.

For a real reliable drive, i always recommend SSD. Use SSD as main O/S drive, and a second drive - WD Black as storage etc....

For san....I have build a few using SAS-2 drives. Seagate normally....and havent had any problems.

as for raid 6...havent had much experience on that, I mostly use 5EE...

ofcoz these are all just my personal view :)

[Xtrema]

RAID 6 or sometimes called double parity is default for Netapp. It allows failure of 2 drives within a disk group. Many time 1 disk will fail and the stress of rebuild will fail another. Having 2 parity disk will buy you that tiny bit of extra safety.

Now the problem is if you just happen to buy a shit load of Seagate Moose (7200.11, .12 and LP series) drives, and integrated into your storage system, no amount of parity will save you other than RAID10. There are many times where a 3rd disk will fail on rebuild. Many Netapp customer got burned by this and no firmware update helped.


Any RAID is better than none and always backup to a different media on top of your RAID setup. Same is true for home. Everything will break in time and on heavy use.

And don't trust SSDs. They are fast and on most consumer application, it may last the life of device. But they still have write wear issue and will wear out in 2+ years in commercial applications.

[syscal]

Originally posted by Xtrema
RAID 6 or sometimes called double parity is default for Netapp. It allows failure of 2 drives within a disk group. Many time 1 disk will fail and the stress of rebuild will fail another. Having 2 parity disk will buy you that tiny bit of extra safety.

I kinda liked HP's solution to this in their EVA series (this info might be a little old). They would spread the redundant drive(s) across all drives in the disk group, so if you chose two drive redundancy the extra space was carved out as a piece of every spindle so the rebuild would happen as soon as the drive cut out...something like a hot spare except the entire disk group performed the rebuild, preventing any single disk from getting hammered. Then the recreated data would simply copy over to the new drive when it was inserted.

[rlim891]

yes i agree most ssds for commerical aplications, the write is probably 2+ years....thats why when its storage...one should always go with a enterprise one. DC 3700 is one I have tried on a file server....been good so far..hope i dont jinx myself

also write is limited...but you can always read...data will be there and much safer on an SSD then HDD.

[UndrgroundRider]

Originally posted by syscal


I kinda liked HP's solution to this in their EVA series (this info might be a little old). They would spread the redundant drive(s) across all drives in the disk group, so if you chose two drive redundancy the extra space was carved out as a piece of every spindle so the rebuild would happen as soon as the drive cut out...something like a hot spare except the entire disk group performed the rebuild, preventing any single disk from getting hammered. Then the recreated data would simply copy over to the new drive when it was inserted.

The benefit of the HP EVA solution isn't on rebuild (which is actually harder on the drive), but on nominal use of the array. In a RAID 5/6 deployment where you have a hot spare already connected to the machine, often the entire raid array suffers normal wear while your spare just idles. This also leads to decreased performance since you could in theory be utilizing an extra stripe on that spare drive.

Instead the EVA solution takes the space of the spare drive and spreads it out across all of the drives. Then when a drive fails, it's rebuilt using the free space on all of the remaining drives. It's the same principle, same redundancy, just a different allocation of the space.

There are multiple problems with this approach (which is why it's not used in modern deployments). First, rebuilding an array using the same source and destination drive leads to disk thrashing (and this occurs on ALL drives in the WHOLE storage pool). Second, you still end up copying those stripes to a new drive when you replace the failed drive. So you've actually increased the duty cycle of the rebuild when the array is in a failed state.

With modern deployments of RAID 6 such a design is not necessary. As I'm sure you know, RAID 6 has 2 parity drives, which offers the same read performance as the HP EVA solution. If you had that extra hot spare drive you could utilize it as a parity drive instead of just idling.

The one big advantage to the hot spare is that it can be used as a replacement drive across multiple storage groups. The HP EVA solution also has this benefit, since the free space is split across the entire storage pool.

The bottom line is that in the circumstance where you have multiple RAID 5/6 arrays, and are going to utilize a hot spare, you would get marginally better performance with the HP EVA route, at the compromise of marginally riskier rebuilds. Although I would point out that any deployment that warrants the need for a hot spare would probably err to sacrifice the performance in favour of less risk.

As always, the exact design of the storage platform needs to be tailored to the application.

[syscal]

Originally posted by UndrgroundRider


The benefit of the HP EVA solution isn't on rebuild (which is actually harder on the drive), but on nominal use of the array. In a RAID 5/6 deployment where you have a hot spare already connected to the machine, often the entire raid array suffers normal wear while your spare just idles. This also leads to decreased performance since you could in theory be utilizing an extra stripe on that spare drive.

Instead the EVA solution takes the space of the spare drive and spreads it out across all of the drives. Then when a drive fails, it's rebuilt using the free space on all of the remaining drives. It's the same principle, same redundancy, just a different allocation of the space.

There are multiple problems with this approach (which is why it's not used in modern deployments). First, rebuilding an array using the same source and destination drive leads to disk thrashing (and this occurs on ALL drives in the WHOLE storage pool). Second, you still end up copying those stripes to a new drive when you replace the failed drive. So you've actually increased the duty cycle of the rebuild when the array is in a failed state.

With modern deployments of RAID 6 such a design is not necessary. As I'm sure you know, RAID 6 has 2 parity drives, which offers the same read performance as the HP EVA solution. If you had that extra hot spare drive you could utilize it as a parity drive instead of just idling.

The one big advantage to the hot spare is that it can be used as a replacement drive across multiple storage groups. The HP EVA solution also has this benefit, since the free space is split across the entire storage pool.

The bottom line is that in the circumstance where you have multiple RAID 5/6 arrays, and are going to utilize a hot spare, you would get marginally better performance with the HP EVA route, at the compromise of marginally riskier rebuilds. Although I would point out that any deployment that warrants the need for a hot spare would probably err to sacrifice the performance in favour of less risk.

As always, the exact design of the storage platform needs to be tailored to the application.

Good info, thanks.

[rage2]

Any of you SAN guys looking for work? We have an opening on my team, datacenter/enterprise IT tech with a focus on SAN expertise.

[thetransporter]

i had 5 seagate 7200 all fail .... made from 2007 on ward


however 20GB WD digital Hard drive i bought in 1999 is still running..

[thetransporter]

I agree, i dont trust anything with "cloud" in it.



Originally posted by firebane
This is a VERY biased report and I'll explain why. This is a "cloud" computing company doing backups for people and they are using desktop grade drives. Of those desktop grade drives they use Western Digital Green drives which have a HORRENDOUS failure rate due to the IDLE park they do.

The IDLE park essentially makes the drive park its head ever 5-7 seconds I think and the first generation drives allowed this to be modified but more recent versions have a different firmware that doesn't allow this so the wear and tear on the drives is ten fold.

Regarding the Seagate drives yes they have known to have a bad track record lately but put any desktop drive into a server environment and you'll compound the failure rate ten fold again.

To properly test this test they should have used Seagate ES and Western Digital Red drives.

So any company who is going to rely on desktop drives in a server environment to store clients data is just asking for trouble and a company I would steer clear of.

[J-D]

I've had a number of WD and Seagate consumer drives fail on me in the last 2-3 years in my NAS. It's just for storage (performance doesn't really matter within reason) so I just run two R6 arrays with a hot spare. I've considered switching over to ZFS but it's way too much work to migrate everything :rofl:

I do agree with some of the logic - if you have some degree of redundancy coupled with frequent backups... it might just make sense to cheap out and buy whatever is cheapest.

[A790]

For $5/mo they can back up my shit. Sold. lol

[msommers]

That online/cloud storage, Cam?

I wouldn't be opposed to that, but I have at least 2TB of music and photos on internal drives at the moment. Not only would the initial upload take next to forever, but subsequent photoshoots would take quite awhile to back up too.

Unsure what is the most idiot-proof, cheaper(est), reliable solution for personal storage and syncing.

[A790]

Originally posted by msommers
That online/cloud storage, Cam?

I wouldn't be opposed to that, but I have at least 2TB of music and photos on internal drives at the moment. Not only would the initial upload take next to forever, but subsequent photoshoots would take quite awhile to back up too.

Unsure what is the most idiot-proof, cheaper(est), reliable solution for personal storage and syncing.
I signed up with BackBlaze. We'll see how it goes. Ideally I'll never have to test to find out... lol