HDD Manufacturers Moving To 4096-Byte Sectors 442
Luminous Coward writes "As previously discussed on Slashdot, according to AnandTech and The Tech Report, hard disk drive manufacturers are now ready to bump the size of the disk sector from 512 to 4096 bytes, in order to minimize storage lost to ECC and sync. This may not be a smooth transition, because some OSes do not align partitions on 4K boundaries."
disable ECC? (Score:4, Interesting)
I heard some talks from the ZFS folks at Sun about how they were floating the idea to HD mfgr's of just disabling ECC on the drives. ZFS checksums every block, and in a RAID configuration, it would be able to transparently correct any checksum errors. I think this may have also been the motivation behind bringing triple-redundant RAID to ZFS.
The motivating idea was that this would reduce the overhead involved on ECC and gain extra space.
Thoughts?
Re:So only XP is out of luck? (Score:2, Interesting)
Couldn't agree more. Hopefully I don't have to rehash how horrible Vista was [pcworld.com], and Windows 7 came out a few months ago so it's a bit early to proclaim XP is dead when it's hopeful replacement just showed up.
I think 4096-byte sectors are Very Bad News. I have no experience with these drives but XP doesn't like them [anandtech.com] which is reason enough for me to avoid them. I hope hard drive manufactures come out with a standard naming scheme for these new drives so they're easy to identify online, like IDE, SATA, PATA, etc. Maybe AFD for Advanced Format Drive?
Why are there sectors? (Score:3, Interesting)
i'm asking because i don't know, not to troll.
What is their purpose? Does the purpose still matter?
Re:disable ECC? (Score:4, Interesting)
That wouldn't work with existing file systems that assume the drive does this. That's like deciding to remove the checksums from TCP and IP because a few protocols provide their own checksums. Might work in specialized cases. Probably just adds risk though for no benefit.
Re:Why are there sectors? (Score:4, Interesting)
Re:Why are there sectors? (Score:4, Interesting)
You are confusing physical sector size with cluster size. May file systems are already addressing data in larger blocks. 4096 is very commonly used. They are generally multiples of 512 which is the physical sector size; so that its is easy to calculate the physical sector that needs to be changed when you know the logical.
Its quite possible to have a cluster size smaller than the sector size; the file system would need to be smart enough to determine what other clusters fall on that sector and write them all though.
Re:disable ECC? (Score:4, Interesting)
I can see this working for drives made specifically for RAIDs. Lose ECC on single drive configurations and you're asking for trouble. At least for RAIDS, a controller would need to be aware of this and do the remapping themselves, in the end, I don't know if it's worth doing this at all. If some enterprising RAID controller company could prove it works better to do it this way, then I can see it happening.
Re:Why are there sectors? (Score:3, Interesting)
Ah, so it's saying, "Turn left on Evergreen... it's on that block". And the monstrous estate is from Elm to Fern at State. As opposed to GEOCOORD 32'57"(bunchOfDigits) by 32'57"(more digits).
Got it.
With the 1024 byte example, could the address just be "from bit X to bit X+1023"? i guess that too would be too much. All those tiny .dlls and .inis would take more space to define than they actually take.
Thanks!
Re:Care to provide examples? (Score:4, Interesting)
It took me longer than it should've to answer this riddle. Shortcut for the similarly caffeine deprived: andrewd18 means "P" as in Windows XP.
Seriously, I was like "Win...dows?" "U...nix?" "Micro...soft?" "OS...X"? "BS...D"?
Re:disable ECC? (Score:4, Interesting)
ZFS might checksum every block. But what happens when ZFS is not everywhere? Does the BIOS or whatever equivalent support ZFS checksumming for reading the boot sectors? So those sectors better be 100% or you better be turning it off for boot drives. You have to use ZFS everywhere and for everything. For example, if you ever try to image a 1TB disk without ECC, the odds of bit errors will be high. Even if ZFS can repair it - you'd only find out much later (too late?) and likely after another error prone write.
Such a feature would just be creating more opportunities for people to get things wrong.
And for what benefit?
> The motivating idea was that this would reduce the overhead involved on ECC and gain extra space.
I think the people who'd want ZFS or RAID would rather have better reliability than the 10% or so extra space.
Even if they don't know it at first
Re:So only XP is out of luck? (Score:3, Interesting)
The ECC (Error Check and Correct) is used for error correction. But generally speaking, the number of bits needed to check a block of data rises slower than the number of bits in the data - probably as the log of the number of bits, though I don't know. So grouping up sectors and providing a slightly longer ECC will save a significant number of the ECC bits. Of course a sector having eight times as many bits is eight times as likely to get corrupted, simply because of its size. But such faults are rare, though not rare enough to ignore. Of course, it will be a fraction less reliable. But the manufacturers do reliability/performance trade-offs all the time, and this is only one more of them. Presumably they reckon they have the reliability under control. If you want greater reliability, you need RAID anyway - to protect against drive failure as well as localized corruption. The probably reckon that anybody with really valuable data will be RAIDed anyway,
Actually no. (Score:5, Interesting)
Most of the drive manufactures are releasing tools to align the drives to 4k clusters so they can be used under XP. WDC already has theirs out here: WDC Adv Format [wdc.com] Plus instructions on all of their new 1TB and higher drives on how to set them up properly. You do have to jumper them, then format them specially but the drives work fine with 4k clusters. I put one in my work machine on Saturday, works flawlessly.
*I only used WDC because that's the brand I picked up recently. I do know other companies have similar tools and jumper settings on their newer drives as well.
Re-Format old drives? (Score:2, Interesting)
Re:So only XP is out of luck? (Score:2, Interesting)
So some enterprising young person will be the first to invent IPv6 for Windows XP just like Trumpet Winsock was cobbled onto Windows 3.1 to provide Internet access way back before Win95. If there is some compelling reason to join in IPv6 network then such capability will be built.
Re:Moot Point (Score:3, Interesting)
I do believe those are carry overs from Magnetic Media. There is no need for it to be that way (I think).
My point, was more or less, that we'll need to RETHINK how we define things. SSD will become more of an extension of the Operating Space we call "RAM". Much like we now have RAM, L2, L3, and L4 cache (and even maybe RAID Cache) are now.
I think, and this is just my opinion at this point, that we'll start to name memory by nearness to the Core(s), and SSD will join that space.
I'm not sure we need block level devices any longer. It will require re-thinking much of how we view things no doubt, but I think it is inevitable at this point.
Re:Windows XP end-of-life? (Score:4, Interesting)
Well, it's in Extended Support which for one thing means MS doesn't give a rats ass whether or not XP works with the more efficient AF HDDs, since that's not a security related patch.
Well, that's a fair assessment. Of course, that's a monopoly tactic — any business that dropped support for that widespread of a product in a legitimate competitive environment would find themselves with no customers for the newer product because customers would be trying to migrate out from under that vendor at all costs.
Re:So only XP is out of luck? (Score:4, Interesting)
Most used or not, it's 8 years old, and the update cost of a newly purchased machine with a plain OS installation disk includes roughly 2 Gig of downloaded data, and at least 5 reboots. (Measured last week on a clean installation of Windows XP Pro.) Even popular games that are shipping now do not run under it: that tells me it's obsolete.
ECC is NOT linearly proportional (Score:3, Interesting)
See the comment [slashdot.org] below. It takes 320 bytes for 8 512-byte ECCs, only 100 bytes for a single 4096-byte ECC.
Re:So only XP is out of luck? (Score:5, Interesting)
Sidestepping your ignorance or deliberate deception on periods of typical Linux support contracts
He didn't say if he was stating lengths from release or length of overlap (to me the latter is the more important figure)
Who cares if support goes out 10 years
It's 10 years (5 mainstream, 5 extended) minimum from release, 7 years (2 mainstream, five extended) minimum overlap between releases and 2 years (all extended) minimum overlap if you skip a release. IIRC XP will have exceeded all of those.
if you can't buy a new hard drive that will work with the OS?
These "advanced format" drives will work fine with XP, they just require a little extra effort (either using a third party paritioning tool, fitting an extra jumper to change the sector mapping or using the WD tool to realign the partitions after setup) if you want maximum performance. Besides I can still by PATA drives so I doubt these drives will be the only ones on the market any time soon.
Similarly if I go to almost any major vendor I can still get computers and computer parts that are supported with XP, some of the consumer crap isn't but virtually every buisness machine and seperately sold peice of hardware i've seen lists XP as supported.
It's articles and comments like this that give me difficulty discerning what exactly Microsoft "support" entails.
For most of us the most important part of the support is continuation of security updates (though they have occasionally refused to release one that they really should have released by claiming that it's not nessacery in a default environment), I would be very uncomfortable running exposed systems (and I coun't any machine used to browse the web as exposed) on an OS that was no longer getting security updates.
There is also problem support and non-security hotfixes (free if created while in mainstream support, pay for if created during extended support) but for most of us these are fairly irrelevant.
As I alluded to above though what really matters is support from third party vendors, I can still buy the latest hardware and run XP on it with no problems, just try doing that with a comparable aged linux distro (e.g. debian woody).
Re:Characters were not always 8-bit (Score:2, Interesting)
Re:Factors of 10 (Score:1, Interesting)
4096 byte sectors are fine. It's nice that we'll get roughly 10% more usable space for no cost.
But I think it'd be nice if when I open a 4KiB file it said 4KiB. According to metric prefixes accepted in virtually every other field, 4096 bytes is 4.1KB (or 4.096KB to be exact) Being "digital" does not give the right to use the wrong prefixes and cause confusion.
It's also worth noting that this is Microsoft's fault. Other OS's are doing it properly. Microsoft only does it properly when it benefits them. HDD manufacturers have faced numerous lawsuits simply because Microsoft is using the wrong prefix, so people feel cheated out of space. And the issue will only get worse... Every time we jump from one incorrect prefix to another.
1024 bytes - 1KiB - 2.4% discrepancy.
1048,576 bytes - 1MiB - 4.85% discrepancy
1073,741,824 bytes - GiB - 7.37% discrepancy
1099,511,627,776 bytes - 1TiB - 9.95% discrepancy
When you buy a 1500,000,000,000 byte HDD (which includes free error checking bits, so really has over 1650,000,000,000 bytes - 1.5TB usable), Windows reports the capacity as 1.36TB (incorrect) - not 1.36TiB. (correct)
Can't wait for the new round of lawsuits when we hit petabytes - 1125,899,906,842,624 bytes, a 12.6% discrepancy. And the ridiculousness of this is, it's not even a real issue.
Why aren't we suing SSD manufacturers? They often give us less bytes than advertised in both metric and incorrect metric.
Does nobody care about filesystem overhead? We lose as much as 20% of our disk capacity to shitty filesystems. That's way more than the Metric debate.
What about disk performance? It's harder to measure, but doesn't I/O performance matter more than capacity? Sun proved you can design a FS that doesn't lose significant performance or require defragmenting - ZFS. Microsoft with their NTFS is probably costing us 30% I/O performance. Wouldn't it be nice to have Reiser4 available for servers or computers with UPS's?
As long as we remain fixated on something that isn't an actual issue, we'll never correct the ones that are.