Long Block Data Standard Finalized

An anonymous reader writes "IDEMA has finally released the LBD (Long Block Data) standard. This standard, in work since 2000, increases the length of the data blocks of each sector from 512 bytes to 4,096 bytes. This is an update that has been requested for some time by the hard-drive industry and the development of new drives will start immediately. The new standard offers many advantages — improved reliability and higher transfer rates are the two most obvious. While some manufacturers say the reliability may increase as much as tenfold, the degree of performance improvement to be expected is a bit more elusive. Overall improvements include shorter time to format and more efficient data transfers due to smaller overhead per block during read and write operations."

Higher Reliability?

[MankyD]

How does larger block sizes result in better reliability? Intuitively, I would almost think the opposite, since a single byte corruption means a much larger block is now erroneous. I obviously am missing something though.

Re:Higher Reliability?

[silas_moeckel]

I would think it has to do with the ability to have more bits for ecc type functions. Blocks would need to be terminated somehow so there is a fixed overhead per block. Reducing this overhead by a factor of 8 would leave more room for a larger parity type field and the more bits in there the larger failure that it can detect, fix and relocate. This would all assume they will not use the new space to push up sizing. Course this is all my rather speculative guesswork.

Re:

[The_Wilschon]

So, supposing your speculation is correct, then the longer block size doesn't actually enable better reliability, it just lowers the capacity cost for a given "amount" of reliability. They could build in as much reliability as they cared to with the current block size, but then they would have to decrease the capacity by more than they would have to with a larger block size.

Re:

[msauve]

I don't have access to the actual standard, but would guess that they're really claiming more reliability for the same storage capacity, not more reliable in absolute terms.

They can take what would have been per-block overhead with smaller sector sizes and reuse that data space for more robust error correcting codes, while maintaining the same capacity.

But, good question, since in terms of absolute reliability I can't picture anything in the current spec which would prevent private (not visible at the i

Re:

[GooberToo]

I don't have access to the actual standard, but would guess that they're really claiming more reliability for the same storage capacity, not more reliable in absolute terms.

In the real world this translates into, "more reliability". Reliability has always been relative to dollars spent. This means given the same dollars you are more reliabile. This means, given absolute dollars, you are more reliable.

Error correction better over larger blocks

[EmbeddedJanitor]

If you're working with a certain number of ecc bits per data bit, then the number of corrections you can perform increases with an increased data block size. Oversimplifying, just for explanation here:

Let's suppose you can fix one error per 512 byte block or 6 errors per 4096 byte block. Intuitively that might seem like a step back because 6/8 is smaller than 1, but that is not so. If you have 512-byte blocks and get two errors in a 512-byte sequence then that block is corrupt. However if instead you're using 4096 byte blocks then a 512-byte sequence within that block can have two errors since we can tolerate up to 6 errors in the whole block.

Or put another way, consider a 4 k sequence of data, represented by a sequence of digits dependent on the number of errors in each 512 bytes. 00000000 means no errors, 03010000 means 3 errors in the second block and 1 in the fourth block (ie a total of 4 errors in the whole 4096 bytes). With a scheme that can fix only one error per 512 bytes, the block with 3 errors cannot be corrected (because 3 > 1), but in the system which fixes up to 6 errors per 4096, the errors can be fixed because 4 6. This means that the ECC is far more reliable.

Re:

[Jartan]

Let's suppose you can fix one error per 512 byte block or 6 errors per 4096 byte block. Intuitively that might seem like a step back because 6/8 is smaller than 1, but that is not so. If you have 512-byte blocks and get two errors in a 512-byte sequence then that block is corrupt. However if instead you're using 4096 byte blocks then a 512-byte sequence within that block can have two errors since we can tolerate up to 6 errors in the whole block.

Parity data is parity data. It doesn't matter where it's stor

Re:

[hamanu]

OK, yes you COULD move the parity dta around but you'd get shitty performance. Hard drives are made so that each sector is independent of another. That makes each sector a seperate codeword on disk. What you are proposing is to introduce dependency between sectors, and that would mean having to read adjacent sectors in order to write a single sector, which means goin through 2 revolution of the disk instead of one.

Re:

[Ben Hutchings]

No, every block has to be independently writable. That's why they can't just use 4 Kbyte sectors on the disk and present them as 8 blocks through the ATA interface.

No, the logic is not flawed.

[EmbeddedJanitor]

Consider it this way

Let's say you have 4096 bytes arranged as 8x512-byte blocks and each block can correct one error. Now lets say that we RANDOMLY (ie statisticly independently) introduce, say, 4 errors into that set of 8 blocks. Sometimes the errors will fall so that there are at most one error per block. That is correctable. Sometimes the errors will fall so that there are more than one per block. In that case data will be lost.

However, if we can correct up to, say, 6 arbitrarily placed errors per 4096

Re:

[Hell O'World]

the reliability may increase as much as tenfold
I think ten-fold may be an exaggeration, but I could believe an eight fold increase.

Re:Higher Reliability?

[5pp000]

The longer block sizes add reliability because the error correcting codes have more to work with at a time (more data bits, but also more ECC bits).

As for wasted space, that's under the filesystem's control, not the drive's.

Re:

[cerberusss]

As for wasted space, that's under the filesystem's control, not the drive's.

I use a raw device [wikipedia.org], you insensitive clod!

Re:

[imsabbel]

Sorry, but your operation systems FS works in 4k blocks anyways, so you dont actually save anything compared to 4k phys-layer blocks now.

OTOH, maybe checksumming could improve from the larger blocksize.

Re:

[drinkypoo]

CDs for example are highly resistant to bit errors and their block size for data is 2048 bytes.

At least half the reason that CDs are resistant to error is that the laser is not in focus when it passes through their surface.

CD error recovery unrelated to block size

[_Shorty-dammit]

Block size has absolutely nothing to do with how much redundancy you can build in, and I fail to see the logic in assuming so. Makes absolutely no sense. The 2048 bytes stored on a sector of a CD only refers to your data, and absolutely none of them have anything to do with the CD's error-correction mechanisms. They add lots of extra bits to make up their error-correction, over and above your 2048 bytes of data. But, the point is it doesn't matter how much space you reserve to hold user data, you can ar

Re:CD error recovery unrelated to block size

[ElecCham]

I can speak with some authority on this - I work for one of those aforementioned hard-drive manufacturers, and have been doing a small amount of work on this exact thing.

The easy answer is this: in order to do ECC-like data checking on a larger set of data (say, a group of eight 512-byte sectors), it means that if you want to write sector three of that eight, you end up having to re-read the whole thing before you do anything else - thus basically giving you 4,096-byte "sector" anyway.

The other half of that answer is this: do you know what the "real" storage capacity of a CD is, without all the error checking? It's a bit less than double. Even most of the enterprise folks wouldn't accept a 40% hit in data density in return for what works out to not that big an increase in reliability (data redundancy doesn't buy you that much unless that data is on different spindles). They'd just rather get the whole data space and do a RAID, especially since that's what they're going to do anyway.

Re:

[_Shorty-dammit]

Was this supposed to convey some bigger idea you had? Anyway, you need to do some more reading about just how error recovery works, and how user-data-block-size is irrelevant to how much redundancy you can/do have. You're wrong, and you don't know what you think you know. Go read about the actual bit layout of a CD, since that was your example.

Re:

[hamanu]

the rate of a code measure how much redundundacy it has, correct. But why do you think block length doesn't matter? Just because you have high redundancy doesn't mean your errors are going to magically be recoverable. To actually recover the data you need enough distance between valid codewords so that when a codeword is perturbed by errors you can still see which valid codeword it is closest to. With short block lengths you get small decoding distances, and low error correcting power. If you learn informat

Re:

[hamanu]

I guess I should pre-emptively point out that for a hard drive you want to be able to modify each sector atomically, which means that a single sector corresponds to a single codeword, and increasing areal density means you need longer codewords to maintain error correction. So either you decrease the rate of the code, and use extra redundancy, which lower capacity and defeats the purpose of increasing areal density, or you us longer codewords at the same rate, which means using longer sectors.

Re:

[SailorFrag]

Perhaps they've discovered some new codes for larger block sizes that are closer to being perfect codes than the current ones they use. If the codeword size is 6x what it was before, then maybe the % of possible read vectors that get the "too many errors; don't know how to correct this" case will be smaller. Also, I'm not sure where you're going with your CD example: the coding they used in CDs is really obsolete by today's standards anyway.

Why 4096?

[MBCook]

Is there a good reason why 4096 was chosen? Is that just an artifact of this being designed in 2000? At this point very few files on the average system would be smaller than this. It seems to me they could have quite safely chosen something like 16k which would have improved things more, future proofed them more, yet still have been small enough as to not waste a tremendous amount of space (like if they chose 512k).

Why not make it variable, in that each drive can have it's own value (limited to a power of 2, between 512 and say 512k)? That way one drives today could be 4k, with drives in a few years being more without requiring another 7 years for a new standard?

Re:Why 4096?

[42forty-two42]

Operating systems tend to use 4096-byte blocks already, as that's the size of a memory page on x86 and amd64. If you were to require 16kb transfers, then the block cache would have to start allocating contiguous four-page groups for DMA transfers and the like, which could be difficult if memory is fragmented; in comparison, pages are the basic allocation unit for RAM, so 4kb's easy to find.

Re:Why 4096?

[AKAImBatman]

Parent is correct. Pretty much every paging-capable microprocessor in existence uses 4K memory blocks, thus why they're the natural size for a hard disk. In the x86 world, the next step up is 4MB blocks. Burst performance of modern hard disks is quite good, but I have to wonder if 4MB blocks would be helpful or harmful to overall system performance? It might reduce the number of pages, that's for sure.

Re:Why 4096?

[42forty-two42]

Using 4MB blocks for everything would kill memory performance - and more specifically, mmap performance. Each library loaded in your system would require at least 4MB of ram - probably more, as they have code, data, and zeroed data segments. Additionally, each process would require another 4MB*n. There's no gain for doing this either, except under specialized circumstances, as the OS can already request a batch of sectors from the drive in one operation.

Re:

[Nutria]

Each library loaded in your system would require at least 4MB of ram - probably more, as they have code, data, and zeroed data segments. Additionally, each process would require another 4MB*n.

One word: Vista. :(

Re:

[pchan-]

Pretty much every paging-capable microprocessor in existence uses 4K memory blocks, thus why they're the natural size for a hard disk.

AMD64/x86-64 uses 8KB pages. ARM uses 1KB pages.

Re:Why 4096?

[Scott Wood]

No, x64 and ARM both use 4K pages (though ARM has 1K subpages that you can set permissions on individually). Alpha and sparc64 use 8K pages, though.

So it would be better to say...

[Ayanami Rei]

4k is the common base of the most widely used operating system page sizes. :-)
(1k being more popular for embedded systems that don't have HDs anyway)

Re:

[Afecks]

It might reduce the number of pages, that's for sure

You may be sure that it might but I'm unsure that it won't...

Re:Why 4096?

[geekoid]

yeah, sure. Give a logical AND knowledgable answer.
Way to ruin the curve.

Re:

[major.morgan]

From the press-release:

Large blocks also provide a clear path for future gains through further increases in block size.

Seems to imply that the standard does perhaps address variable block sizes.

Re:

[macshit]

Is there a good reason why 4096 was chosen? Is that just an artifact of this being designed in 2000? At this point very few files on the average system would be smaller than this.

Do you actually have any data to back up that assertion? On my system there are tons of small (< 1 KB even) files around; of course lots of large files too, but it's certainly not obvious that your claim is true.

Morever, even if say the average file was, say, 16KB, using smaller blocks helps reduce wasted space in file tail blo

Oh noes!

[RingDev]

All of my 400b files are now going to take up 10 times as much space!!!

Heh, glad to see this is finally going through!

-Rick

Re:

[drinkypoo]

Actually, they're going to take up eight times as much space... YOU FAIL IT! They will waste 3636b space unused in blocks, however, instead of only 112 bytes, so they'll be wasting over 32 times as much space. But then, won't ReiserFS already store multiple files in a single block in some cases?

Re:

[StikyPad]

won't ReiserFS already store multiple files in a single block in some cases?

Yeah, but when you ask it, it will deny any knowledge of the whereabouts of the file in question.

Re:

[wexsessa]

With some probably minor inconvenience, you could fix that by using a Zipped archive. And someone will likely come up with a low-impact solution based on that.

Thats a lot a bits

[rambag]

Yeah why 4092 bytes? Why not 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 bytes? It seems to me to be the best option

Mod it Funny/Redundant!

[RingDev]

I would love to see that post as +5 Redundant. We all know 09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0 is going to be waaaaaay over abused for ever now, but at least that post was a good use of it.

-Rick

Re:

[nuzak]

In soviet slashdot, D84156C5635688C0 09F911029D74E35B's you!

Not all good....

[RobertM1968]

It also means more wasted space on a Windows machine where the user wants a block size of say... 512bytes, or OS/2 and eComStation's HPFS that only uses 512bytes to prevent space waste. It doesnt seem like much, but it does add up if you have a lot of files (pr0n, music, data, images, etc).

Re:Not all good.... You said it!

[freeze128]

The larger block size is bad if you have a lot of little files, but better if you have fewer large files.
This will just encourage Microsoft to stop using INI and XML files and to store more settings the in the big REGISTRY...

Re:

[RobertM1968]

Hopefully it will... but as I put my entire CD collection onto my machine, that will only alleviate MS's part in the wasted space issue... :-(

Much of the other issues a larger sector size alleviate would also be addressed if MS would revise NTFS so it wouldnt fragment. They know how, as they have access to the HPFS internals (HPFS rarely exceeds 1 or 2% fragmentation). That is something else I dont understand... they have the answers to many complaints about Windows (that being only one of them) and do not

Well you're already wasting your disk space....

[EmbeddedJanitor]

...if you have Windows loaded.

Re:

[RobertM1968]

:-)

Only on my gaming machine... my servers are all eComStation, and one or two will become some variant of Linux soon. Problem I will run into is that HPFS (eComStation) has a fixed block size of 512 bytes.

Re:

[Kjella]

it does add up if you have a lot of files (pr0n, music, data, images, etc).

No, it doesn't. it'll be something like 0.1% of a 3.5MB MP3, which is probably saved by having smaller file allocation tables anyway. The only way it could possibly matter is if you have an extreme amount of small files, like say hundreds of megs of source code and even then it's a pittance on a HDD from the last decade.

Re:

[RobertM1968]

Hmmmm, obviously you never stopped to do the math. Lets say I have 900,000 files (I have partitions with more actually). That is an average of 1,843,200,000 bytes wasted using a 4K block size. Compare that to a 512 byte block size with 230,400,000 bytes wasted.

Please explain to me how:
1.8GB
is not much different than
230MB

Or are you claiming that the file allocation table on a 512byte sector drive would be over a gig and a half? Mine is using a whopping 80MB for file tables and extended attributes and r

Re:

[Blakey Rat]

NTFS is actually designed to put small files (smaller than the block size, IIRC) into the directory structure itself instead of wasting a bunch of space required to put them in individual files. So it already takes care of this issue with current drives, and will continue to do so with future drives.

But, you know, good troll and all.

Re:

[RobertM1968]

Yes, but you are forgetting three factors...
1 - Any file that doesnt end on a block size boundary (ie: multiples of 4096) will waste space (ie: a 6144 byte file - which is larger than the block size - wastes 2048 bytes).
2 - What files nowadays are smaller than 4K? (keep in mind the quantity of files that fit that criteria and read #3)
3 - The quantity of files is what exasperates the situation. (Quantity x free space at end of last block).

NTFS does nothing to alleviate this.

So, I guess my post wasnt a t

Re:

[RobertM1968]

Well, Vista I only use at work (because I have to), and OS/2 is still in active development under the name eComStation, and still has a better threading model than Linux (which from what I understand is being addressed by the Linux community, so that may change). Also, I prefer the WorkPlace Shell to any other GUI I have used (though OSX comes close) - I like being able to extend the GUI in an infinite number of ways with very small and simple coding, and I like the fact that it is object oriented unlike th

Re:

[RobertM1968]

Um, go read my post again where (1) I indicate my knowledge of the default block size and the ability to change it, and (2) indicate that on my eComStation servers, the default (and only) block size is 512 bytes.

Sorry if my post confused you...

What about the MBR?

[QuantumG]

Trying to fit an entire virus into 512 bytes was always a challenge.. but 4096 bytes? That's too easy!

Re:What about the MBR?

[Godji]

The parent raises a point though: now that we have bigger sectors, are we finally getting a standard for partition tables with more than 4 entries without using logical partitions?

Re:What about the MBR?

[ottffssent]

The word you're looking for is GPT [wikipedia.org]. It has nothing to do with 4k hardware sectors, but it does support up to 128 partitions. Which ought to be enough for anybody (says the man with a 1 average number of partitions per disk in his household).

Re:

[shawnce]

Extensible Firmware Interface (EFI) [wikipedia.org] supports GUID Partition Table (GPT) [wikipedia.org]

Plan for the future!

[operagost]

These kinds of incremental standards are simply not forward-looking! I propose that the data block size be set to a minimum of 2^32 bytes.

Bootloader now 4096 bytes?

[Kjella]

Did the space for the bootloader just increase to 4096 as well? For those unaware, the BIOS loads just the first sector of the disk into memory, the bootloader takes it from there. It would certainly let them get a lot more resilient, now they only barf if things are not as expected.

Longer != Better

[snoyberg]

I have to disagree with the whole premise here. I know that people always say that longer is better when it comes to hard drives, but I've never had any reliability problems with my smaller one. Not only that, but I've had very fast transfer rates under all sorts of strenuous loads.

Wait, we're talking about storage devices? Never mind...

Already Obsolete

[Detritus]

See the work on ANSI T10 Data Integrity Field, that provides end-to-end error detection. It bumps the standard block size from 512 to 520 bytes.

Re:

[Wesley Felter]

520-byte sectors were a nice idea back in the day when all RAID arrays used SCSI/FC disks. But these days you have to support SATA and its 512-byte sectors, so you just find somewhere else to store the checksums.

blocks and clusters

[ceroklis]

To all the posters complaining about the loss of space when they will be forced to use 4096 instead of 512 bytes to store their 20 bytes file:

• The cluster size (unit of disk space allocation for files) need not be equal to the physical block size. It can be a multiple or even a fraction of the physical block size. It is fairly probable that you are already using 4K clusters (or bigger), so this will not change anything. This is for example the case if you have an NTFS filesystem bigger than 2GB.
• Not all filesystems waste space in this manner. Reiserfs or EXT3 can pack several small files in a "cluster" .

Re:

[ceroklis]

s/EXT3/EXT4/

Thank you, Captain Obvious!

[billcopc]

It's about effing time!

512 bytes was good for floppy disks. I think we should have started upping the sector size around the same time as we hit the 528mb 1024-cylinder limit back in the early 90's. Considering that a modern hard drive has anywhere from one-half to two billion sectors, and that's some serious overhead for no reason. Error-correction is "easier" if it's spread over larger blocks. Why ? Because most files are quite large, and corrupting a 512 byte chunk is just as bad as corrupting a 409

Slashdot Article in 2010

[/dev/trash]

Debian Finally Supports Long Block Data

Re:

[WarwickRyan]

> NOTHING is 512 bytes anymore.

Shortcuts can easily be 512 bytes long.

For example I've got a shortcut to a file on C:\, which is 391 bytes actual size, 4096 bytes on the disk.

Re:

[Animaether]

In addition, it doesn't matter whether the file is less than 512 or, in this case, 4096 bytes. What matters is if the 'size % block_size' is non-zero. I.e. let's say the file is 4090 bytes. It will fit just fine, and you'll only waste 6 bytes. Now the file is 4100 bytes, only 4 bytes over. Except now you need 2 blocks, and thus waste 4092 bytes.

Sure, on a multi-GB file that's not going to matter too much, as even on a TB drive you can only have a few hundred of those, and who's going to miss that 1MB?

If you need to put millions of tiny files on disk

[grahamsz]

Then you should really be considering a database.

Re:If you need to put millions of tiny files on di

[angst_ridden_hipster]

Evidently, you don't think EFS3 is the right database?

I mean, it's not relational, but it's pretty damn powerful.

Re:Sounds like a good idea to me.

[Dan Ost]

If that kind of lossage bothers you, use a file system that can pack multiple file tails into the same block (reiserfs for sure, ext4 will too, I think). If you've got lots of small files, the impact can be surprising (my portage tree shrunk by about 100MB just by moving it from ext3 to reiserfs!). I've never noticed a difference anywhere else, however.

Re:

[pair-a-noyd]

I use EXT3 currently to be specific. I'm looking forward to trying ZFS. I don't need to boot from a ZFS disk, I can boot fine from an EXT3 disk and run everything else on ZFS.. I'm hoping to give it a trial run later this year.

But as to the hardware itself, I don't know why they don't make the size user selectable by jumper or software. "One size fits all" is no longer relevant. Many people have many different needs, uses and applications. Some people have lots of little files. Some people have lots o

Re:

[PlusFiveTroll]

What's so hard about that?

Go read the Linux Kernel mailing list, and you'll find interactions between the block layer and the virtual memory are one of the most difficult things to make work right in an operating system. The size of the block on the hard disk matters most to the driver, its mostly transparent to the rest of the operating system. The only thing it changes on actual file systems is the minimum filesystem block should be 4K minimum.

For USB hard drives?

[tepples]

use a file system that can pack multiple file tails into the same block

Which mainstream operating system can read such a file system? Or should I just abandon tail packing for use on a USB hard drive that will be used with multiple operating systems, at least one of which was made by Microsoft?

Re:

[mi]

For example I've got a shortcut to a file on C:\, which is 391 bytes actual size, 4096 bytes on the disk.

Hopefully, the filesystems, which can pack multiple short files into a block, will become more popular... ReiserFS [namesys.com] does this, supposedly — I wonder, if anything else does...

Re:

[mj01nir]

NetWare has had block suballocation for a long time: http://www.novell.com/documentation/nw51/index.htm l?page=/documentation/nw51/trad_enu/data/hotiiczz. html [novell.com]

Re:

[mj01nir]

Sooo, when they became EVIL they dropped block suballocation? I wasn't making a statement about Novell's business practices, only making a minor point relevant to the discussion at hand. You should try it. You may even use shift-4 less.

Re:

[644bd346996]

I think NTFS does something like that.

Re:

[cnettel]

Even NTFS does it, to some degree (with the side effect that some 3rd party/open source implementations have special problems when it comes to accessing or writing such files). As a sibling post mentions, it's not only a matter of small files, though, it's also a matter of large files that are not a multiple of a suitable power of 2 in size.

Re:

[sconeu]

Didn't the Berkeley FFS do that back in the '80s?

Re:

[AJWM]

Didn't the Berkeley FFS do that back in the '80s?

Yes it did, as of (if I recall correctly) BSD 4.2 for VAX -- at the same time it made the jump to a 4K block size.

Re:

[pair-a-noyd]

I lost too much data to Reiser problems, I used it for about two years and I decided it sucks.
It's EXT3 for me, at least right now. Like I said earlier, I hope to convert to ZFS sometime this year.

Re:

[EvanED]

Ask Wikipedia [wikipedia.org]

It's in the table "Allocation and layout policies". Look at both tail packing and block suballocation.

There are a few others that do, but not many. (JFS, QFS, NWFS, and VMFS are marked yes; NTFS and ZFS are marked partial.)

Re:

[Kadin2048]

For example I've got a shortcut to a file on C:\, which is 391 bytes actual size, 4096 bytes on the disk.

I think that sorta proves his point, though. With today's filesystems and other equipment, even a 391 byte file ends up taking over 4k bytes on disk, and nobody really seems to care. At least when you're talking about storage, virtually nothing is that small in terms of actual space-on-disk on a modern system.

Re:

[Nutria]

At least when you're talking about storage, virtually nothing is that small in terms of actual space-on-disk on a modern system.

For standard usage, you are correct.

Maildir-based IMAP servers are all that I can think of which would really benefit from tail packing, although there will also obviously be some custom apps that need or make use of millions of small files.

Re:

[The MAZZTer]

It's not just 512 bytes, it's MULTIPLES of it. For example, on a usb flash drive, low cluster size is important to avoid wasting space and fitting as much data as possible.

Which brings up the question, what's the difference between sector size and allocation cluster size? I assume the former is hardware while the latter is software, but anything else? Does the sector size limit the minimum allocation cluster size?

As an example of the reason why you might want to keep cluster/sector size low, this one

Re:

[Tridus]

The Filesystem can't allocate a unit smaller then a Hard Drive sector without combining multiple files into a single sector, so yes. The sector size puts a lower limit on the FS block size.

Honestly on a 200GB disk, how many file systems allocate blocks smaller then 4k right now? This seems like a good thing in general.

Re:

[garett_spencley]

NOTHING is 512 bytes anymore

Unless you've got a powerful fetish for ASCII pr0n

Re:

[aliquis]

Not that it matters but what kinds of asciiporn can you really get within 512 bytes?
Sunet ascii-girl [sunet.se]
Sunet vicki [sunet.se]
Both around 8 kB.

A lot of files are smaller than 512b...

[benhocking]

NOTHING is 512 bytes anymore.

Perhaps, but you might be surprised at how many 0 byte size files there are. They now take up 8x as much space. A reasonable trade-off, but not completely undebatable.

Re:

[a_ghostwheel]

Many modern file systems do not allocate space for 0 length files. NTFS, e.g., will store files up to 512 bytes (may be different number - not sure) directly in MFT. While I don't know details about ext3/hfs/zfs/jfs/etc, I would assume they have similar features.

0 bytes = 0 blocks

[benhocking]

You are correct. I just looked at one of my 0-byte files and found that it occupied 0 blocks. I guess I'm a little out of touch with file systems.

Re:

[drinkypoo]

Are there any filesystems that store empty files without using up blocks?

Yes, I (incorrectly) was

[benhocking]

As others have pointed out, most file systems already allocate 4K per block.

Re:

[peragrin]

that's funny my computer is filled with files of a dozen kb. and web servers are filled with thousands of files only a few kb in size.

Re:

[jgrahn]

NOTHING is 512 bytes anymore. Back in the early 80's IE DOS 2.11 it may have seemed a great idea.

Do you happen to use Windows, and not do much programming?

find ~ -type f -size -512c |wc -l

21% of the files in my home directory are less than 512 bytes. And I don't even use Maildir.

Re:

[Anonymous Cowpat]

My HTPC has hundreds of files that are an average of 1 gigabyte and quite often, twice that size.

So... 2 gigabytes?

Re:Discussed Since 2000?

[RingDev]

Saying 4096 was probably the easy part. Of course someone probably had to research what the largest (time efficient) and smallest (space efficient) block size would give the greatest advantage in space/time for current average files. But eventually you get into the issue of working with Hard Drive manufacturers who likely have to redesign some circuits and controls _from scratch_, BIOS developers who have to recode to detect and support two different standards, and OS/Driver developers who also have to deal with any low level changes...

You're talking about interacting with likely hundreds of companies trying to come up with a single standard that 1) they can all agree on and 2) won't make any of them lose money. Good luck.

-Rick

Re:Discussed Since 2000?

[HtR]

Creating new standards takes time. After some searching, I found the minutes from their annual meetings since they started in 2000.

2001 Chair: "How about we double it?" Vote: Nay

2002 Chair: "How about we triple it?" Vote: Nay

2003 Chair: "How about 4x?" Vote: Nay

2004 Chair: "How about 5x?" Vote: Nay

(minutes from intervening years were tragically lost)

2007 Chair: "How's about 8x?" Vote: Yay

Re:Oh great

[avxo]

Now when I want to update just 256 bytes, instead of reading 512 bytes, changing 256 of them, and writing 512 back, I now have to do this with 4096 bytes. So I end up transferring 3584 more bytes than I otherwise needed to.

So, your O/S requires that you issue all read and write operations using the hard drive's native block size? That must suck. What else must you do? Setup DMA manually in your app? Solder a microcontroller onto the board perhaps? Sarcasm aside, you seem to have a fundamental misunderstanding of what this change achieves, who it will affect, and how. Other posters have addressed those very issues eloquently, so I won't go into that.

They really could do this transparently. Let the driver write anything in any range.

Sorry to burst your bubble but it already is done transparently. The O/S lets you write anything -- from a single byte, to gigabytes -- transparently; all you do is tell the O/S read n bytes of file F so and so into buffer at x, or write m bytes from buffer at y into file F, which is the interface that 99% of programmers use. And after what you wrote above, I find it hard to believe that you are writing the specialized software, low-level drivers and/or controller microcode that could potentially be affected by this change.

Re:

[Skapare]

If it's already transparent, then what is the big deal? If what you say is true, they could make blocks/sectors as long as they want and we won't need to know (except the driver writers need to know what constraints exist in the interface to send the read and write commands to the drive).

Sorry to bust YOUR bubble, but I do know how the OS works, and how it's interface works. The issue depends on what blocksize the commands between driver and hardware require. If you cannot instruct the hardware at a fin

Sector Size vs Cluster Size

[camperdave]

Chances are you're reading 4096 bytes already. Disk access hasn't been sector oriented since before the turn of the century.