The Lies Disks and Their Drivers Tell 192
davecb writes "Pity the poor filesystem designer: they just want to know when their data is safe, but the disks and drivers try so hard to make I/O 'easy' that it ends up being stupidly hard. Marshall Kirk McKusick writes about the difficulties in making the systems work nicely together: 'In the real world, many of the drives targeted to the desktop market do not implement the NCQ specification. To ensure reliability, the system must either disable the write cache on the disk or issue a cache-flush request after every metadata update, log update (for journaling file systems), or fsync system call. Both of these techniques lead to noticeable performance degradation, so they are often disabled, putting file systems at risk if the power fails. Systems for which both speed and reliability are important should not use ATA disks. Rather, they should use drives that implement Fibre Channel, SCSI, or SATA with support for NCQ.'"
almost clicked the link... (Score:5, Funny)
But you lost me the moment you mentioned ATA drives.
Re:almost clicked the link... (Score:5, Insightful)
And yet fails to name any. Looking at Seagates site about NCQ pretty much every consumer model since 2004 has NCQ. This seems overblown.
Re:almost clicked the link... (Score:4, Interesting)
I still bet those drives if you pull power on them will lose the data in their onboard caches.
Which means they are lying about fsync.
Re: (Score:3, Insightful)
As weighty of an argument as your bet might seem to you, I'd refer actual evidence.
Re: (Score:2)
Try it.
There are some decent tools out there to test it.
Re: (Score:3)
If so, the article should link a proper study or basic attempt at surveying drives and how well they survive such behaviour instead of surmising.
Re: (Score:3)
Ok. All my drives, which range in age of at least 4-5 years, support it and they are all the same models that Seagate lists support for. So once again, this sounds like overinflated sensationalism. If it was really such a problem he could have listed a few models to support his claim instead of nebulous handwaving, no?
Re: (Score:2)
They SAY they support it
How can you really tell? It's well established that drives lie about their capabilities.
Re: (Score:2)
Because you have no evidence showing my drives don't whereas Seagate lying would be fraud? Prove the assertion rather than merely repeating it.
Re: (Score:2)
Yeah, it's all a Seagate conspiracy to lie to me. Sorry, but A 7-year-old LJ post hardly has much weight considering NCQ didn't become common in consumer drives until late 2005/early 2006.
Re:almost clicked the link... (Score:5, Informative)
File systems need to be aware of the change to the underlying media and ensure that they adapt by always writing in multiples of the larger sector size. Historically, file systems were organized to store files smaller than 512 bytes in a single sector. With the change in disk technology, most file systems have avoided the slowdown of 512-byte writes by making 4,096 bytes the smallest allocation size. Thus, a file smaller than 512 bytes is now placed in a 4,096-byte block. The result of this change is that it takes up to eight times as much space to store a file system with predominantly small files. Since the average file size has been growing over the years, for a typical file system the switch to making 4,096 bytes the minimum allocation size has resulted in a 10- to 15-percent increase in required storage.
just to clarify what the author's point was:
The conclusion is that file systems need to be aware of the disk technology on which they are running to ensure that they can reliably deliver the semantics that they have promised. Users need to be aware of the constraints that different disk technology places on file systems and select a technology that will not result in poor performance for the type of file-system workload they will be using. Perhaps going forward they should just eschew those lying disks and switch to using flash-memory technology—unless, of course, the flash storage starts using the same cost-cutting tricks.
if you want to argue that, great, go nuts. nobody who actually RTFA thinks the argument is really about ncq. the ac you responded to said
the way I interpret TFA, the problem also applies to SATA drives which do not implement the NCQ specification.
well, here's what TFA actually said:
Luckily, SATA (serial ATA) has a new definition called NCQ (Native Command Queueing) that has a bit in the write command that tells the drive if it should report completion when media has been written or when cache has been hit. If the driver correctly sets this bit, then the disk will display the correct behavior.
In the real world, many of the drives targeted to the desktop market do not implement the NCQ specification. To ensure reliability, the system must either disable the write cache on the disk or issue a cache-flush request after every metadata update, log update (for journaling file systems), or fsync system call. Both of these techniques lead to noticeable performance degradation, so they are often disabled, putting file systems at risk if the power fails. Systems for which both speed and reliability are important should not use ATA disks. Rather, they should use drives that implement Fibre Channel, SCSI, or SATA with support for NCQ
i hope it's painfully obvious by now that the point about ncq is not that some drives don't have it; it's that some don't use it -- mostly so you don't go giving their drives bad reviews for being slow but unnoticeably reliable. if it's disabled, you can enable it. what sata drives don't have ncq? i asked wikipedia:
SATA revision 1.0 (SATA 1.5 Gbit/s) .... During the initial period after SATA 1.5 Gbit/s finalization, adapter and drive manufacturers used a "bridge chip" to convert existing PATA designs for use with the SATA interface. Bridged drives have a SATA connector, may include either or both kinds of power connectors, and, in general, perform identically to their PATA equivalents. Most lack support for some SATA-specific features such as NCQ. Native SATA products quickly eclipsed bridged products with the introduction of the second generation of SATA drives.
so yeah, probably not a whole lot of these drives being sold new, but there are lots of shops that buy used gear because it's cheap. these older sata drives haven't all just disappeared when revision 2.0 came out.
Re: (Score:2)
probably not a whole lot of these drives being sold new, but there are lots of shops that buy used gear because it's cheap
We're talking about hard disks, the disks old enough to have a bridge chip are also puny by modern standards. It would be possible for those drives to sneak into workstations if you are buying them from some marginally reputable local PC builder but you're not going to see them come from any tier 1 manufacturer as they buy new stuff in lots. And you probably won't see them in your servers, most of which seem to be booting from SSD and carrying disks over 1TB each for storage.
Re: (Score:2)
Re: (Score:2)
Re:almost clicked the link... (Score:5, Insightful)
LOSE LOSE LOSE LOSE! YOU WILL LOSE DATA!
Sorry... I'm usually a calm rational person. I almost never become a grammar-nazi, spelling nazi, or troll. It's just that I see this so often I'm afraid one day Webster will just give up and switch the definitions of Lose and Loose.
Re: (Score:2)
All the data gets loose from the drive, that's why you lose it.
"Webster will just give up and switch the definitions of Lose and Loose."
and then...?
Re:almost clicked the link... (Score:5, Funny)
"and then..."
and then all hell will break lose, obviously.
Re: (Score:2)
It's depressing when an engineer that hasn't read much other than textbooks, journals, SF, comics and newspapers starts looking like an intellectual in comparison
Re: (Score:2)
I don't have any figures, this is just my impression, but they seem to go in clusters. You have the ongoing classics lose/loose they're/their/there but once in a while you see a new one, and then you see several examples of it.
It's almost as if people who can't write are copying people who can't write.
Re:almost clicked the link... (Score:5, Informative)
yeah, well, I have quite a bit of experience with samsung (not seagate branded but the older samsungs) drives.
they REPORTED having ncq but you always had to disable them.
I got so that I do this at bootup:
if [ -e /sys/block/sda/device/queue_depth ] ; then /sys/block/sda/device/queue_depth
echo " sda NCQ now off"
echo 1 >
fi
and so on.
performance does not suffer (that I would care about) BUT the data reliab was more than making up for it. no more timeouts, no more syslog 'scaries'.
vendors really do fuck up the protocol implementations. seagate is 'strange' in ways, so is WD, so is hitachi and ibm (I know they are not even in the biz anymore, at least for consumer drives).
windows has a 'blacklist' of what things to not use when talking to drives and so does linux. its a fact of life.
drive vendors are borderline idiots. sad but true ;(
Re: (Score:3)
Intel's early SSDs such as the Intel X25-E were the last time I really got screwed by SATA drives that screwed this up very badly. See the PostgreSQL page on Reliable Writes [postgresql.org] for a lot more details on this subject.
Re: (Score:3)
Intel's early SSDs such as the Intel X25-E were the last time I really got screwed by SATA drives that screwed this up very badly. See the PostgreSQL page on Reliable Writes [postgresql.org] for a lot more details on this subject.
This is why I am never an early adopter. If there were some tremendous emergency that only an early SSD could solve, and life-and-limb were on the line, I suppose I would take my chances. But I've never had that much of a need for an SSD.
I suppose I have pioneers like you to thank, however, for helping to identify and work out the problems so that people like me who wait a little while have such a good experience. It's like volunteer work, except of course that you had to pay in order to do it.
Re:almost clicked the link... (Score:5, Informative)
~$ cat
31
1
1
1
1
~$ cat
simple
none
none
none
none
Re: (Score:2)
Re: (Score:2)
~$ sudo hdparm -I
Model Number: ST2000DM001-9YN164
* Native Command Queueing (NCQ)
Model Number: ST2000DL003-9VT166
Model Number: ST2000DL003-9VT166
Model Number: ST2000DL003-9VT166
Model Number: ST2000DL003-9VT166
Re: (Score:3)
Falllacious appeal to authority. I know who he is yet if it was as common as he claims he could do better than nebulous handwaving.
Re: (Score:2)
Given that we are talking about Kirk McKusick an appeal to authority is entirely fair. Just because he didn't have a bunch of citations or references listed at the bottom of the article does not mean they do not exist somewhere. For you to say it is a "fallacious" appeal to authority is unfair - it has not been proven as fallacious. (You assert it to be fallacious due to a lack of reference... the culture created by Wikipedia and all the "[Citation Needed]" slackers never fails to impress me.) Surely there
Re: (Score:2)
Given that we are talking about Kirk McKusick an appeal to authority is entirely fair. Just because he didn't have a bunch of citations or references listed at the bottom of the article does not mean they do not exist somewhere. For you to say it is a "fallacious" appeal to authority is unfair - it has not been proven as fallacious. (You assert it to be fallacious due to a lack of reference... the culture created by Wikipedia and all the "[Citation Needed]" slackers never fails to impress me.) Surely there exists blacklists in source in Linux/FreeBSD/other publicly viewable code, I also will not hold your hand [slashdot.org] and show you where.
I have personally seen these kinds of issues (with writes not happening soon enough and fsync calls introduced for data integrity) with flash media which is something mentioned in the beginning of article. I would like to further comment that the article talked about other things such as sector size side effects and the impact on useful space. ++Great article. Does anyone else remember how he (Kirk McK.) used to sell shirts and pc stickers? I still have the bsd daemon logo sticker on the case of my first pc.
I think GP has a good point. Sure, Kirk knows his shit. Sure, he really could be considered an authority. Yeah, I can just take his word for it with some confidence.
But that doesn't help me to understand. It's just a memorized factoid. I just know that the statement has a boolean condition of "true". This does nothing to help me understand if *my* hardware is affected. Since just about everyone here has a hard drive (or severa), this would be useful information.
I don't care if Kirk has a 16 inc
Can he turn water to wine too? (Score:3)
Given that we are talking about Kirk McKusick an appeal to authority is entirely fair. Just because he didn't have a bunch of citations or references listed at the bottom of the article does not mean they do not exist somewhere. For you to say it is a "fallacious" appeal to authority is unfair - it has not been proven as fallacious
It's usually up to the one who makes a claim to back it up with evidence, not for others to disprove it - and they can't either, because there's falsifiability here. If I show that my drive has NCQ that works, that still doesn't falsify his claim. I can't bloody well test every drive on the planet, so there's no way to disprove him.
So yes, this is appeal to authority and what you do is putting the onus on those who disagree to prove a negative.
He may be right, and he's certainly renown, but to jump from t
Re: (Score:2)
Which is basically none of them. I would be astonished if anyone could link me a drive sold on newegg, amazon, or by Dell that does not implement NCQ when set to AHCI mode.
Re: (Score:2)
how can you actually tell that it's implemented properly, or at all? What if it's all a big lie, just like other parts of the protocol?
Re:almost clicked the link... (Score:4, Informative)
you'll see it in syslog!
timeouts, retries, even exiting the bus and doing full bus resets (which are slow and you'll NOT miss them).
as I posted before, older (5yr) samsungs were notorious for SAYING they support ncq but you would be foolish to let it just negotiate it and use it.
this was how things were in the very early days of 10/100 ethernet and full/half duplex. yes, the early models 'negotiated' duplex but many of them got it wrong and you'd have to manually set this on hubs/switches since you knew better than the equipment. there were even early NIC chips that worked better at 10meg ethernet than 100baseT! we would do ftp transfer tests and quite often a GOOD 10baseT was more reliable (over time) than 100baseT. the same happened to gig-e, too, in the early years.
Re: (Score:2)
http://www.amazon.com/Seagate-Barracuda-3-5-Inch-Internal-ST2000DL003/dp/B004CCS266/ref=sr_1_1?ie=UTF8&qid=1347056020&sr=8-1&keywords=ST2000DL003 [amazon.com]
OF coutrse, you will need to buy it and tested it for yourself since you dismiss what other people who've tested say.
Re: (Score:2)
So it's claimed. Provide evidence by listing the models which do so rather than handwaving supposition.
Re: (Score:2)
Re: (Score:2)
Except the OS, drive, and driver all claim that NCQ is working on every single SATA disk I have seen that has been set to AHCI. To say "yea but theyre still lying".... why shouldnt we ask for specifics? Are we expected to go out and test every extant drive to see whether it supports NCQ?
If the author had specifics that he tested and found to improperly implement NCQ, perhaps he should have included his data so that it could be verified. All he gave was a general overview of tagged queuing and NCQ, and th
Re: (Score:2)
No. The person making the claim should state "I have a Foobar model X. I ran command 'wibble' and the result was 'herpderp'".
And then anyone else who has the same disk can try it too, and ask if running 'yipyip' makes any difference, or ask if that applies to the model Y too...
Specifics, or STFU.
Re: (Score:2)
Read it? Just did. Nothing concrete in there, just vague scare mongering. I am likely to get more useful information from the peanut gallery here. I've already seen one guy with an actual real world example.
Re: (Score:2)
I've got a wireless router to sell you. It has all these features for QoS and firewalling and it's secure and it gets 300 mbps and it's got the DLNA,I swear.
You mean this thing? [newegg.com] (after installing dd-wrt)
On a serious note, exactly how is one supposed to purchase a drive if we cant trust anything on the product page? Just guess?
2 out of 3 (Score:4, Insightful)
Cheap, fast and reliable.
Pick any two.
Re: (Score:2)
Only because of market segmentation. They sell the same drives as Enterprise Grade SATA with these NCQ turned on in firmware as they do to consumers with it turned off.
Even worse are the RAID controllers(looking at you DELL) that do not disable the cache on the drives when you tell them to disable the write cache. You think your data is safe, then you lose power and what should be an oops has you going to your backups and doing a rebuild or swapping over to a replicated box.
Re: (Score:2)
They sell the same drives as Enterprise Grade SATA with these NCQ turned on in firmware as they do to consumers with it turned off.
What you get with an "enterprise" sata drive is higher MTBF and a firmware tweaked to work well with RAID (desktop drives try to be more forgiving for IO errors, while the enterprise drives are more quick to decide "ive failed, let the raid controller do its work").
Im not aware of any sata drive that doesnt support NCQ-- its certainly on every desktop drive ive used excepting MAYBE the very first sata drive I bought in 2003. Certainly all SSDs I am aware of (except niche super-low-end ones) and all mass-ma
Re: (Score:2)
Even worse are the RAID controllers(looking at you DELL)
You buy RAID controllers from DELL? You deserve what you get. Buying DELL server gear is like bringing a Schwinn Varsity to the Tour de France.
Re: (Score:2)
Re: (Score:2)
Heard of it? I remember living it in high school! I think at the time it was a 2.1GB drive and then the next size up. And "kibibytes" was still dog food, not an overindulgent unit of measurement. Ah, the days...
Re: (Score:2)
Re: (Score:2)
you need to define terms before I can even pick one.
BTW, new manufacturing technique can accomplish all three.
Performance degradation (Score:2)
One can't have ones cake and eat it. Speed or reliability, there should be more differentiation and more clarity in the specs. I want my backup-disk to be very reliable, I want my boot-disk to be fast. Best performance for both, but different circumstances.
Re: (Score:2)
In that case boot should just be on an SSD, where these issues pretty much disappear anyway.
Re: (Score:2)
That's not true at all. The X25 drives from Intel were terrible in terms of safe writes. The newer Intel drives are better, but only because they added a capacitor to allow in-process writes to complete -- simply being solid-state does not resolve these issues, and in some cases can make them much worse.
Sorry, what? (Score:4, Insightful)
We're talking about ATA drives?
As in non-SATA drives?
Who has those anymore?
While the article is good for publication in an academic journal like ACM, it's useless for the real world.
For that, the author should tell us whether most drives on the market have NCQ already or not. Popular drives like WD Green and Seagate's various lines.
Otherwise, saying "$A is useless without $Y" is pointless.
Re: (Score:2)
i'm guessing that since he's talking about 4K sectors, he means SATA since none of the PATA drives were large enough to warrant the switch from 512.
Re: (Score:3)
An SATA drive is a subset of ATA drives. You're thinking of PATA or IDE drives.
http://en.wikipedia.org/wiki/Serial_ATA [wikipedia.org]
In other words, when someone says "ATA drives" they aren't exclusively talking about non-SATA drives.
Re: (Score:2)
Wrong. ATA is the original name of what was renamed to PATA once SATA was introduced. So if he is saying what you claim he is using the term incorrectly.
Re: (Score:2)
ATA doesn't exist anymore. It's like saying you are going to New York by saying "I'm going to the USA". It might be technically correct, but entirely useless, especially if one is in California telling all his friends he's going to the USA. It's not only tech
Re: (Score:2)
Regardless, the author's choice of terms plus lack of additional clarification totally muddled what he might have been trying to say.
Also, there's no context for what he's saying ("SATA without NCQ is bad"). It's like saying MySQL without foreign keys is bad, without mentioning the context that MySQL does have foreign keys these days.
Re: (Score:2)
Re: (Score:2)
No, YOU are wrong.
AT refers to ATA and ATAPI command-set; which SATA uses; with improvements and new features.
It was change to PATA to more accurately describe how it's moving data through it's channels i.e. parallel
Re: (Score:2)
Re: (Score:2)
dude, the ata vs sata is ONLY on the controller card!
the drive spindle is the same. its funny to hear someone say that older ide drives are 'noisier'.
you CAN say that older drives are noisier than new ones. and I'd respond with "DUH!"
but scsi, sata, ide, sas, fc: the drives are still the same. controllers are what varies.
Re: (Score:2)
ATA drives...? WTF (Score:4)
We shouldn't even be writing for ATA drives anymore. And any name brand manufacturer that you would trust (on a mediocre level) WD, Seagate etc... all support NCQ.
Re: (Score:3)
Are you saying we should cast the ATA driver out of the kernel and dispose of all our ATA hardware?
Even though it's not in new hardware any more, we still need to support it in existing hardware. The driver still needs work when the kernel APIs change.
Re: (Score:2)
Good point. That said, ATA hardware is really quite old. I don't know that it would hurt to say, you know what if you want to run 3.6 of Linux you aren't going to have an ATA drive. If they run to run ancient hardware, let them run older hardware (note I didn't say ancient hardware).
I use zfs (Score:2)
I put my important files (pr0n, etc.) on my zfs mirror file server and scrub each week. The really important stuff (tax returns, etc.) I put in a safe deposit box at the bank.
I work in the storage industry. (Score:3, Informative)
Don't assume that "enterprise" disks do this correctly either.
Many have options to make them behave properly but out of the box have write back caches and ignore FUA or similar, leading to the same problems.
Duh (Score:3)
I never recommended ATA drives for servers. Really old stuff that used MFM and RLL drives was back in the era where the just anything else. I used ATA drives for my home stuff and lab where it wasn't expected to be very reliable, and SCSI was all I used for a very long time. Even today I recommend against SATA though it seems tolerable, but SCSI drives are still my standard.
Mostly I thought SCSI drives were also made better, but Seagate and WD convinced me otherwise.
And yes, MFM drives in a Novell DCB setup were among my first servers. Making NW 2.15c mount a 4 GB volume just so you can say you did it would not be fun today, but back then it was work, and clients paid for it. I'm glad it wasn't a VINES server.
Re: (Score:2)
Funny, google has tens of thousands of servers and they put cheapo SATA drives in them.
Re: (Score:2)
And Google relies on multiply redundant servers and data, both for performance and reliability. Not many small businesses are gonna want to put in 5-way clustering.
Re: (Score:2)
Not to mention Google doesn't have to provide a "right" answer. They can provide any answer that seems approximately correct. However for their accounting, payroll and tax systems I'd bet $20 that they use name brand servers running name brand OSes and name brand software.
Re: (Score:2)
If you take a few moments and look into it, google buys custom servers that are more like commodity boxes than premium servers.
Re: (Score:3)
And google is not your average company.
Google has a LOT of servers running much the same workloads. As such it makes sense for them to put in the software engineering effort to achive higher level redundancy. They engineer things so they don't have to care if a server dies.
Most companies have a relatively small number of servers each with a particular task. If one of those servers fails it's a much bigger deal that can mean significant downtime and/or data loss. IIRC restoring a big database from backup an
Re: (Score:2)
Naw, just old.
Not about ATA, about enterprise data storage (Score:5, Informative)
2) The article's point on NCQ is that many consumer drives do not implement it correctly, and disable the write cache on the disk and issue cache-flush requests to increase performance, but leading to possible file-system failures if there is a power outage.
I think this article is saying that for the enterprise, buy enterprise drives, not consumer drives. Most consumers use laptops now, so power failure doesn't fit in, and consumers prefer speed over reliability, which is why I've always been stuck using laptops lacking ECC RAM.
Re: (Score:2)
When the power goes out, all cards are in the air anyway. We had a UPS boo-boo and our enterprise drives (both SCSI & SAS) managed to corrupt data, even with a battery on the controller itself (battery was in good health.)
Shit happens. It's pretty damn difficult to account for power failures... even with battery backups on the local controllers you can only do so much.
Re: (Score:3)
Re:Not about ATA, about enterprise data storage (Score:5, Informative)
The "Turn off Windows write-cache buffer flushing on the device" option activates an ancient windows bug, and should never be used.
When Windows 3.11 was released, MS accidentally introduced a bug, whereby a call to "sync" (or whatever the windows equivalent was called) would usually be silently dropped. At the time, a few programmers noticed that their file I/O appeared to have improved, and attributed this to MS's much marketed new 32-bit I/O layer. What a lot of naive developers didn't notice was that the reason their I/O appeared to be faster was that the OS was handling file steams in an aggressive write-back mode, and then calls to "sync" were being ignored by the OS.
Because of this, there was a profusion of office software, in particular, accounting software, which would "sync" frequently - some packages would call "sync" on every keypress, or everytime enter was pressed, or the cursor moved to the next data entry field. As on 3.11, this call was effectively a NOP, a lot of packages made it onto client machines, and because it was fast, no one noticed.
With Win95, MS fixed the bug. Suddenly, corporate offices around the world had their accounting software reduced to glacial speed, and tech support departments at software vendors rapidly went into panic mode. Customers were blaming MS, Win95 was getting slated, lawyers were starting to drool, etc. Developers were calling senators and planning anti-trust actions. The whole thing was getting totally out of hand.
In the end, MS decided the only way to deal with this bad PR, was to put an option into windows, where the bug could be reproduced for software which depended upon it. The option to activate the bug was hidden away reasonably well, in order to stop most people from turning it on, and running their file-system in a grossly unstable mode. However, in Win95 - Vista, it had a rather cryptic name "Advanced performance", which meant that a lot of hardware enthusiasts would switch it on, in order to improve performance, without any clear idea of what it did. At least in Win7 it now has a clear name, even though it still doesn't make clear that it should only be used for when using defective software.
Re: (Score:2)
Or you can buy a real RAID controller with battery backup for the cache, in which case you are just fine with the cheap SATA drives.
NCQ - Native Command Queueing (Score:5, Informative)
Native Command Queueing [wikipedia.org]
Because not everybody knows everythingTM
Get Hardware RAID (Score:5, Insightful)
The people who make hardware RAID know all about the lying drives, they get good information from the manufacturer on how to make the drives play nice with the RAID controller.
Just read the compatibility charts for your RAID controller, many drives have footnotes with minimum drive firmware requirements and other odd behavior.
Re:Get Hardware RAID (Score:4, Interesting)
The only real advantage to "Hardware RAID" is the battery backed cache. Hardware RAID comes with the disadvantage of a whole other operating system "firmware" with its own bugs and often proprietary disk layout. Parity calculations are nothing for current CPUs, so the onboard processor is not so useful. Advanced filesystems such as ZFS or BTRFS need direct access to the disks. I'd like to see drives and/or controllers with battery backed cache. Until then, I rely on my UPS.
Get Hardware RAID outside of edge cases like above (Score:2)
how about a utility or SMART patch (Score:2)
Re: (Score:2)
Whether this sort of thing works correctly can change based on drive firmware. So even a given model/serial number combination can change which type of results it gives over time. There is no substitute for testing yourself.
Linus's Input on Write Cache (Score:4, Interesting)
I think this is quite interesting.
http://yarchive.net/comp/linux/drive_caches.html [yarchive.net]
While I've often gotten the impression that the write cache opens up a large "write hole", Linus says that data is cached only for milliseconds, not held in the cache for several seconds. Still, I'd like to see battery backed caches in regular drives and/or controllers.
Would be nice to hear from some drive firmware writers.
Read the solution here .. (Score:2)
ATA? (Score:2)
The real problems are entirely different (Score:2)
The article is total crap, every disk supports NCQ as half the world's population has pointed out in the comments.
The problems are elsewhere: When a disk suddenly loses power while it is writing, there is a risk of various interesting errors. The disk may a) write nulls instead of the correct data, b) write garbage instead of the correct data, c) fail in the middle of a Read-Modify-Write operation and therefore destroy data in files which weren't written to at all, d) write good data to the wrong place on t
Or you could just use a decent controller (Score:2)
I can see how it's a big deal with workstations/desktops/laptops but that's really only a small chunk of storage in general.
Some Quick Facts (Score:2)
This is a lot of noise for nothing. For kids and amateurs, here's a quick summary...
fsync used-to be the go-to, but that was decades ago, when IDE was in full-swing. Back then, there was a big hub-hub about drives lying. Since then, it's been common knowledge and status-quo that fsync is not trustworthy, end of story.
Today, we have WRITE BARRIERS, and they work great. Ever since, say, the advent of 60GB IDE drives, I've never found a drive that doesn't support write barriers, and in my conversations wit
Re: (Score:2)
news flash (Score:2)
Shitty consumer oriented hardware not suitable for enterprise class data integrity and retention.
If you need data integrity and cache, you need a battery backed up IO controller and UPS for a start. If you're relying on the fact that turning cache off on the drive is going to ensure that your writes complete before the power goes out to the drive, you've already set sail for fail.
Re: (Score:2)
They might SAY they support it, but HOW CAN YOU REALLY TELL?
We all know that hardware LIES all the time about its ACTUAL capabilities, just READ the article!
Re: (Score:2)
They do despite the people parroting his words without being able to back up the statements beyond a fallacious appeal to authority.
Re: (Score:2)
Re: (Score:2)
Except on the many Linux versions where O_DIRECT doesn't work properly. I have kernels where it works as expected; ones where it quietly fails to sync to disk; and ones where using it causes a PANIC. It's never been a priority for that API to function correctly given that Linus thinks direct IO is totally braindamaged [lkml.org].
Re: (Score:2)
Re: (Score:2)
well given that the deflector dish willbe/was a huge most likely METAL dish then while it may not have been designed to take power from the warp drive it most likely could (just like you could wire your home power into your antenna).
Re: (Score:2)
actually, the deflector dish is designed to take power from the warp drive. Otherwise you can't extend shields to protect that other ship or station when you need to. Thus it wasn't a crazy idea on the part of the engineers to divert power to the deflector dish though it should have been divert power to the navigational deflector as that's what the dish is.
Re: (Score:2)
I don't know what you are smoking but NCQ isn't an acronym for "caching". It's native command queuing and what it does is allow your OS to have multiple commands inflight simultaneously. Which is a big deal and very helpful.