SCSI vs. IDE In The Real World

An anonymous reader writes "Gerard Beekmans has a really good comparison of the speeds of IDE and SCSI drives up over on devchannel.org. Should help put an end to the myth of IDE erasing SCSI's speed advantage." Note that Beekmans' test handicaps the SCSI disk a bit, with interesting results. (DevChannel, like Slashdot, is part of OSDN.)

FYI: Gerard Beekmans...

[Aardpig]

...is the original creator of Linux From Scratch [linuxfromscratch.org], and therefore registers very high on all standard 7331-meters

Re:Not an accurate test

[Frostalicious]

He reads one and only one file.

No, he read 50,000 files.

every email message is stored in a separate file....I used a maildir folder containing 50,000 emails.

Sure it would have been better for him to run a number of scenarios, but he couldn't build himself a whole lab due to cost factors. Even with this rudimentary testing, a difference of 8 times indicates to me that there is an effect.

Re:SCSI vs. IDE: Same experiences

[jawtheshark]

Same experience here too... I have a P166 server with 128Meg RAM, and before it ran just from an IDE disk. It was terribly slow. I later replaced it with an Adaptec 2940UW with an relatively old IBM 18Gig SCSI harddisk. The difference was mindblowing. It was as if I had upgraded the CPU with a P-II...

It's insane... SCSI is worths it's money... I just don't have the money... ;-)

Re:Does it matter?

[hayesjaj]

Yes, it does matter. The applications that he tested did not include cpu usage, which are assumed to be neglegable. Try doing an IDE Raid setup while your cpu usage is already high (e.g., a cpu and disk intensive game or running many applications in virtual memory). The simple fact that the scsi controller handles the io operations will cause your performance to increase even more than the article suggests. Now, do you need to spend 500+ dollars to store mp3s and your pron? Probably not.

Re:You get what you pay for.

[rangek]

If you actualy run your tape backups on a nightly basis, religiously, you have no business buying anything less than a DLT drive. The DAT drives and all the other cheaper solutions are "consumer-grade", meaning they're only intended for the occasional use (such as doing a full backup right before wiping out or upgrading your hard drive). They'll quickly wear out and break down if you do much more than that with them.

What about VXA [exabyte.com]? We run at least five tapes a week through our VXA-1 drive, and have had almost zero problems with the tapes (I think we had one tape "wear out" and get eaten in two years.) And it's not like we don't know if our backups are good. Just last week some one snagged some files they accidentally deleted off an old tape.

But *why*?

[pla]

I have done similarly-informal tests myself, with comparable results.

I can't say I understand why SCSI performs so much better than IDE, however. In this particular test, he compared what amount to evenly-matched drives, specs-wise, and even gave the IDE drive the better machine. Yet, the SCSI drive completely crushed the IDE drive, no question about it. And as I mentioned, my own informal tests have shown the same results.

What explains the difference? Same spindle speeds, similar read rates (both buffered and unbuffered), similar seek times... What other factors exist that make so much of a difference? Just higher quality controller hardware? And if so, would an IDE drive on a high-end controller perform comparably?

Personally, I'll still take 4x the size for the same price, since cost and size (with "okay" performance) matters more to me than raw speed. But I wish I knew why one performs so much better than the other.

Differences between SCSI and IDE

[vlad_petric]

The main difference is how the bus deals with transactions. SCSI is a split transaction bus, which means that you tell the drive - give me D, A, C, B (in a non-blocking way). The drive will give you back data in the order it finds more convenient (let's say A, B, C, D if it were at one end of the drive) - it effectively rearanges the requests/responses. The operating system already does that to some extent (elevator algorithm), but the OS can't do a better job than the disk controller itself. This is very crucial for servers, where multitasking happens a lot (clearly mutt won't see any difference)

The second - CPU utilization. A SCSI controller does a lot more work by itself than an IDE one - therefore it requires much less interaction with the CPU.

Then we have the bus bandwidth - this is probably no longer an issue, as ATA/66/100/133 can pipe enough bytes per second

Finally, the most important one - manufacturers simply don't make a 10k RPM hardrive IDE drive ... And 10k - 7200 makes a hell of a difference.

Re:Unfair comparison

[fishnuts]

The number of tracks a drive has is inconsequential to the access time. The time it takes for the read/write head to move across the platter (since the platters in ide and scsi drives are nearly the same size) is a product of how heavy the heads are and the strength of the motor assembly (which is actually a pair of coils moving between two very strong magnets). I'm inclined to believe that the IDE drive's slower seek performance is actually because if a cheaper head/motor mechanism. After all, the IDE drive _is_ cheaper, so this is one place where costs are cut.

As far as buffering is concerned, random access reads, such as when reading in many small files, does NOT benefit in speed from larger on-drive caches. That cache is used mostly for multi-block read-ahead and write-back on sequential accesses, and is usually segmented into "areas", where sections of cache memory are linked to sections (cylinder groups) of the platters. The more cache you have on the drive, the less "thrashing" the drive has to do when many disperse files are written to, since it can group many operations into their appropriate "areas" of cache. This doesn't help as much for random reads (like maildir!) as for sequential reads.

So why does scsi really have an advantage?
1) tagged queuing. The host controller can queue up to 255 (depending on the controller and drives) commands to each drive, and can request the drive to "update" the controller on its status completely asynchronously, and in groups of responses. With one single "message" from the controller, the scsi drive gets up to 255 times as many low-level commands (block reads/writes/seeks/cache ops) as one command between an ide controller and its drive. For the test in that article, the host OS simply had to read the directory, and send off a bunch of commands to the SCSI controller to read a list of blocks, and notify the host when it was done moving everything into memory. Meanwhile, the IDE drive was busy doing every operation one at a time.

2) multi-processing-friendly communications protocol. Because of the way scsi offloads most of the processing from the host, the host can spend less time waiting for an interrupt to complete (remember, an IDE device can only complete one low-level command at a time, per interrupt), and group commands from many processes together into one stream of scsi commands, or multiplex commands to multiple devices on the scsi bus in one operation. IDE is incapable of sending commands to two devices on the bus simultaneously. i.e. It must wait for an operation to the master drive to complete before sending something to the slave on the same channel, or to a drive on a different channel. In the test, the SCSI drive performed so well even though it was on a 'server' sharing resources with other processes because the OS was able to group commands together from all the different processes, while the IDE system had to "pause" programs that were waiting "in line" for ide operations from another process to finish. These processes can also include kernel disk read-ahead threads, write-back buffering, other programs on the system, etc.

IDE will eventually have tagged queuing (some controllers/drives already support it), but there's still the problem of the IDE controller holding the host in an interrupted state (interrupt contention) while the operations complete. It's how the IDE spec works. DMA helps a lot here, but only makes the data transfer faster. The time the various low-level commands take and how they tie up the host cpu is still the bottleneck. To be fair, SATA should have all this stuff covered, and will likely bridge the gap between IDE and SCSI performance, but I don't believe it'll approach the overall reliability and robustness of SCSI.

Disclaimer: I've described the advantages/disadvantages above as best as I could recall. corrections welcome.

Re:Meaningless..

[Hoser McMoose]

Sure it's huge, but I could get just as big of a difference between two runs using the EXACT same hard drive.

The tester didn't even bother to check and see if the files are fragmented, let alone checking to see if the files are on the same part of the disk. The original poster was right, this was NOT in any way a "good" comparison.

If you actually do want a good comparison, head on over to www.storagereview.com. They have compared many different SCSI and IDE drives and have a VERY good grip as to where and when SCSI's performance advantage comes into play.

Here's a quick and easy way to do things: Click on "Performance Database" at the top of the page, and then do a head to head comparison of a bunch of few SCSI drives and a few IDE drives. This will give you a whole whack of benchmarks. What you'll find is that on desktop applications, a 7200rpm IDE can almost always outperform a 7200rpm SCSI drive and is usually about on-par with a 10,000rpm SCSI drive. But, as soon as you get into their server benchmarks, the SCSI drives wipe the floor with the IDE drives.

Then it simply becomes a question of whether you run a server or a desktop. Different drives for different markets.

Re:Meaningless..

[BrookHarty]

Storagereview.com rocks, I use to check on HD statistics, speed, cpu usage, compare against current HDs. The Benchmarks section is the only place to look for true speeds on HD compared to all brands.

They have a leaderboard which REALLY shows how HD's compare. One thing I really like is how CPU usage is going down with newer IDE HD's. I always hated how IDE spikes the CPU. Ive always tempted to buy an addon ide raid controller to help smooth out the IDE spikes, but keep the size advantage of IDE. If money was no object, I'd be running a Raid setup with Maxtor Atlas 15K 73gig SCSI drives.

Re:SCSI vs. IDE: Same experiences

[pjrc]

Obviously someone has never read the ATA specs... even ATA-2 from the early 90s...

In IDE, the OS has to position the head, wait x sectors, read a sector, save it into memory, go to the next sector, read again, store again, and so on.

No. It's NEVER been that way in ATA. Not even the earliest IDE drives. With MFM drives before IDE, and on the Apple ][ and C-64 this sort work was done, but IDE has never been anything like this.

With ATA (a.k.a. IDE), you write 5 bytes to registers to indicate the starting sector number and the number of sectors you want. Then, you write to the command register to transfer control to the drive and it begins working on your command. All modern systems will (usually) issue the "read multiple" command, which instructs the drive to read many sectors into its buffer and give an interrupt when they are all available in the buffer. This isn't something new. The read multiple command has been in the ATA specs for a long time, and PCs have made use of it since at least the days of Windows95 and Linux kernel 1.0. When the drive has all the sectors in its buffer, it asserts the interrupt pin. The read multiple command comes in PIO and DMA flavor, and if you wrote the DMA version to the command register, a DMA operation happens to transfer all those sectors to whereever you set up the DMA controller to store them.

SCSI gets most of its advantage from tagged command queuing and disconnection. These features have appeared in the very latest IDE specs, and so far very few ATA drives support them.

Re:Not an accurate test

[pjrc]

I've always wondered why they don't upgrade IDE with a better command set much like SCSI,

ATA-5 added multiple command queuing and disconnection (the primary benefits of SCSI). A few drives today support it. Someday, almost all will.

Tagged command queuing and better drivers in linux are most likely the reason the SCSI drives were able to read 50000 files much more rapidly than a similar IDE drive.

well they haven't they just increase the clock speed

DMA modes 0 -> 1 -> 2, followed by UDMA 66 -> 100 -> 133, and now SATA gen1 (150 Mbytes/sec), and in a year or two SATA gen2 (300 MBytes/sec)... and maybe someday gen3 (600 MBytes/sec)... and likely faster speeds thereafter, but this is what is officially approved and published today.

and offer better buffering.

Shopped for drives lately. 2 and 8 meg buffers?

This review does one and only one test which proves that SCSI wins on one test this is not a good article. He reads one and only one file.

Yes, 1 test. No, 50000 files rather than just 1 file.

Slashdot editors should be rejecting this article in favour of one with a real indepth analysis.

Slashdot "editors" should do many things to be worth of the title "editor". At least lately they've been a lot better about spelling errors.

Perhaps they didn't actually read the article (or merely skimmed it), or they weren't very knowledgable about the evolution of IDE and SCSI... much like their readership posting the majority of these comments.

Real-world Price Comparison

[SlashChick]

Okay, I know it's bad form to reply to your own post, but I thought I'd add the results of a little Pricewatch search I ran.

I said in my earlier post that you can get 3x8MB cache drives plus a 3Ware IDE RAID card for about the cost of one SCSI drive. Here are the actual cost breakdowns. All prices include shipping.

IDE SYSTEM
1 x 3Ware 7500-4 4-port RAID card: $250
3 x Western Digital WD800JB hard drives (IDE; 80GB; 8MB cache) = $219.

TOTAL for IDE system: $469.
Total usable space: 160GB.
Bonus points for RAID-5 redundancy.

SCSI SYSTEM
1 x Adaptec 29320 U320 SCSI adapter (64-bit PCI card): $179.
1 x 73GB U320 10,000RPM drive by Maxtor: $296.

TOTAL for SCSI system: $475.
Total usable space: 73GB.
Bonus points for raw speed.

Now, if you want to run a real benchmark, pit these two systems against each other in a nice server (dual Xeon preferred.) Make sure it's the same hardware being used. This would be a benchmark that I'd be interested in seeing the results from. I'd take any test pitting these types of systems against each other a lot more seriously than any "benchmarks" in the current article.

Re:Meaningless..

[adam872]

A bit of both. There is no doubt that the Tagged Command Queing in the SCSI protocol helps enormously with high transaction rates...The drives are often better made as well.

Re:Does it matter?

[ckaminski]

I got a Raid 5 ide setup on the Promise SuperTrak Sx6000, and that 3disk array doesn't even come CLOSE to touching the performance of some of my SCSI arrays at my workplace. Do one task, and it's fast. Start any amount of multitasking, and it dies, quick. SCSI would never fall over so badly.

Granted the large caches make up for it, but comparing speed of IDE vs. SCSI is as pointless today as it was 10 years ago. (try finding cheap reliable 15K RPM IDE drives). Price/performance, maybe IDE wins depending on who you are...

Re:Meaningless..

[omaha]

Poppycock. That used to be true but it isn't anymore. Many IDE and SCSI drives use the exact same platters. There is no difference in the technologies. Desktop users wouldn't accept the failure rate. In fact, everytime a series came out that had problems of this type, that company suffered quite a bit. (Everone quit buying them and the word spread fast) As a result, the big name manfs use quality platters.

As to the head and mechanisms, that may differ. The way I see it, the biggest stick that SCSI currently has over top of the line IDE is command queueing.

Re:Real-world Price Comparison

[raptor21]

Why use a U320 card if all you plan on connecting is one drive?

The bus bandwidth only comes into play if you have more than 4 disks on the SCSI bus. The max sustained bandwidth of a 10k RPM SCSI disk is ~70 MB/s. You would need a U320 controller if you plan on putting about 4 of these disks on the same bus that is 4x70 MB/s = 280 MB/s.

You could get away with an LSI logic single chanel U160 64 bit PCI controller for $46.75 shipped (pricewatch). Maxtor 10K rpm 73GB U160 drive can be had for $147 shipped.

So your total for SCSI with two drives = $294 + $47 = $341

Added bonus you can connect 15 drives to the SCSI controller and only 4 to the IDE RAID controller. you can get a dual channel lsi1010 controller for $64.

Better comparison

[ValourX]

I wrote this months ago using newer hardware and comparing RAID arrays as well:

Comparing Hard Drives [thejemreport.com]

It was really meant to be a comparison of brands, RAID vs single drives, and SCSI vs. IDE. At the time SATA wasn't out yet and I couldn't get any SATA drives to add to the comparison later.

-Jem