Ask Slashdot: Breaking the Computing Bottleneck? 254
MidKnight asks: :
"With CPU speeds venturing into the GHz, typical DRAM
access times falling towards 6ns, and network speeds trying
out the 100Mb waters, the major computing bottleneck
continues to be the hard drive. While other hardware
speeds grow in orders of magnitude, hard drives aren't
much faster than they were in the 70's. So, the question
is, when will mass storage speeds start catching up, and
what will the cause the revolution?"
Price is key! (Score:1)
they command anywhere from a 40-100% price increase for the same capacitiy and rotational speed.
Most people only have 1 hard drive and would never see the performance improvement from SCSI. Most of SCSI's benefits are when you have more than 1-2 drives and lots of simultaneous traffic which the average PC user simply doesn't do.
The PC BUS is the real bottleneck. (Score:1)
Re:Crystal Storage? (Score:1)
Multithreaded architectures... (Score:1)
Only if the CPU has no way to tolerate the latency involved. Go read about multithreaded architectures. The principal is that the CPU should never wait for a high-latency event to complete before continuing. Instead, it just switches to a new thread. Normally these threads are much lighter-weight than the POSIX variety, but they're of similar flavor. I do not know a good, on-line summary of the various types of multithreaded CPUs. If you're interested, look around U. Wisc., Stanford, and the ACM Digital Library for phrases like `multithreaded', `SMT', etc. You could also go to a bookstore and flip through Culler and Singh's ``Parallel Computer Architecture'' book for some now-slightly-dated info.
One of these machines does exist. It's from the Tera Computer Company [tera.com]. It has no data cache, and it's pretty fast for scientific-style jobs. IIRC, we were getting the equivalent of an 8-way UltraSparc-I on only one or two processors (argh, I really can't remember) on a list ranking benchmark. They're only considering the latency to memory locations (around 170 cycles, which is between 9 and 1.5 insn word opportunaties for a stream), but the general idea should hold for any high-latency event. The engineering for extremely high latencies probably gets trickier, though.
Expect multithreaded architectures in the higher end workstations within the next 3-4 years (at the longest). A certain up-coming architecture has the ability to grow in this direction pretty easily (I believe, haven't thought about it too hard), but one of the folks involved in the design was quite down on multithreading a few months ago.
Jason
Whatever happened to software ramdrives ? (Score:1)
Onei
Re:Crystal Storage? (Score:1)
Exciting stuff! More storage/less space.
SCSI will not help (Score:2)
You can think of all the storage space in your computer as being arranged in a heirarchy from fastest to slowest: CPU registers, L1 cache, L2 cache, RAM, disk, nonlocal media (network, tapes sitting on the shelf, etc.).
The CPU works directly on registers. When it needs something else, it sends a request to the next level, L1 cache. If it's not there, that level requests the data from the next level, and so on. The first four levels of the memory heirarchy are fairly close to each other in performance:
That's five orders of magnitude worse than RAM (by contrast, RAM is one order of magnitude worse than L2 cache). This is why virtual memory thrashing is so disastrous for performance.
RAM-to-disk is the biggest critical gap in the memory heirarchy. SCSI hardly does anything to close the gap. A 7-ms latency is not much better than a 9-ms latency-- it's still millions of cycles-- and SCSI's higher transfer rate doesn't mean a thing for virtual memory since pages are pretty small to begin with.
OTOH, applications that are limited by transfer rate rather than latency may benefit from SCSI, but transfer rate has never been the big problem with hard drives.
~k.lee
High density, non-volatile memory (Score:2)
The Hitachi press release [hitachi.com]
Re:SCSI! (Score:1)
real-world comparison of access times (Score:1)
Let's say you have a large forklift which is capable of carrying a few hundred boxes of paper from your archives. You're the forklift operator, and I'm the CPU.
Ok forklift, go get me box of paper #421769, which is in our on-site storage facility downstairs.
You drive off, and are back about two minutes later with the paper. That's the speed of memory.
Now, forklift, I'd like box of paper #090210, which is in our long-term storage facility.
You drive off. I go home. I have dinner, come back the next day, do stuff, go home
FOUR YEARS later, you come back. That's the speed of hard disk.
No matter how big that forklift (bus bandwidth) is, the latency difference between two minutes and four years is a pretty huge one.
--
Re:SCSI! (Score:1)
I can imagine one really big head the size of the platter itself with 1 mini reader/writter per track, hell you wouldn't have the spin the thing
Of course, that might cost too much
--
Network is the bottleneck, not disk drives (Score:1)
As for latency, hard drives are behind memory and CPUs, sure, but they're still ahead of networks. Most large networks have much worse than 9ms latency.
Re:Crystal Storage? (Score:1)
I disagree, compare the price of a decent tape drive to the average harddrive size, the price is almost on par with each other, hence Tape media is not a useful option for people anymore, unlike years ago when everyone wanted a 'colorado jumbo 250'
it only takes time...
Re:Mechanical Parts (Score:1)
I agree. less moving parts also means === SMALLER
Re: RAID: a godsend... (Score:1)
Re:Display resolutions, too. (Score:1)
Re:SCSI! (Score:2)
--Zach
Re:hd not the issue (Score:1)
Yeah, a new hardware architecture can help lots. At Cambridge Computer Labs (part of the University) they have a prototype computer built on the Desk Area Network. The premis here was that the main system bus was replaced with an ATM switching fabric, so any device could talk directly to any other device, and these conversations could go on in parallel.
A good example is drawing video to the screen from a camera. The data would come off the capture device, and talk straight to the video card. No going thru main memory. Or network packets could go straight into the processor cache.
I saw it running 6 odd real time video displays (some TV, some local camera) at a very good framerate, yet the processor was only an Arm 6.
So, to cope with modern loads (especially multimedia data types) a good architecture helps a lot. You can download the PhD thesis describing the DAN architecture from the cambridge FTP stie somewhere. Can't remember the URL off the top of my head though.
-- Michael
PS: Little factlet - the DAN prototype site right next to the (in)famous coffee pot that was the subject of one of the first web cams :-)
Re:Something faster Solid stat hard drives (Score:1)
HD's aren't the problem...(or YEA BACKUS!!!) (Score:1)
Yea Lisp, scheme, Turing, Backus....
I'm done
hard drives are the bottle neck? (Score:1)
Back to Knuth (Score:1)
One of the more insigtful comments is that we need to go back to Knuth's Fundamental Algorithms and see how he handles the memory hierarchy. In this case, tape drives. Tape drives in those days had reasonable bandwidth (for the time) and miserable latency. Sounds like hard drives now. So we need to reread the old algorithms and do a little translation:
Tape drives -> disk drives
Disk drives -> DRAM main memory
Core (RAM) memory -> 2nd level cache
Cache -> 1st level cache
This isn't simple, of course, because things have changed, amongst other things there is no detailled control over what is in which cache level, but the gist of it is that we need to drop the assumption that accessing random addresses in RAM is just as fast as accessing RAM in such a way that cacheing and readahead are possible. This is still a universal assumption in algorithms textbooks.
Similarly, we need to drop the idea that we can get to any part of the disk within a reasonable time. If we treat modern disks like a tape drive with really good mechanics for fast-forward and rewind, then we are closer to the reality, where you can transfer over a megabyte in the time it takes you to read a random bit from the other end of the disk, (about 10ms).
This implies our 'hard disks' should be RAM disks, and the current hard disks should be relegated to some sort of backup role.
The mainframe I used at Cambridge (Phoenix, a heavily hacked IBM 370 - this is only 10 years ago!) would move your files out to tape if you didn't access them for a few weeks. Next time you accessed them it took a few minutes for the tape robot to find your tape and put it back on the disks.
Just buy more RAM? (Score:1)
Hard disks are about 3 cents per megabyte [rcsystems.com].
RAM is about $1 per megabyte [rcsystems.com] And according to rumour, IBM is about to speed up the capacity race in hard disks, while memory is about to hit the quantum effects wall.
RAM isn't a real alternative to disks and if you want to use RAM to cache disks, then you need to be very careful that your access patterns are such as to let the cache have an effect.
Research should be into how we make a large amount of RAM and a humongous amount of disk space work as fast as if the disk were RAM. Perhaos ReiserFS [devlinux.org] will help Linux with this.
Incidentally, DAT tapes aren't much cheaper than hard disk space these days, though they are more portable, which is important (if you back up by copying to different disks within the same LAN then you have a problem if lightning strikes your ethernet).
Re:RAM and more RAM (Score:1)
Actually, I think hard disk capacities are pulling away from memory capacities.
After all, it's almost trivial to have as much as 1GB in your home machine right now
Yes, if you buy a new PC, but then it will probably come with a 10GB hard disk.
Argument: how about power failures?
Reply: can you say UPS?
Can you say "irritating patronising phrase"
If a new fast version of ext2fs came along that always did a reformat instead of a file system check would you use it? Even if you had a UPS? You'd be crazy. Linux may be stable, but we are not at the point where I would be willing to bet the entire contents of my harddisk that none of the drivers have a lock-up bug in them.
The solution probably is something that uses RAM to hide the slow disk speed, but it has to be more complex than your suggestion if it is going to work.
Re:They're fast already (Score:1)
I don't think that's the limitation just yet. Light can cross 1m in 3ns, that's less than the distance from my processor to it's RAM and back, yet the RAM has a latency around 100ns. And it's less than the distance to my hard disks and back, but they have a latency over a million times more than this.
Re:Multithreaded architectures... (Score:1)
In both cases the main problem is finding some other useful work for the processor to do while it is waiting for the answer. A lot of the time, the user is waiting for one particular job to finish, it isn't easily parallelisable and what the processor does while it waits for high-latency IO isn't that relevant.
On the contrary, it gets much simpler, and any decent OS will already context switch while waiting for a disk. Introducing new architectures is a difficult business. I don't expect more than two new architecture on high end workstations, they are the Transmeta processor (if that is what they are doing) and IA64. If you mean IA64 I can't see how. All those registers would make fast context switches more difficult. They have other ways of dealing with high latency memory, though.If you mean Transmeta, then I understand why you are posting as an AC :-)
If I understand correctly, the Alpha 21464 goes a different way, and puts 4 processors with fast interconnect on one chip. Again, the trick is to make your compiler so clever that it can split the job into several parallel parts.
Re:They're fast already (Score:1)
Rubbish, it's because IDE isn't terminated and isn't differential, and for all I know it isn't asynchronous. You can make Ultra-2 SCSI much much longer, even extend it to a couple of miles with fibre.
I think we need CPU architecture to build upon single simple devices that communicate in a way that is not bound by the speed of light. Once the CPUs get there, the rest will follow, as is seen to fit. Read ``Communicating sequential process'' by Hoare et.al. and think about the communication happening way above c. That's sweet :)
Either you or Hoare or Einstein has misunderstood something. I can't say for sure which of you it is...
Re:Multithreaded architectures... (Score:1)
IA64 can issue multiple bundles at once (if the stop bit isn't set)
All modern processors can issue lots of instructions at once, the only difference is that IA64 expects the compiler to do the analysis, whereas others think it is better done at run time
I was surprised to see how many similarities there were between IA64 and Tera. There are huge differences, though. IA64 has one set of registers, and I think adding 4 or 8 would be quite an overhead. I don't think making IA64 into a low-latency multithreaded architecture would be that simple, and anyway the point is moot, since AFAIK neither Intel nor HP plan to try.
If CPUs support many suspended threads, DB servers can use them for tolerating disk latency, too. Yes, it's a really long latency, but CPU-level context switching can get rid of the OS-level context switching overhead for responses.
Context switch times are measured in microseconds, while disk seek times are measured in milliseconds. There's little need to optimise the OS context switch time if you are waiting for the disk. And remember, hardware support for hundreds of Tera-style threads would cost you a lot of performance in other areas.
And DB servers already do use threads to tolerate disk latency, but that only works if you can find something else to do in the meantime, and a 10ms seek time is an age. And a user waiting for several minutes an answer that needed thousands of disk seeks to find isn't going to be comforted by knowing how many other users got their answers at the same time, especially if they are using the same disk.
Re:SCSI! ide/scsi is not the point (Score:1)
quick. scsi hardware tends to be higher speed.
however, the gap between scsi and ide is never
more than an order of magnitude even when you
consider raid solutions. ram is 5 orders of
magnitude faster than disk - scsi or ide. it's
not like doubling disk speeds would help. we
need to multiply them by 100,000 to catch up.
Re:Crystal Storage? (Score:2)
Diamonds aren't that tough to manufacture in labs, either, believe it or not (after all, diamonds are just carbon, which is plentiful; add a bit of boron and you get one hell of a semiconductor).
However, you still have one major problem: speed. Consider the following: CD/DVD has made creat strides in the past years speed-wise, but they're still much slower than a hard drive. And that's only on 2-dimensional media. Add a third dimension, and you get something which is absolutely wonderful for backups, but not for general use.
Though I'll be the first to admit, the day I can use Trek-like "isolinear chips" will be quite cool indeed...
Why are we still using mechanical heads? (Score:1)
Instead of a single head sitting on an arm that has to move back and forth to reach a particular track, why not use chip manufacturing technologies to build "head chips" which have thousands of heads, at least one per track.
Then transfers will be purely related to rotation speed.
Cheaper too.
Re:They're fast already (Score:1)
But it doesn't either on CPUs or RAM (well it's done some, but it won't go on).
The fact is, information travels thru copper with less than the speed of light, and certainly never with more than this speed. We can only increase transfer rates (by using wider busses), we can't decrease latency, using current techonology that's bound by the speed of light.
Until we start using tunneling (that some physicists seem to believe can transfer information above the speed of light) we will not scale well in latencies.
Disk latencies, being so much larger than that of RAM and CPUs, will scale for some time to come. RAM and CPUs are almost dead in the water, until we get this breakthrough.
Until then, we can only re-arrange our problems to be transfer-rate bound, more than latency-bound. That requires thinking ofcourse.
I don't know much about physics, but I try to keep up by asking the right people the right questions. And there seem to be some possibilities in either tunneling or super-conducters, both which in some cases might allow information to travel above this limit of c. Meanwhile I'll be doing what I can do, to make problems rate-bound rather than latency-bound.
Re:They're fast already (Score:1)
And light travels thru copper wiring at about 0.6c.
Why do you think your IDE cables can't be any longer than [insert spec number here. it's less than 1 meter]. It's because of 0.6c and the latency that follows. The speed of light is a real limitation even in low end devices _today_.
Disks can improve latency for quite some time to come, because they're bound by the rotational speed rather than the speed of light, so no worries there. But the rest of the machine is dead in the water in this comparison.
I think we need CPU architecture to build upon single simple devices that communicate in a way that is not bound by the speed of light. Once the CPUs get there, the rest will follow, as is seen to fit. Read ``Communicating sequential process'' by Hoare et.al. and think about the communication happening way above c. That's sweet
Re:They're fast already (Score:1)
But light of course doesn't trave thru cooper. What I mean was: Information travels thru copper at around 0.6 times the speed of light.
Re:They're fast already (Score:1)
Build your disks infinitely fast, with 1ps seek time, you'll still have more than several nanoseconds of latency.
Build the same disks, but use superconductors/tunneling for transport, and you'll be rockin'.
I believe neither I, Hoare nor Einstin misunderstood anyting in this subject. Because neigher of us really cared/cares much.
As long as information travels at a speed slower than some fixed boundary, the information latency is going to become a bottleneck at some point in time.
Still, to get back to the original point of this discussion, disks have a long way to go, and CPUs, RAM etc. doesn't.
Expect superconducting/tunneling to become big, or expect disks to scale much better than CPUs/RAM/etc. in the future.
Or, perhaps you have some theory that I'm not aware of ? I'd appreciate to know about it.
Re:They're fast already (Score:1)
As was pointed out in another post, another solution might be (temporarily) to decrease the length of the wiring. Still, in the long run, we need communication faster than the speed of light.
They're fast already (Score:2)
I can transfer 34 MB/s from my disks (sustained), and around 100 MB/s from my main memory (RAM). As I see it, disks are not the problem here.
RAM today does some sort of parallel-like access to get the performance they get today. If you do the same with you disks, (and why wouldn't you, if performance is that important?) you get equal speedup. Also put more CPUs in your box, when the CPUs become the bottleneck.
Today in PCs, the PCI bus is the only real bottleneck, perhaps along with the memory subsystem. The memory subsystem has improved a lot over the last few years, but the majority of PCs still come with 32 bit 33MHz PCI busses.
When we start seeing motherboards with 64 bit 66 MHz PCI, that's when we'll again be looking at disks, cpus and memory as the bottleneck.
So good you asked (Score:2)
Actually you can even buy Solid-state drives today, but it costs much, isn't big enough for your bucks...
Can anyone throw in a few URLs?
Re:real-world comparison of access times (Score:1)
Well, you're never going to send one forklift to the long-term facility. You're going to send a convoy of trucks.
More than anything, my point is that "only a car is like a car". Can't we just evaluate computer technology using its own terms? Bringing forklifts into the discussion won't help much, I think.
--
Re:real-world comparison of access times (Score:1)
> More than anything, my point is that "only a car is like a car". Can't we just evaluate computer technology using its own terms? Bringing forklifts into the discussion won't help much, I think.
I think his point is not about forklifts, it is about time, and the difference between two minutes and 4 years.
But then, it does not take four years to get data off your drive, and as he pointed out, the cpu can keep working on something else while it waits those four years.
Cheers
Eric Geyer
Has anyone tried Ultra DMA/EIDE Raid Controller? (Score:1)
The price is good, about $125 (US), and four Ultra DMA drives to fill this up would be much, much less expensive than four equivalent SCSI drives. Granted, an uIDE Raid array will be different than SCSI-based array, but it should be close enough for a meaningful compairison, I think.
Has anyone used uDAM Raids? Heard Any reviews?
Re:Has anyone tried Ultra DMA/EIDE Raid Controller (Score:1)
Re:Has anyone tried Ultra DMA/EIDE Raid Controller (Score:1)
Organic Storage a reality? Opticom claims so (Score:1)
They recently presented an operative one-gigabyte ROM version of their product to the public, and from what I understand, they're currently working to commercialize this technology. They also claim to be able to produce a working RAM version of their product in less than a year. This article [yahoo.no] (in Norwegian, I'm afraid) sums up their product and marketing ambitions for next year.
I'm not qualified to judge neither the quality nor the feasability of their technology, but they've been highly debated in Norway. The debate has apparently subsided somewhat now when they have working prototypes that may indicate they're moving in the right direction. If somebody knowledgeable out there finds their theories worthy a comment, I'd really like to hear about it!
Re:Head-per-track (Score:1)
And despite what you said, you would have to spin head-per-track. Maybe you're thinking of the obvious extension of head-per-bit, but without motion there is no signal for a magnetic change detector.
More to the point, if you were going to make a head per bit, just use a memory chip: if you're already going to have O(one piece of circuitry) per bit, just make it a flip-flop or a capacitor instead of a magnetic reader/writer.
Actually, I have been thinking that having maybe two or three arm assemblies, instead of just one, would be a good idea: they could each be assigned to a smaller group of cylinders, reducing seeks, or all cover all cylinders but be arranged around the surface to reduce latency, or some combination of these. It would be expensive, and it would make the electronics in the controller a lot more expensive, but that's what these new faster ICs are for, right?
I remember the other day someone was complaining that the drive manufacturers keep concentrating on making the drives bigger instead of making them faster, which is the opposite of what is needed for big RAIDs. I guess the problem is that capacity looks more impressive, and the extra expense for a faster drive of the same size wouldn't impress most normal people, so it wouldn't pay off so well.
David Gould
Re:Display resolutions, too. (Score:1)
I guess I don't quite understand your point about display resolutions. Why do they need to increase? Aren't they dependent on the size of your monitor and the amount of memory on your graphics card? Who would want to view 3200x2400 on even the largest monitor? Talk about eye strain.
Steve
Re:Protein-based storage systems (Score:1)
Re:Price is key! (Score:1)
Stan "Myconid" Brinkerhoff
faster reading speed. (Score:1)
Fuel injection not new (Score:1)
('course Diesels have always been fuel injection)
No progress in Drive speed? Sure is! (Score:1)
My new Quantum does 15Mb per second. That's a factor of 30. Moore's law predicts a factor of 30 every 10 years. (= A factor of 1000 every 20 years, = a factor of 5 every 5 years).
How long ago did I have my first hard drive? '87: 12 years go.
I'd say that harddisks follow Moore's law pretty closely.
Indeed, processors seem to have outperformed moore's law a bit the last decennium, so indeed the gap has increased a bit. (Oh well a factor of 20...
Does anybody remember those PC indexes that measured how much your PC outperformed a 4.77Mhz IBM PC? What would a Celeron/400 score now?
Is that realistic?
Roger.
Gotta be a hoax... (Score:2)
with the assistance of "Tata Industrial Semiconductors" (name changed for security purposes), a Taiwan based (location changed for security purposes) silicon foundry.
"new Electron Trap design..."
"We may just NOT sell it to the top 10 PC Companies who monopolize our industry."
and my favorite:
"We have no idea where the drawings from which we derived our TCAP came from."
Umm.. aliens, maybe? And put it in a PII shell? Come on.... The whole thing just sounds ludicrous. :-)
More info. (Score:3)
Here's a guy who has some info on their claims, mostly debunking them. http://www.uni-weimar.de/~kuehnel/TrueorNot.htm
Re:SCSI! (Score:1)
Re:Another Layer? More RAM! (Score:1)
Holographic storage (Score:1)
It's in the labs, but who knows when it'll be available....
--
Re:SCSI! (Score:1)
Andrew N.
(I probably own Apple money for saying Firewire)
--
Raided Firewire (Score:1)
--
Re:RAM and more RAM (Score:1)
Argument: how about power failures?
Reply: can you say UPS?
Static RAM. NVRAM.
Re:Price is key! (Score:1)
aren't trying for a high-performance computer,
which is what this discussion is about.
Re:SCSI! ide/scsi is not the point (Score:1)
matguy
Net. Admin.
Re:SCSI! ide/scsi is not the point (Score:1)
matguy
Net. Admin.
http://www.proteinmemory.com (Score:1)
Protein-based storage systems (Score:2)
Physical solution: Integrated "RAID 1" (Score:1)
This is of course not much help for small files, but for larger...
Even if the seek time is not much improved, the total read/write time should be at least a half magnitude faster. Think of bigger programs, graphical files a s o.
Hard drives have advanced (Score:1)
infrastress (Score:4)
Better to remain silent and be thought a fool... (Score:1)
RAID 1 is mirroring and therefore gives no performance advantage. In fact, there is a slight performance hit for the overhead of writing the same data twice, once for each disk.
RAID 5 is a good compromise between data integrity and speed. You should really go for at least 4 disks and more if you can afford it as the % of space lost to parity information goes down as you increase the number of drives.
Re:Better to remain silent and be thought a fool.. (Score:1)
RAID 0/1 requires 4 disks and it is configured with 1 pair of disks (the RAID 1 part) mirroring a striped pair of disks (the RAID 0 part).
RAID 0 is just a striped pair. The data is spread out over two disks so that reading and writing data can be distributed over both disks.
RAID 5 is the addition of parity information to a striped set. The data and parity info is spread out over all the drives with the parity info never residing on the same drive as any other part of the file.
Look at the following diagram (fig 1.1)
drive #1 | drive #2 | drive #3
-------------------------------
file 1.1 | file 1.2 | file1.p (parity information)
file 2.p | file 2.1 | file 2.2
file 3.2 | file 3.p | file 3.1
If you go to read file 1, the RAID controller will read part 1 from drive #1 and part 2 from drive #2 simultaneously. Read performance from RAID 5 is essentially the same as RAID 0.
If you want to write file 1, the RAID controller will write part 1 to drive #1, part 2 to drive #2 and the parity information to drive #3. This is slower than RAID 0 because of the parity information but still faster that RAID 1 because you are not writing the entire file to two places. You are splitting up the write duties to 2 of the drives and taking some overhead time to calculate and write the parity information.
Read a little bit more about RAID. It can be complicated but it'll be nice to know. Good luck.
Re:Better to remain silent and be thought a fool.. (Score:1)
Re:Or..pot, meet kettle.. (Score:1)
You are correct. I wasn't trying to write a complete discourse on RAID, just point out details relevant to the original post. My mistake for intimating that it was only striping.
You will have to re-read my post as none of what you said contradicts me. I made the point that, with a hot spare, not until the 3rd disk fails will you be unable to use the system. Just a matter of how you phrase it, but the point is, you can lose 2 disks and still function. Granted, the performance is degraded by using the parity information.
RAID 5, according to everything I have been able do read on the subject, does give you increased performance over RAID 1, especially when reading data. Essentially, it is like reading from a RAID 0 striped set. Writing data does take some more time due to the parity bits as you mentioned. I believe it was pointed out elsewhere that file servers are perfect for RAID 5 due the the increase in performance coupled with the data redundancy but Database servers would not function very well with RAID 5. I was merely contradicting the previous statement that a file server should use RAID 5 because of the lack of performance. To repeat another poster, if money wasn't an issue, we'd all have RAID 0/1.
Two simple solutions, cheap, available quickly. (Score:1)
Instead of one carrier arm with read/write heads, why not just two, three, or four with a simple multiplexing controller?
The positioning commands would be synchronized, while data bits would be fed in alternate, optimally sized data packets to the heads.
And if they're not already, the heads themselves, on alternate sides of the platters should be running with a split data stream, too. I think that they're not now, otherwise, multiple platter drives would have a much higher sustained data transfer rate than single platter drives.
The old way of thinking has the formatting tracks, as viewed from above with theoretical x-ray vision, laid directly above and below each other on opposite faces of the drive. This is probably thought of as a challenge to maintain, but this is completely unnecessary.
One set of tracks will be offset by a slight amount under my scheme, but that would be completely harmless, as the second head would always be fed and read by the controller with the exact same delay that was incurred by the drive while formatting. Thus the data remains in sync automatically, althought the formatting tracks are offset.
Simple, cheap, effective, and very quick to develop and implement. If the controller were 100% efficient and the data chunks even, dual multiplexed controller arms would double the performance (and half the access time), and multiplexed heads would then again double the data transfer rate. Ta-da! 3ms or less access, and a theoretical potential to quadruple the sustained data transfer rate. All without increasing the platter speed any more.
With further research, a bit of time, and some new logic circuitry, I'm sure that the drive head carrier arms could later be ran asynchronously, therefore reducing even further the random access time.
All rights reserved, manufacturers are invited to contact me at mturley@mindspring.com for reasonable licensing terms.
Re:Two simple solutions, cheap, available quickly. (Score:1)
I have taken apart several drives, and know that what I'm proposing is entirely possible with few changes to the current technologies in today's hard drives.
Maybe you haven't been thinking about this long enough, or perhaps you're locked into the same "that's how it is" pattern that HD engineers are in now. There is more than one way to fit multiple actuator arms on a drive platter.
This first is obvious. Mount the arms in opposite corners of the drive chassis. This would increase the length of the chassis a little bit, but if Quantum has no trouble selling 5.25" "Bigfoot" drives that don't perform any better than standard 3.5" drives, I don't see how they could have trouble selling 3.5" drives that are an inch longer if they perform significantly better.
Keep in mind that the drive actuator has a fixed pivot point, which only allows the heads to travel in an arc across the surface. This arc can only sweep across a fixed portion of the total surface area. No drive head carrier arm known to me could ever sweep even close to 180 degrees of the 360 degree circle, always leaving room for at least one additional actuator arm. In the drives I've disassembled, there is more than enough unswept disk surface space to fit at least two more arms. With a synchronous controller, and/or arms made with an arc cut out of their inner span, there is enough platter space for at least four arms, perhaps as many as six, without a redesign of current platters.
The second solution is just as simple, but would decrease the storage capacity of the drive. I see it as just another tradeoff to be faced in industrial design. Significantly increase the spindle diameter of the drive. This would allow multiple shorter arms without the potential for them to knock into each other.
Sketch it out on paper with a compass. You'll see that what I'm proposing makes perfect sense.
Further, regarding fragmentation, if the drive arms are ran from a synchronous controller, and the heads are fed a data stream that is composed of striped bits of the data being written, fragmentation is no more of an issue than it already is. (Think a moment about a two armed drive. The arms are synchronized, so when head 0 on arm 0 is at point A, head 0 on arm 1 is always exactly 180 degrees away, at the exact same distance from the spindle. One file divided into two data streams gets written simultaneously in two locations, just as it gets read back later. The file is intentionally fragmented into two equal parts, but because these parts are always accessed simultaneously, no detrimental fragmentation is induced.)
I cannot think of any other computer subsystem that is so ready, so ripe for parallelization as the hard drives, yet no one seems to be thinking of parallel computing here.
As stated before, all rights reserved, and manufacturers are still invited to contact me at mturley@mindspring.com for reasonable licensing terms for use of these ideas.
Re:They're fast already (Score:1)
Re:SCSI! (Score:1)
They physcial side of SCSI is exactly the same as with IDE: new stuff is put into higher end drives first, and higher end drives have SCSI interfases.
The reason why SCSI drivers are at 10k and IDE is at 7200 is because people who want/need/can afford 10K drives have SCSI, anyway.
Re:In 1970??? I don't think so (Score:1)
How old are you? I suppose you think everything before 1980 was in black and white, etc. etc.
Consciousness is not what it thinks it is
Thought exists only as an abstraction
I dispute the premises. (Score:1)
I lack the hard (no pun intended) data to back this up, but I think that hard-drive raw transfer rates have been rising and access times have been falling. I will agree that they haven't gotten better anywhere near as quickly as other parts of a computer system though.
That being said, the only thing I can see as being an eventual competitor to hard-drives is the drawing board idea of holographic storage. Whether or not any products based on it ever make it to market is anyone's guess.
Hard Drives (Score:1)
Re:Or..pot, meet kettle.. (Score:1)
Couple of corrections:
1) RAID 0 can either be striping, doing writes a chunk at a time to each disk in the stripe set, or concatenation, which is slapping multiple disks together to make one big virtual disk and then writing so you fill up the disks in sequence.
2) RAID 5 can only tolerate losing 1 disk in its set. The volume can continue in "degraded" mode where for every read off of the volume, the system has to calculate the parity bit to make up for the lost disk. If you throw in one hot spare you could lose 2 disks total. (The hot spare takes over for the 1st disk to go..the volume needs a period of time to sync the data/parity to it, but then continues normally. You can then lose 1 more disk..this will put you in "degraded" mode.)
3) I'm not sure where you've gotten the impression that RAID 5 is excellent for performance. Calculating parity bits for *every* read is the antithesis of performance. RAID 5 (or "poor man's mirroring") is good for systems that need the data redundancy but cannot afford the cost of doubling their disk usage for mirroring. However, no compentent SysAdmin would use the terms "performance" and "RAID 5" in the same sentence.
Re:So good you asked (Score:1)
Re:Agreed. Programs get smaller (Score:1)
A nice laptop with 64 magabytes of memory, and half a gig of solid state disk would be enough. Espsically, if the VM scheme just mapped programs to the memory. And need a game, pop in a 64 meg solid state floppy with the game on it.
Now all we need is for reasonalbly prices flashdisk memory.
Re:FireWire (Score:1)
Re:Two simple solutions, cheap, available quickly. (Score:1)
Re:how about this... (Score:1)
Make drives fast (Score:1)
"Faster hard drives" misses the point. (Score:1)
IBM's hype on Ferro-magnetic (-electric?) memory (Score:1)
they've developed the "holy grail" of memory:
SRAM speed (10ns access)
DRAM density
nonvolatility
If they realize the promise of this type of memory, many computers may not need hard disks at all in 10 years, when 4Gb chips are being manufactured.
6ns is NOT a DRAM access time (Score:1)
to get stuff off of the chip once it's asked for
at the chip.
In reality, it takes more than 120ns to do this:
1) Miss the cache
2) Ask for the data on the bus
3) Get the data off of the chip
4) Ship the data back to the CPU
5) stick it in a register
6ns is just the clock period of the memory bus
on the very fastest SDRAM interfaces. 133MHz
memory does NOT imply a 6ns access time: it
means that you can put data across it every
6ns.
indium phosphide (Score:1)
Gallium arsenide? I don't think so.. maybe something like indium phosphide.
There was an interesting piece recently on sciencedaily [sciencedaily.com] about some research being done on electro-optical effects in photo-reactive crystals.
If you like that link, Salamo has a (very brief) page [uark.edu] describing some of his other work as well.
Re:SCSI! (Score:2)
faster than IDE drives in their media transfer rate and average seek times. Also, note that the top-end scsi drives now spin at 10Krpm, while IDE is only now getting to 7200. With ATA/66, the interface speed isn't actually all that different.
There is also the point that SCSI can generally queue commands to multiple devices, while ATA is just getting into that.
Re:SCSI! (Score:2)
Of course, (as mentioned elsewhere on this thread) the point is the actual storage speed, not the interface speed.
-Chris
Mechanical Parts (Score:2)
The questions of course are: reliability, performance, and durability. Even using old 30-pin simms in huge arrays attached to a constant power source would probably be faster (overall, not bursting, unless there was a really nice cache on there) than =5400rpm drives, but how long is that going to last? And if you build a twenty-meg portable "disk" out of this, what happens when you drop it on the floor?
Oh well. I think I'll go drool about that atlas 10K now...
-Chris
Crystal Storage? (Score:2)
As a second note, with faster/cheaper/denser ram comming all the time, ram based static hard drives might come to replay magnetic media. (might) I know I want one.
-Crutcher
Leave the mechanichs behind (Score:2)
If we want to get out of the slow storage days, solid-state mass storage devices will have to become practical for everyone. Or some of the other fancy stuff that only exists in labs right now will have to become a parketplace reality. Hard disks as we know them appear to be coming to the end of their evolution, at least in terms of speed.
Holographic Data Storage (Score:5)
There's ongoing research ( http://nanonet.rice.edu/research/boy d_res.html [rice.edu]) to use photopolymers as a cheaper holographic medium. If such research comes to fruition, you're more likely to see CD like disks coated w/ a holographic layer than the typical science fiction "data crystal."
Other problems w/ holograms:
- materials are not totally transparent, so "cubes" might be out of the question
-materials must be chemically resistant to the atmosphere (e.g. oxidation, humidity), which might necessitate that they are coated. Such a coating might have deleterious effects on the substrates optical properties.
- storing a hologram changes the structure of the crystal, which can cause limits of data density and beam penetration.
- multiple holograms can be stored at the same location by rotating the crystal, but each hologram attenuates the possible intensity of subsequent holograms in that location.
- holographic "efficency" is a funciton of the difference between the refractive indices of the substrate components. photopolymers have a very small range of refractive indices as oppossed to inorganic crystals.
Overall the medium might not be rewritable, but a high density, long lasting storage medium would be ideal for back-ups.
Anyway, it's been awhile since I "got out of the business" of chemistry, but this is what I remember.
In 1970??? I don't think so (Score:2)
Hard Drives have come a long was since 1970. Now CARS. CARS have gone nowhere since 1970.
Re:Crystal Storage? (Score:2)
Anyways, i'd really like to see some new storage medium, using magnetic storage in the decade of the CD just seems wrong somehow. I remember a short segment on the news a while back (like 4-6 months) about some college lab developing a new storage medium using a silicon needle a few atoms wide to read and write on these little penny-sized wafers that was supposed to hold 100 times as much as a CD and be available in wristwatch-sized readers for music in like 5 years.. but havent heard a Word since then. Perhaps somebody wants to post some URLs for the more interesting data storage projects that never made it? (or are still struggling toward making it)
Dreamweaver
Found the holographic storage link! (Score:2)
I read the above link about two years ago, and was amazed by the technology. Quarter-sized 100TB holographic discs. They're supposed to have something viable in another year; here's hoping!
Cache: a necessary evil; RAID: a godsend... (Score:2)
Sure, it would be ideal for all storage to be as fast as the processor, but the current cached model works pretty well provided you have enough rams and a OS that makes good use of that ram...
If you really need more performance, use RAID...
www.cdrom.com doesn't seem to be disk limited, that's because it has boku memory and raid arrays.
Besides, hard drives today are about 10x as fast (10x more throughput, initial latency has been cut about 4x) as those of 10 years ago, which isn't too bad in machanical engineering terms...
Something faster Solid stat hard drives (Score:2)
http://www.accpc.com/tcapstore.htm