Forgot your password?
typodupeerror
Data Storage The Internet

Mailing Disks is Faster than Uploading Data 581

Posted by CowboyNeal
from the data-access-versus-data-excess dept.
CowboyRobot writes "Who would ever, in this time of the greatest interconnectivity in human history, go back to shipping bytes around via snail mail as a preferred means of data transfer? Jim Gray would do it, that's who. And we're not just talking about Zip disks, no sir. We're talking about shipping entire hard drives, or even complete computer systems, packed full of disks. David Patterson (one of the developers of both RISC and RAID) interviews ACM Turing Award winner Jim Gray." Back in school we always had a saying, "Never underestimate the bandwidth of a station wagon filled with backup tapes." Seems like that still holds true.
This discussion has been archived. No new comments can be posted.

Mailing Disks is Faster than Uploading Data

Comments Filter:
  • Tapes too... (Score:5, Informative)

    by inertia@yahoo.com (156602) * on Thursday July 10, 2003 @08:27PM (#6411320) Homepage Journal
    This reminds me of how data is collected for SETI@Home:

    After the data is recorded onto tapes at Arecibo, they are shipped back to the SETI@home lab in Berkeley, California. The data are then broken up into workunits, which are sent out to the client screensaver program for candidate signal detection. So far, SETI@home has generated 189,598,882 workunits from the data received from Arecibo. SETI@home has split 1,139 tapes, meaning that the average tape yields 166,709 workunits. This is somewhat lower than the optimal yield of roughly 200,000 workunits per tape because of radio frequency interference, gaps in recording, problems with the recording equipment, etc.

    I think a work unit is 65,536 bytes. Even if it takes a week to ship one tape, you can't beat that throughput! But the latency is the worst.
    • Re:Tapes too... (Score:4, Informative)

      by Rura Penthe (154319) on Thursday July 10, 2003 @08:30PM (#6411338)
      Well one tape = 166709 units * 64 (k) / 1024 / 1024 = ~10.175GB. 10.175GB a week is not particularly impressive. :)
      • Re:Tapes too... (Score:5, Informative)

        by pixelite (20946) on Thursday July 10, 2003 @08:41PM (#6411423) Journal
        Well one tape = 166709 units * 64 (k) / 1024 / 1024 = ~10.175GB.


        That figure is per tape, the actual shipment has 1,139 tapes, I think. 10.175GB * 1,139 = ~11.6TB. That *is* impressive bandwith.
        • yeah, but the ping sucks...
        • Re:Tapes too... (Score:5, Insightful)

          by pclminion (145572) on Thursday July 10, 2003 @09:08PM (#6411584)
          That figure is per tape, the actual shipment has 1,139 tapes, I think. 10.175GB * 1,139 = ~11.6TB. That *is* impressive bandwith.

          Well in theory, that's not really "bandwidth," it's just a number of bytes. The bandwidth would be the maximum sustained throughput. Essentially, how much data could be delivered per second, if there were a constant stream of trucks pulling in, each carrying 11.6TB. Assume the trucks drive bumper-to-bumper, at 60 MPH. Assume each truck is what, 25' long. At 60 MPH it takes about 0.28 second to travel one truck-length. Therefore, the actual bandwidth is 11.8/0.28 = 42.1 TB per second.

          • by autopr0n (534291) on Thursday July 10, 2003 @09:47PM (#6411742) Homepage Journal
            First of all, how could you drive into a garage at 60 miles an hour, stop, unload, and get back out before the next truck, only four inches behind you comes in? In your system the trucks would crash into eachother, and you'd get no effective bandwidth.

            In actuality, you need to figure how long it takes to unload all the tapes from the truck at least. Assuming about 10 seconds a tape (ones in the back take longer) on average and that's 1.16gb/second.
            • by pclminion (145572) on Thursday July 10, 2003 @09:57PM (#6411810)
              Well, I was assuming the trucks would drive into one of about 50 million different parallel unloading docks, but you know, I was just estimating...
              • hm... (Score:4, Informative)

                by autopr0n (534291) on Thursday July 10, 2003 @10:03PM (#6411847) Homepage Journal
                But they could also take 50 different routs to get there. There are all different kinds of ways to rejigger the figures, but were talking about what's practically possible. Employing enough people to man those 40,678 loading docs full time (what you would need to offload 1 truck/ 0.28 seconds), would be at least 5.55*40,672*24 is about $1,083,667 dollars per day, or almost $400 million a year. For that kind of money you could probably afford to lay down multiple parallel multifrequency optical cables.
              • This is the dorkiest conversation I have ever heard in my life.
            • by sn00ker (172521) on Thursday July 10, 2003 @09:59PM (#6411821) Homepage
              Containerise them. Lift the entire container off the back of the truck in one hit and off the truck goes again. You could cut unload time down to a couple of minutes.
              Reminds me of the supermarket distribution plants. Trucks are allocated time slots that are 10 minutes long, and the trucks must arrive exactly on time or they miss out. One forklift unloads the entire truck, and another shifts the pallets into the shelving system. The one nearest me has 20 bays and runs 24x7.
              Never underestimate the ability of a logistics facility to chew through trucked goods.
          • "Assume the trucks drive bumper-to-bumper, at 60 MPH"

            Where do you live, so that I can avoid the area?
    • Re:Tapes too... (Score:5, Interesting)

      by CptChipJew (301983) * <michaelmiller AT gmail DOT com> on Thursday July 10, 2003 @09:23PM (#6411653) Homepage Journal
      Work units are ~300 kilobytes a piece, or at least thats how much the client downloads.
    • Datacom 101 (Score:5, Interesting)

      by EmbeddedJanitor (597831) on Thursday July 10, 2003 @10:23PM (#6411942)
      This reminds me of a question on a 1980s data comms paper.

      Q:A man with a delivery bike can pedal at 20mph between the organisations two offices that are 5 miles apart. The basket on the bike can carry five half-inch tape reels. What is the effective throughput of this datalink? For extra credits: A modem can transfer data at 300bps. At what distance does this outperform person with bike.

    • Re:Tapes too... (Score:3, Interesting)

      by ADOT Troll (687975)
      I am a mainframe console operator at a large computer datacenter, with several state government agencies outsourcing their mainframe processing to our center. We have every day, UPS and Fedex shipments of 3480- and 3490-format tapes (look like 8-track audio tapes) to load into the mainframe, even some old arse 3420 tapes (the big magnetic reel tapes)

      some of the files on these tapes are litereally only a few kilobytes large.. (omfg wtf lol!)

      certainly is NOT faster than ftp'ing the data over, considering th
  • Storage grows probably more quickly than bandwidth.
  • by Anonymous Coward on Thursday July 10, 2003 @08:29PM (#6411329)
    "Never underestimate the bandwidth of a station wagon filled with backup tapes."

    "Hurtling station wagon", "8-track tapes".
  • by theGreater (596196) on Thursday July 10, 2003 @08:29PM (#6411331) Homepage
    Darn you people! How the heck am I supposed to get a proper astrophysical mental image if you consistently refuse to put things in terms of multiples of VW bugs (the old ones, not the faux ones).

    -theGreater
  • Netflix (Score:5, Interesting)

    by geekee (591277) on Thursday July 10, 2003 @08:30PM (#6411341)
    Netflix has made a business out of shipping data via snail mail, since the bandwidth isn't really there yet to do it over the internet.
    • Re:Netflix (Score:4, Interesting)

      by G27 Radio (78394) on Thursday July 10, 2003 @08:43PM (#6411442)
      I think it's more a matter of them not wanting to get sued into oblivion by the MPAA. With broadband and DivX, downloading the movies is relatively fast and easy. They'd make a killing if they were able to make movies cheaply available online.
    • by twitter (104583) on Thursday July 10, 2003 @09:00PM (#6411535) Homepage Journal
      Netflix has made a business out of shipping data via snail mail, since the bandwidth isn't really there yet to do it over the internet.

      What a great example you picked! Cable TV companies are pumping dozens of digital movies accross their system at once, live. Yet they crimp your upload speed to DSL rates or lower, 30KB/s, because they are afraid of people "stealing" movies. This is not a technological problem, it a social one. Big publishers and telcos are afraid of competition and are doing everything in their power to keep you from enjoying technology that's already in place. It's the same old fight Ma Bell used to wage back when they would not alow you to so much as plug a modem into your phoneline.

      How long are people here in the US going to put up with this monkey business?

      • Cable TV companies are pumping dozens of digital movies accross their system at once, live.

        Um, no. They're broadcasting one movie at a time. You're not receiving all channels at once. That makes a huge difference in your argument.

        • Hey guess what (Score:3, Informative)

          by autopr0n (534291)
          you are reciving all the channels at once, it's just that you're only decoding one at once. Lots of people decode more then one at once, such as using the TiVO or a VCR, or using picture in picture.

          If you wanted too, you could record all of them at once, quite easily.
      • Here in NYC, Time Warner now allows us to pick from several dozen movies to be played at any time, including the ability to pause, FF and REW (with preview), etc. (video on demand). All of this at close to DVD quality too.

        So how do they do this? I've always been under the impression that with digital cable and cable internet, all of the data has to be sent to everyone (in the same neighborhood anyway), so how can they handle the hundereds of channels (some of which are actually lower quality than others),
        • by zenyu (248067) on Friday July 11, 2003 @12:15AM (#6412501)
          So how do they do this? I've always been under the impression that with digital cable and cable internet, all of the data has to be sent to everyone (in the same neighborhood anyway), so how can they handle the hundereds of channels (some of which are actually lower quality than others), the multiple VOD streams (even for the same movie), and eveyone's porn and mp3 activities all at the same time?

          This one is simple they ran fiber to the curb a few years ago. They even ran new coax to our apartments to handle more bandwidth. There is effectively infinite bandwidth running into your apartment.
      • by Slurpee (4012) on Friday July 11, 2003 @01:22AM (#6412720) Homepage Journal

        What a great example you picked! Cable TV companies are pumping dozens of digital movies accross their system at once, live. Yet they crimp your upload speed to DSL rates or lower,


        very wrong.

        But enough truth to fool people into believing what you said.

        You are correct in saying that a digital cable system pumps out lots of bandwidth. They do. A movie chan is generally about 4mb/s, possible 8. A chan such as the shopping chan may be 1mb/s. So your cable company with 100 chans is pumping out approx 400mb/s.

        Thats a lot of data.

        But it is broadcast. Each customer is not individually downloading 400mb/s each. They are sharing *one* broadcast. It is not one stream per customer, but one stream is shared between all customers.

        To use a cable for internet, assuming no TV is being broadcast, you can share that 400mb/s between all your users. Customers will have 4kb/s (thats kilobits) EACH (assuming its all shared equally). Not huge.

        Obviously this is not the whole story. Your bandwith is shared between all customers on a node of the cable network (think of them as hubs). If you are the only person in your node, you will get full bandwidth. A node could cover tens, if not hundreds of thousands of users. If every person on your node is using the net to download porn, you will have a very slow connection (better using a modem). Also, the cable company wants to not just do internet, but TV too! In fact, most of the bandwith is used with TV/Movies.

        So, they end up using part of their bandwith for internet, and part for broadcasting TVs.

        How much they set aside for each is a buisness decision, as well as a technology one. If they sell cable internet, the costs are huge, setup, support, network, etc. Costs go up *per user*. Costs for TV is small (ish). Pay for content (movies), get money in from advertising, users, etc etc. No big support costs, no extra costs for bandwith etc etc. One stream can support hundreds of thousands of users.

        It is both a technological problem *and* a buisness problem. They aren't giving you small limits cause they are afraid you will download videos. Don't be paranoid. They don't give you unlimited bandwith cause they can't, and it costs them a lot anyway.

    • Re:Netflix (Score:5, Funny)

      by appcoal (688275) on Thursday July 10, 2003 @09:25PM (#6411659)
      Not for very much longer. The RIAA's lawyers have just discovered that the U.S. Postal Service is in fact a p2p network. (Not to mention the highway system.)
  • by Capital_Z (682911) on Thursday July 10, 2003 @08:31PM (#6411349)
    We had a saying back in school too --

    "If you're driving a station wagon around you ain't doin' too well with the ladies"

  • by Ribo99 (71160) on Thursday July 10, 2003 @08:32PM (#6411354) Homepage Journal
    Of course playing Quake would be out of the question I would think

  • by gotr00t (563828) on Thursday July 10, 2003 @08:32PM (#6411355) Journal
    Though it is true that a box filled with hard disks that is snail mailed has a higher rate of transfer than actually uploading the contents of all those hard disks, there are problems with this argument as well.

    First of all, when downloading, you have the benefit of instantly recieving the file that you need, as opposed to waiting at least a day for your shipment to arrive.

    Secondly, remember that bandwidth is probably cheaper than postage. Shipping a carton with a few hard disks and proper insulation would cost at least $30 to overnight it.

    Really, the title of the article comes upon the conclusion way too quickily. You must consider much bandwidth the sender and the reciever have. If both have a several gigabit OC line, then perhaps uploading it would be faster.

    • by tarius8105 (683929) on Thursday July 10, 2003 @08:50PM (#6411474)
      First of all, when downloading, you have the benefit of instantly recieving the file that you need, as opposed to waiting at least a day for your shipment to arrive.

      The average home user still uses a 56k modem, I dont see how it would be faster to transfer a gig on a 56k then priority overnight (much less then 160 gigs).

      Secondly, remember that bandwidth is probably cheaper than postage. Shipping a carton with a few hard disks and proper insulation would cost at least $30 to overnight it.

      Depends on how you ship and what you ship and the connection the person has.

      Really, the title of the article comes upon the conclusion way too quickily. You must consider much bandwidth the sender and the reciever have. If both have a several gigabit OC line, then perhaps uploading it would be faster.

      Not all companies can afford an OC line thus shipping would be cheaper.
    • by captain_craptacular (580116) on Thursday July 10, 2003 @08:51PM (#6411482)
      Lets see, local cable modem is $39.95 for 5 gigs and $10 a gig past that. So if you can ship 3 160GB HDD's for $30 thats:

      160GBX3 = 480GB / $30 = 16 GB/Dollar

      Cable modem = 1GB/$10 = 1/10 GB/Dollar

      So the mail is cheaper. And probably faster if you consider how long it would take to DL 480GB @ 32KB/sec compared to next day or 2nd day air.
    • I was going to mod the parent "overrated", but it wasn't worth the points, so I'll argue it here.

      The poster just didn't read the article.

      First, he naively says that the file is there "instantly" if you transmit it. That's not true for big files, which will take size/bandwidth to arrive. It does you no good to get the first file if you need all of them anyway.

      Second, the bandwidth is NOT cheaper than the postage. That's one of the main points. A gigabit OC line costs significant money, and even i

    • Secondly, remember that bandwidth is probably cheaper than postage. Shipping a carton with a few hard disks and proper insulation would cost at least $30 to overnight it.

      Really, the title of the article comes upon the conclusion way too quickily. You must consider much bandwidth the sender and the reciever have. If both have a several gigabit OC line, then perhaps uploading it would be faster.


      (Article is old news. Film at 11.)

      I can assure you that it is cheaper still to ship a box full of 10G DAT's, let
  • by privaria (583781) on Thursday July 10, 2003 @08:32PM (#6411357) Homepage
    PING privaria.org (64.33.49.48) 56(84) bytes of data.
    64 bytes from privaria.org (64.33.49.48): icmp_seq=2 ttl=242 time=2 days, 7 hrs, 37 min
    64 bytes from privaria.org (64.33.49.48): icmp_seq=1 ttl=242 time=2 days, 17 hrs, 14 min
    64 bytes from privaria.org (64.33.49.48): icmp_seq=3 ttl=242 time=3 days, 2 hrs, 41 min
    • New RFC! (Score:3, Funny)

      by twoslice (457793)
      Actually the ping will never traverse the same network. The data will go over the postal network and a single ACK will come back over the telephone network. Data loss will occur at unacceptable rates due to the medium (postal service) and ACK loss will be high too (Telco)

      New proposed RFC 5433 for postal ACK format:

      "Yup, I received your package"
    • by taniwha (70410) on Thursday July 10, 2003 @09:04PM (#6411557) Homepage Journal
      uucp to Australia used to be done by uploading a spool dir somewhere in the US to a tape and airfraighting it to Oz, then doing the same at the other end. You'd post something to usenet and get a reply 2 weeks later
    • PING privaria.org (64.33.49.48) 56(84) bytes of data. 64 bytes from privaria.org (64.33.49.48): icmp_seq=2 ttl=242 time=2 days, 7 hrs, 37 min 64 bytes from privaria.org (64.33.49.48): icmp_seq=1 ttl=242 time=2 days, 17 hrs, 14 min 64 bytes from privaria.org (64.33.49.48): icmp_seq=3 ttl=242 time=3 days, 2 hrs, 41 min

      traceroute privaria.org

      1 privaria.org-package.ready 50000ms
      2 Picked-up-USPS 900000ms
      3 Transfer-to-USPS-depot 300000000ms
      4 (unknown)
      5 (unknown)
      6 (unknown)

      Packet Loss 100%

      Blockwars [blockwars.com]: multiplayer and it's free.

  • by grub (11606) <slashdot@grub.net> on Thursday July 10, 2003 @08:32PM (#6411358) Homepage Journal

    Of course, we could put the Library of Congress holdings on it or 10,000 movies

    10,000 movies? The MPAA would like to have a word with him..
  • Ping (Score:5, Funny)

    by felonious (636719) on Thursday July 10, 2003 @08:32PM (#6411360) Journal
    The ping on a station wagon sucks and don't even get me started on the routes...
  • I don't know (Score:5, Interesting)

    by desenz (687520) <roypfoh&gmail,com> on Thursday July 10, 2003 @08:33PM (#6411364)
    The figures, but does the cost of the bandwidth exceed the price of gas?

    Eh. Guess it doesn't matter anyway. Its still cooler to be seen driving down the street w/ lots of tapes.
  • ArsDigita University (Score:3, Interesting)

    by jrothlis (223342) on Thursday July 10, 2003 @08:33PM (#6411367)
    This is how ArsDigita University distributes its course material: http://aduni.org/drives/ [aduni.org]
  • It's one thing to complain about the lack of growth in bandwidth of current storage (and there is quite a bit of complaining in this article about it), but to think that there is something wrong with having all this data that is theoretically impossible to access because the bandwidth is insufficient is clearly false.

    Whether data is ever used or not, it is important to have it. I have tax records from the last 7 years that I never plan on opening. They are stored in a couple shoeboxes in the back of the
  • by Naikrovek (667)
    Back in school we always had a saying, "Never underestimate the bandwidth of a station wagon filled with backup tapes."

    unless you went to school in the late '60s or early '70s, you copied it from someone else.

    i know you're younger than me, and I went to college in the mid-nineties. We didn't say that.

    We said "never underestimate the power of magnetism of a beach house and three kegs."
  • Yes, but what's the bandwidth of a minivan full of CDROMs [laminack.com]? I get 235 Mb/sec. Enjoy.
  • Next on /. (Score:5, Funny)

    by Wrexen (151642) on Thursday July 10, 2003 @08:35PM (#6411382) Homepage
    This week: You can make a trade-off between latency and throughput!
    Next week: Cars that can haul less can be more fuel-effiecent!
    The week after: Algorithms that use more memory, but are faster to execute!

    Wonders never cease!
  • Lag (Score:2, Funny)

    by arevos (659374)
    A truck full of harddrives may have some astronomical bandwidth, but the lag on that thing is murder. I tried a few experimental pings:

    PING www.google.com (216.239.37.99) from 192.168.0.7 : 56(84) bytes of data.
    64 bytes from 216.239.37.99: icmp_seq=1 ttl=51 time=87303012 ms
    64 bytes from 216.239.37.99: icmp_seq=2 ttl=51 time=130256230 ms
    64 bytes from 216.239.37.99: icmp_seq=3 ttl=51 time=110821205 ms
    64 bytes from 216.239.37.99: icmp_seq=4 ttl=51 time=990602128 ms

  • Physical transfer of data storage devices will conceivably always be the preferred method of transferring enormous amounts of data in a relatively short timeframe. The throughput for mailing a gross of full DLT tapes overnight is probably a long way from obsolescence, yet there will always be promise in low-latency via a highly specific networking design... trunk a few gigabit ethernet connections together and you can have high bandwidth high throughput low latency transfer of many terabytes of information
    • Re:SneakerNet (Score:5, Interesting)

      by Anonymous Coward on Thursday July 10, 2003 @08:48PM (#6411462)
      I used to work at an IBM site that was used for offsite back-up by major companies. They have these really cool 38 ton trucks that come into the loading bays, where they just connect up a couple of cables and pump the data off the trucks and into the building.

      Basicly they shunt data around, the same way Exxon et al move oil.

  • by irritating environme (529534) on Thursday July 10, 2003 @08:36PM (#6411395)
    Kick off transfer, go to sleep. If it takes three more hours, who cares? You aren't burning wet cycles yourself.

    And, befitting my moniker, it's better for the environment.

  • by Jason1729 (561790) on Thursday July 10, 2003 @08:39PM (#6411406)
    On our Distributed Systems final, we had a question about using an airplane full of CDs being used to replace our school's internet connection. The point was the even though the plane offered 10,000 times more bandwitdh, the 80 minute latency meant it wasn't a viable replacement.

    Jason
    ProfQuotes [profquotes.com]
  • carrier pidgeon...sure, it doesn't get there as quick, but it's great fun attaching hard drives to the feet of pidgeons and dropping them out the window.
  • by jdehnert (84375) * <{jdehnert} {at} {dehnert.com}> on Thursday July 10, 2003 @08:39PM (#6411411) Homepage
    Chips have gotten faster. Ram is bigger faster and less expensive. Disk space is dirt cheap.

    But the telecom industry is just crawling in comparison. I use the same phone line for dial up now as I did 10 years ago, and things like ISDN, DSL, and Cable Modems get you better performance, but nothing stellar. I don't think a T-1 has really changed in cost for a very long time.

    Funny, when the bubble was expanding all the talk was about the bandwidth we were suppored to have access to, but it never made it to my house.

    Eschew Obfuscation
    • by semanticgap (468158) on Thursday July 10, 2003 @09:46PM (#6411735)
      I have this theory that the reason teclo prices have not changed is because of long term agreements. Back in my ISP days, we used to sign 7-year terms on T1's because they were cheapest and we knew we'd need them. This was in 96-97, so these agreements are in force until 2003-04... When time comes to renew this, noone in their right mind will pay, and we will see a drop in high-speed prices (and Verizon and MCI wining to congress probably).
    • by jellomizer (103300) on Thursday July 10, 2003 @10:39PM (#6412026)
      Well there is a Pattern That goes along comparing Desktop Performance and information needed v. download bandwith.

      First during the mainframe era a serial connection was fast enough to operate a dumb terminal hooked to a large expensive mainframe because the information that needed to send was small and processors were expensive.

      Bandwith Wins.

      Then the PC era came about where software starts to become more graphically intensive and needing more direct access to the CPU power which is becoming cheaper.

      CPU Wins.

      Internet Era, when people are getting use to using the internet using now using graphics and bandwith have improved to send the graphics at a resonable speed.

      Bandwith Wins.

      Today. We have been collecting a lot of data and storage is cheap we are use to having a lot of data on hand. Data now is becoming more multimedia based thus taking a lot more space then a text file.

      CPU Wins.

      Possible Future. After echomony picks up and improvements in PDAs and a wide spread of wireless internet ether via WiFi or Cell. The PC will become more pointless in todays life. Servers will be providing the data for the PDA and intercommunicating with other PDAs.

      Bandwith Wins.

      And continues.

      It is not a real circle and these times overlap but there is change in focus of desktop computing to client server.
  • by muonzoo (106581) on Thursday July 10, 2003 @08:40PM (#6411413) Homepage

    CowboyNeal writes:
    Back in school we always had a saying, "Never underestimate the bandwidth of a station wagon filled with backup tapes." Seems like that still holds true.


    That `saying' is from Andrew S. Tannenbaum's [cs.vu.nl] notoriously well written textbook titled simply: "Computer Networks" [amazon.com].

    It was certainly in the 2nd edition, the one I used, and might have even been in the 1st edition. I is still in the latest edition. (One of the young-uns in the office has the 4th edition on his shelf.)

    A famous line if ever there was one in the geek world, although perhaps not as humourous as Chairman Bill's:
    "640K ought to be enough for anyone [ paraphrased ]".
  • Can anyone imagine the bandwidth that AOL is "sending out" with all their worthless CDs. I mean I'm getting about 600megs a day from them. Deliver a batch to Office Depot with a good 1000 CDs and that's some really massive bandwidth.
  • At college we still use diskettes to transfer data from college to home. Mainly because in this information age, my college network is completely shielded, proxied and protected up to a ridiculous level. (Though still pretty much ineffective) So we cant make outgoing FTP connections to upload stuff to our home computers (if anyone else is crazy enough to run a server at home to start with) and VPN from home to college is most likely out of the question as well since they still use Windows aNTique Server ove

  • Why does everyone only count bandwidth as the time to do the transport? The same comparison has been made of Netflick. Retrieving from storage and placing it back into a usable format takes time too.

    Example: station wagon full of backup tapes. Presumably, you are going to store your data at both locations (onsite and offsite copies). Now count the time mounting each tape and it's target, doing the copy, and returning the original to the car. Yes, even at 15MB/s (LTO drives) it's good, but it's still a l

  • by Anonymous Coward
    What about giving each car a portion of the data each, and having each of them carry parity information in case one of the cars fails?
  • by skogs (628589)
    the easy access of removable drives. Simply pull it out of your computer, and walk down the hall. Or Simply pull it out, and walk home with it. Very easy way to transport all those important files(work and pRon) home with you. If you don't have a OC3 line at your house, it definitely is better to carry the drive with you than download it.

    Honestly, there is never a substitute for remote archives and such in case of a fire or something.

  • There was an article on /. on the maximum physical speed possible for CD's and DVD's,
    seems appropriate here...

    Ah...here it is:
    exploding cd's [slashdot.org]
  • by SharpFang (651121) on Thursday July 10, 2003 @08:47PM (#6411458) Homepage Journal
    ...when the modems were scarce and phone bills high. Every more or less respectable demoscene group had a member whose function was listed as "swapper".

    Swappers would get in contact with swappers from other groups, and exchange floppies full of newest stuff, productions, news, and everything of any interest (plus some exotic stuff other than floppies - a chicken bone, The Party membership ID, misprinted train tickets, and whatever interesting that caught the eye and filled the envelope up to (but not above) another price-weight treshold.)

    One of the most specific swapper activities was "faking stamps". With 80 and more contacts, at least one letter a month exchanged with each of them, you had to cut on stamp prices, so you smeared the stamp with water-washable glue and wrote in the letter "stamps back", so your contact ripped your stamps off the envelope and sent you in his reply letter together with floppies. Then some washing and stamps could be reused - one set of stamps could go the same way 5-6 times before they needed to be replaced because they started looking suspect. And if it was found - you never put return address on the envelope and nobody in the post office could ever read an Amiga floppy :)

    Another practice was making the floppies sent pretty. You almost never sent back the same floppies - they were in constant flow. Adding a marker signature was the default. Often some sticker or a drawing was common. But there were true masterpieces: A floppy painted gold, with the metal part (and under it) painted silver, the metal part without the spring but removable and attached with a thin chain to the write-protect hole, so you removed it before inserting and it was hanging from your floppy drive while the floppy was inside.

    And finally all the "disk hunt" methods. Famous swappers were rarely replying to newbies who were asking for contact - you had to gain some fame on the scene with your group's productions - or get a recommendation from another swapper. So - the unanswered letters were a good supply of floppies. Sometimes they would even put an ad in some zine (spread by swapp of course ;) which said a girl wants to swap, everyone welcome etc. This was bringing a good deal of free floppies, often with some quite funny stuff on them.

    Well, Internet was what put end to it. Plus average data size - sending 6-8 floppies in one letter wasn't cheap or easy anymore, and with A1200 getting more common, high-level languages, multi-disk demos and mpeg movies, it became necessity...

    Nowadays still throwing a CD across a computer lab is way faster than transferring the data over the net :)
  • If you RTA, it sounds like based on current advances, in 10 years we'll be at the point where disks are so large 920 TB each) that access will have to become sequential (making them like tape today, access speeds not increasing as fast).

    That would leave room for RAM to essentially become used for random access in the way the disks are used today and perhaps current cache on the CPU to be used more like RAM is today?

    A lot of wire-speed net devices are starting to look like this, with their info stored in a non-volatile storage device, but loaded into RAM on startup and all "work" done in ram.

    It's easy to image a whole chain reaction of purposes for devices slipping into other functions as a result of varying levels of technological advancement in them.
  • by ralphclark (11346) on Thursday July 10, 2003 @08:55PM (#6411506) Journal
    The academic world has had a common operating system that everybody can talk about and experiment with... It has the downside of creating a mob culture.
    Hey, he can't talk about us like that, can he?
  • RFC 1149 (Score:3, Funny)

    by SharpFang (651121) on Thursday July 10, 2003 @09:01PM (#6411545) Homepage Journal
    You may find RFC 1149 [faqs.org] useful:
    "A Standard for the Transmission of IP Datagrams on Avian Carriers}
  • by cait56 (677299) on Thursday July 10, 2003 @09:06PM (#6411568) Homepage

    Not only does a stationwagon full of harddrives have a respectable sustained throughput rate, the contents don't get screened by the firewall. Ditto for the hardrives in a briefcase, or those USB drives on a keychain.

    Exploding capacities of storage drives have implications on attempts to keep data within boundaries, as well as attempts to getting it from point A to point B.

  • Kitchen Storage (Score:5, Interesting)

    by fm6 (162816) on Thursday July 10, 2003 @09:12PM (#6411596) Homepage Journal
    Each disk was the size of a washing machine and cost around $20,000.
    An interesting gap in Gray's memory. He's not describing the disk, he's describing the drive. In 1970, all storage was removable. A free-standing IBM drive was not only the size of a washing machine, it looked like one, because the top opened up to insert or remove a disk pack [ed-thelen.org]. There were other multi-drive consoles that resembled pizza ovens.
  • by Cplus (79286) on Thursday July 10, 2003 @09:15PM (#6411606) Homepage Journal
    One of the major thoughts in the whole interview is that our storage has increased to such a point that we can't access it all in a reasonable fashion. For my uses (which are far from industrial level) I find that I can only watch one movie or listen to one song at a time. On my 200 gigs of hard disk I've got 60 gigs of music (and growing daily) and at least 100 gigs of movies.

    Don't judge me on the legalities of the situation, but note that this isn't uncommon...I have some very drastic media needs and the media that I like is pretty intensive, but I don't very often need to stream any of it en masse to another location. It suits it's purposes fine exactly where it is, and I haven't had any problem acquiring any of it or accessing it.

    I suppose my rambled-to point is that for my needs I'd rather there was more storage at this point than have higher access speeds as I can get all that I need as fast as I need it. Perhaps our usage of the medium dictates how it develops.
  • by kaltkalt (620110) on Thursday July 10, 2003 @09:37PM (#6411706)
    The essence of the article is snail mail has higher bandwidth than electronic means (or something to that extent). This ignores the fact that most programs/data transmitted today are huge. gigabytes. A one-sided DVD is 4.7gigs. Even if it took, for the sake of argument, 2 days to transfer that DVD electronically and 2 days to ship the DVD across the country priority mail, the cost of badwidth vs. postage has to be taken into account. Postage for a disc is a little less than $3 for priority mail (and less than a dollar for regular 1st class). Is having one's bandwidth tied up (slowing down everything else on the network) worth $3? $1? No, of course not. And as data gets bigger and bigger (it always does), mail will still cost less ... at least for a long time. CDROMs and DVDs are small and light--perfect for sending cheaply in the mail. So, to say mail is faster than uploading data is a shitpoor comparison. And while it sounds fascinatingly shocking, that's only because it's ignoring some pretty big factors.
  • My experience (Score:3, Interesting)

    by Dok Fenderson (650034) <dok@dok.homeunix.com> on Thursday July 10, 2003 @09:44PM (#6411729) Homepage

    I just recently moved halfway across the US from my hometown. A buddy of mine who had a ton of MP3s (mostly legal BTW) had just suffered a HDD crash and his SO's car had been broken into meaning that TONS of music had been lost/toasted. Before I left, I'd copied his whole collection to my drive. Shipping him a drive with the whole contents (60 GB) of my music collection took a hell of a lot less time than letting him download it (at 20 Kb per second (Ghod I hate SBC!)) or worse yet, take the time to pick through it at human speeds, and was far cheaper unless you figure that the cost incured by me sending it overnight was in addition to my regular bills.

    Dok

  • Faster shipping (Score:5, Informative)

    by rice_burners_suck (243660) on Thursday July 10, 2003 @10:24PM (#6411946)
    I think a system should be devised where you could queue a big upload or download, and the network will "know" how to send big chunks of it when things are relatively idle. By queueing things for what basically amounts to "background" transmittal, the network might be more fully utilized.

    The station wagon comment reminded me of an idea that I had a long time ago, when I first read about how the Internet routes packets around. You know how you can ship stuff UPS overnight? It can get pretty expensive, depending on how big and heavy the package is. And sometimes, businesses would pay an even greater price to have a package delivered even faster. Why not introduce a system for getting things delivered extremely fast, and I do mean fast, all around the world?

    Imagine this: Put together a network of railroad-like tracks that are enclosed in concrete tunnels. In a vacuum. Individual cars would travel on these tracks at greater than mach speeds. They would essentially go from one switching station to another, kind of like the telephone network or the Internet. They might come in several sizes, these cars. When you need something delivered fast from, say Los Angeles to New York, the package would be placed on a dedicated car which would take it at blazing speeds through, say, Albuquerque, Oklahoma City and Louisville, to New York. At each station, equipment would adjust switch tracks to route the car to its next switching station; the car would not even have to stop or slow down. The package might be there in four hours, counting the time it takes to bring the package to a station, have it loaded, unloaded, and then transporting it to its final destination.

    This might actually make shipping cheaper rather than more expensive. Automatic equipment sorts mail at the USPS. If this mail were collected, say, once every hour (during business hours), taken to the nearest major USPS distribution center, where it is sorted, placed in boxes heading to the same destinations, and then shipped (tunneled?) through the above method, mail going to a distant location might arrive faster than mail going across town. This could be done with collections of packages that are all going from one major city to another together. Load them in a container and bust them all over there. Sure, it'll still take, say, 24 hours to ship packaged in such groups, to save money, since you have to wait for enough packages, sort them, group them, etc., but if you want something shipped right friggin now, the option to get a dedicated car is still available. This might reduce use of gasoline and use of air and ground traffic. If computers can control the cars on these tracks so that cars are going mach 2 almost bumper to bumper, that would allow for extremely great throughput.

    Back to the station wagon comment, supposing this could be done, (running more tracks all over the world and installing these switching stations at each major city), you could load hundreds of terabytes of data onto a big friggin raid system and then get that data across the world faster than shit going through a tin horn.

    • Re:Faster shipping (Score:3, Interesting)

      by aziraphale (96251)
      What this made me think was, you could install large banks of hard drives into the cargo holds of planes and the back of express long distance trains, and plug them into fast backbone connections whenever they're stationary. This would then let the internet route data that doesn't need low latency connections (such as FTPing terabyte files, where it doesn't matter if you receive the first packet now, because you're not going to be able to use the file until the last packet has arrived anyway) onto the stora
  • Shipping Laptops (Score:5, Interesting)

    by zymurgyboy (532799) <`moc.oohay' `ta' `yobygrumyz'> on Thursday July 10, 2003 @10:35PM (#6412008)

    I work in a AmLaw top 100 law firm in DC. We do a lot of complex litigation work. We use software such as Concordance, Ringtail, and Litgator's Notebook (runs on Lotus Notes) to manage collections of documents. The documents are scanned to group IV tiff; the meta data and OCR text that is extracted from the documents at scan time is loaded into another database that overlays the images.

    These tiff file collections run into the millions.

    Of course the point of doing this is to facilitate collaboration on document review between us, our clients and our co-counsel. These people are often 1000s of miles apart, and nearly as often have crap for IT resources (equipment and personnel).

    There are ways of accessing this stuff over the internet securely but it's never quite the same as having the real version of the software. This form of access often proves to be impractical for the lawyers who travel alot depending on the type of access they can get wherever they end up.

    So what often happens is, we end up dumping the entire collection on a laptop with a big hard drive or a bigger firewire or USB drive, so they can work without access to the internet and then replicate changes when they can get the laptop back on ethernet or a POTS line.

    Collections of images and databases (not to mention the various Power Point presentations and word processing files) can very easily run over 50GB. Moving this across the LAN, over my PC BUS to another hard drive and then FEDEXing it is certainly faster than doing the same transfer using FTP or SCP. Not to mention, that way I can install the software (properly) and test the whole setup before I send it off. The extra wear and tear I save on my psyche from NOT having to explain how to install all of the software, point it to the image collections, and deal with equipment I have no control over while being screamed at by extreme Type A attorneys going to trial makes that laptop look like a pretty good investment.

    These are good [wiebetech.com] if you have someone on the other end of your FEDEX run who know how to open the case on a PC and install a HD themselves. I can setup one machine with everything, image the hard drive, make copies on other drives and drop them into FEDEX pouches as fast as I can make 'em. I can't think of a faster way to move a few 100 GBs of stuff to a half dozen places inside of a day. If someone has ideas, I'm all ears.

  • by Leebert (1694) on Thursday July 10, 2003 @11:43PM (#6412325)
    I'm involved in a project moving 150TiB from the West Coast to the East Coast. I can attest to the fact that it is cheaper and faster to ship tapes via FedEx.
  • by beaverfever (584714) on Friday July 11, 2003 @12:17AM (#6412513) Homepage
    "Today disk-capacity growth continues at this blistering rate, maybe a little slower."

    What is a bit slower than a blistering rate? A skin-reddening, sensitizing-to-the-touch rate?

  • by zymurgyboy (532799) <`moc.oohay' `ta' `yobygrumyz'> on Friday July 11, 2003 @12:22AM (#6412534)

    Favorite quote from the article: "Not many of us know what to do with 1,000 20-terabyte drives--yet, that is what we have to design for in the next five to ten years."

    Heh. I do, so get designing. The various law firms reviewing documents from cases like Enron (criminal , bankruptcy, and civil procedings), Microsoft's antitrust suit, the SCO v. IBM, etc. etc. need that space to store all the materials from their case work. Lots of paper from all those places get turned into electronic images managed by very large custom databases.

    Guess how many Group IV tiffs and pdfs some of these become. Answer: millions. In five or ten years, cases such as these will likely consist of collections of data that large. Terabytes of data for cases such as these are not uncommon now. Enron could get this big by itself by then. It's well on its way to becomming one of the largest cases of all time. Check this out [washingtonpost.com]. Whoa.

  • That's Tanenbaum (Score:3, Informative)

    by code_martial (625004) on Friday July 11, 2003 @12:39AM (#6412591) Homepage Journal
    Never underestimate the bandwidth of a station wagon filled with backup tapes.

    This is a statement by Andrew S. Tanenbaum [cs.vu.nl] from his book titled Computer Networks [amazon.com]. Though it's supposed to be a text book (with 4.5 stars on Amazon.com), I and most of my friends also regard it as a nice collection of stories related to computer networks and communication ;-)
  • by An Onerous Coward (222037) on Friday July 11, 2003 @12:42AM (#6412602) Homepage
    ...will be bored college students knocking on each others doors, with USB hard drives full of MP3s. Nodes can be linked via CD-ROM and DVD media. What's the point of buying CDs when you already have literally months worth of music that you haven't even sorted through?

    Honestly, I think RIAA would do well to back off. If they manage to kill off P2P trading, it will only be replaced by something much, much worse.
  • by KalvinB (205500) on Friday July 11, 2003 @01:10AM (#6412686) Homepage
    I run free anonymous FTP off my server because I can. Occasionally someone asks me if I worry about someone filling up my HD and crashing my server in the process.

    Then I point out it takes around 8 hours to back up the 80GB drive over a 100Mbit LAN. I have a 640Kbit downstream connection. It would take a month to fill the entire drive.

    I had someone connected to my server for 14 hours uploading a pirated game. I let him finish. Opened up the zip file and replaced everything in it with a single text file with the person's IP and log entries showing their attempt to pirate software.

    I've often burned stuff to CD rather than upload it to my server over the net. Even for relativly small sites like my own, it's far more efficient. It's never an emergency situation where the files have to be there "this second" anyway.

    It's not surprising that big companies don't waste their bandwidth that customers need and just transfer physical media instead where possible.

    Ben
  • by MattRog (527508) on Friday July 11, 2003 @10:32AM (#6414672)
    I'm pretty disappointed (although not entirely surprised) in SlashDot posters. This article was clearly more than simply 'mailing disks' which > 95% of the topics (including dupes of dupes of dupes of ...) on this article have been about.

    Sure, he mentioned cost of shipping disks, and actually concluded that shipping an entire computer system is more economical than mailing individual disks. However, there are far more interesting and discussion-worthy conclusions he raises.

    What about disk capacity reaching such incredible sizes as 2TB/disk - and the fact that current random-access methods will render such drives unusable? This affects all of us, since our OS' filesystems will need to fundamentally change to be more sequential (e.g. like tape drives). Personally, I hope that whatever happens to the fs the OS will insulate me from being forced to use it in a sequential manner (e.g. will I be exposed to the sequential nature of the medium or can it be successfully abstracted?)

    He talks about, in almost glowing terms, the SlashDot favorite MySQL and how "At some point, somebody will say, 'I'm running my company on MySQL.' Indeed, I wish I could hear Scott McNealy [CEO of Sun Microsystems] tell that to Larry Ellison [CEO of Oracle]." And, although the Research Area people are pretty independent, this is from a MICROSOFT employee. Not a peep from the /. audience.

    Personally, I think that using MySQL as a 'research tool' as he suggests is a Very Bad Idea - it's not even a mediocre implementation of the relational model and there are better open-source implementations out there (PostgreSQL being the one that comes to mind). Basing scholarly studies on MySQL would be like basing the foundation of a skyscraper on a shack (not that any other SQL DBMS's are much better, but why use one of the worst?). The best 'research vehicle' would be an open-source truly relational database management system (there are no commercial TRDBMS either). It doesn't have to be very advanced, but it has to be architected from the ground up to be a TRDBMS (which means SQL doesn't cut it as a query language).

    One thing he notes which I see as being a large problem in the open-source community as well is how "...The thing that slows Oracle, IBM, and Microsoft down is the testing, and making sure they don't break anything--supporting the legacy. I don't know if the MySQL community has the same focus on that." As a long-time PHP developer and advocate I'm still hesitant about updating our production systems - it seems as if every successive release of PHP has innumerable functions removed or changed with no ability for backwards compatibility. I guess it's a lot easier to say to users 'you get what you pay for' when they are just that - users and not clients. One of my disappointments from many open-source proponents (which I am one) is the hostility to treating clients as clients - 'you can always edit the source', etc. - for the most part large companies don't care/want to edit the source - that is what they want to pay you to do. Until more projects (MySQL included) start to realize this, then they will pretty much always occupy niche roles in the enterprise.

    Finally, even he, an academic seems to (at times) confuse the relational model's implementations' details (e.g. the SQL product performance) with the model itself (of which there is no mention of performance, because it has nothing to do with the model). Theoretically, a TRDBMS should be faster than the SQL implementations we have today. It just takes someone to do it, and I don't see why the open-source community can't build the BEST mousetrap there is - we just have to abandon the 'mob culture' of MySQL.

"Well hello there Charlie Brown, you blockhead." -- Lucy Van Pelt

Working...