Storing Data For the Next 1,000 Years 243
An anonymous reader writes "This may be an interesting take on creating long-term storage technologies. A team of researchers at UCSC claims to have come up with a power-efficient, scalable way to reliably store data for a theoretical 1,400 years with regular hard drives. TG Daily has an article describing this technology and it sounds intriguing as it uses self-contained but networked storage units. It looks like a complicated solution, but the approach is manageable and may be an effective solution to preserve your data for decades and possibly centuries." Nice to see research on this using the kinds of real-world figures for disk lifetimes that recent studies have been turning up.
Re:Sometimes old tech is best (Score:5, Interesting)
Insufficient Research (Score:2, Interesting)
Look, I am into genealogy quite a bit and see this as a tremendous problem.
The only thing approaching a viable solution is the Rosetta Disk ( http://www.rosettaproject.org/ [rosettaproject.org] ) using etched nickel media (rock) in a human readable format, which you could theoretically create a binary cipher for a global archival format.
But, that would take a lot of foresight, which unfortunately us people don't have (yet).
However, seeing that as completely inaffordable for us mere mortals, that leaves me with PAPER, yes, paper, as the only trustworthy medium-term solution.
I do hope everyone here realizes that if we had some sort of cosmic EMP-like event traversing the globe, we'd lose 99% of data and be plunged into the dark ages, right? We couldn't even re-create all of the machines that surround us since virtually all designs are kept digitally now. Factories would just shut down and never be able to be brought back up and every history of our existence would be forgotten in a few generations.
Our civilization is sitting on a house of cards.
Lasers. (Score:3, Interesting)
ah yes, here, [rosettaproject.org] that seeks to preserve all the languages of the world by laser-engraving them onto stainless steel plates. They've changed things up a bit, but the basic idea is the same: put it somewhere it won't get lost or corrupted, and if it's important, people will figure it out later. If it's not important, then it doesn't matter.
Very few things in the world are really worth keeping for even a lifetime. If your grandkids inherit all of your stuff, what will they save and keep, and what will they throw away? If you know what they will throw away, why not save them the trouble and toss it yourself?
We've gotten ourselves into this mindset where making backups of every piece of data you've ever owned ought to be saved, for no other reason than because it's easy and cheap. I think everyone should have a periodic storage meltdown to force them to reconsider what it is they really need to have.
Two methods for long-term reliable storage (Score:3, Interesting)
Electronic storage is by its very nature unreliable -- electromagnetic properties (like charge accumulation, ferromagnetic hysteresis, etc) are inherently volatile.
And even if you manage to solve the problem of transporting your data into the future, you're still faced with the problem of making sense of it. Electronic formats change (just ask the guy out in California who makes a *FORTUNE* charging law people to retrieve files from obsolete formats and/or media). In the physical realm, this is true as well - languages change and become very difficult to read. (If you don't believe me, try reading Beowulf in its original old-English form, circa 700 AD).
Why have physical storage at all? (Score:4, Interesting)
And I mean it literally -- why have any physical storage at all? Why not just bounce chunks of data around forever on the Internet? Presumably the 'net is going to be here for a long, long time. Imagine a mass P2P network where the data being traded is just encrypted chunks of the data of other users. It needn't ever get written to a mass storage device at all -- just received from one peer and immediately sent to others.
A protocol could be developed to allow one peer to request, or steer, the network to locate and deliver requested blocks on demand. This might be a high-cost operation, akin to bringing data in from backup tape. Or, a client could just wait for the right chunk of data to recirculate to its position in the network. But storing data is easy -- just encrypt it, format it a certain way, and inject it into the network.
A natural model for the topology of such a network, and the protocol itself, is the circulatory system. Here, cells move in a fluid, generally in one direction, but through a complex network of vessels, and in a circulatory manner. The immune system might provide inspiration for directed movement of data chunks. (See? The Internet really is just a series of tubes.)
Over time, the infrastructure of the Internet, the P2P clients, and the exchange protocol itself could evolve, as long as enough redundant chunks are allowed to constantly recirculate. Specialized clients could cache data to "long term" storage for periods of a few days or weeks, in case of large, random outages, but permanent data storage would never rely on any specific technology at all -- even TCP/IP itself. It's all just this mass of recirculating encrypted chunks of data, like cells in the blood stream.
Re:Insufficient Research (Score:3, Interesting)
You don't need a supertermite :)
Just an idiot with a political agenda and authority on his hands:
The Nazis used to burn books if I'm not mistaking.
Also, if I remember correctly, there was some pasha or other in the ottoman empire who said that either the kuran is the only truth and then other books have no purpose, or the kuran is not the only truth, and then the fact that there are other truths must be hidden; thus, he burned the library.
It only takes a bunch of idiots.
Modern form of punch card is the solution (Score:1, Interesting)
The biggest problem with archival solutions is that to create a method which is small enough to store in a few boxes, the technology will typically be required to operate on a miniscule scale. This increases the probability that the media won't be properly identified as an archival media a thousand years from now.
While it's possible we'll have simple flatbed scanner like devices in the distant future capable of simply scanning a DVD or BluRay disc and then simply applying an image recognition program to read the data, it's more likely these forms of media will not be sufficient.
Recordable CD and DVD suffer an obvious flaw which is that the method of recording requires the disc itself to be a degradable form of media. Something that can be burned through at a rate of billions of times per minute with a single laser with high precision. Obviously, even using a relatively high power laser, the material must be thin enough to support this. Therefore, it stands to reason that even in a perfect DVD recordable media, time, sunlight, cosmic radiation, and new-age music is bound to degrade the disc past error correction friendly levels in relatively short times (likely years, maybe decades, certainly centuries).
In the case of circular media of high density, it requires precisely timed devices to read a disc. The disc spins and the bits are positioned in locations that are identified by precise timing. The device to read this type of media is very complex. The bandwidth of the laser required to read the disc is also precise. An archival grade media should not have such technical difficulties, otherwise, the effort required to read the media a century from now if no device is left in existance is substantial. Especially since the plans for the reader is likely to be stored on one of these discs.
Punch cards are wonderful since they are linear, rectangular and can be read relatively easy using somewhat primative equipment. Of course, I'm not suggesting using holes large enough to push a pen tip through, instead I'm suggesting a relatively high density punch card where at least a gigabyte can be stored on the surface of a drink coaster sized card.
The card can be made of many different types of materials, in fact, it could be paper or uranium. I would suggest personally a dense metal with a half life relative to the desired duration the media should live.
On one side of each card the plans to build a reader should be stored as human readable images, although on a microscopic scale (similar to microfilm or microfiche) since anyone likely to be able to build a reader will of course have a microscope. The reader presented should be the simplest form and should clarify the encoding used for characters. The device should be able to be built using parts that have been historically available. Meaning that if they existed 50 years ago, and they're still common today, they'll be common 50-100 years from now at least.
The data on the card should be stored through a process of laster engraving or etching for example. Punching directly through the card is ideal, but could interfere with the location of the design. Of course, if you have a box of 1000 cards, you only need the design once.
The process of reading and writing these cards does not need to be incredibly fast. After all, the purpose is for archival. If a single inexpensive device can write one gigabyte an hour, that means with 20 devices, 20 gigabytes can be archived an hour. Besides, unless you're backing up film masters which are typically 3-5 terabytes each or audio masters which can usually be 4 gigabytes, all other files (pictures included) can be backed up in little time.
Every time I read about long term archival, this solution always seems obvious to me. I highly doubt that there's any new patents to be had on it. Pretty sure that pool is not only tapped, but most of the patents should have even expired by now. But if anyone actually decides to make this device from my description or something similar to it, I would love a chance to try one.
Re:Uh, what? (Score:3, Interesting)
Something I didn't realise about the Old Testament until recently is that when they talk of the the Philistines binding Samson in 'chains of iron' it's because the Philistines had managed to master the technology to use iron but the Israelites hadn't.
http://en.wikipedia.org/wiki/Philistines#History [wikipedia.org]
The Philistines long held a monopoly on iron smithing (a skill they possibly acquired during conquests in Anatolia), and the biblical description of Goliath's armor is consistent with this iron-smithing technology.
So 'God's chosen people' hadn't enterered the Iron Age at that point. There's lots of other signs that they were not exactly academically inclined either, like the biblical value of 3 for Pi which was less accurate than the value the competing civilisations knew [st-and.ac.uk].
The Qu'ran is just a bad mashup of the same primitive ramblings that inspired the Old Testament with some self serving editing by Mohammed. Or more likely early Muslims, since Mohammed was not particularly literate and had more important things to do with his time, like capture slaves and booty from more settled neighbouring tribes.
Re:Very little is laughably simple (Score:2, Interesting)
Major flaw in plan (Score:3, Interesting)
"In theory, practice is perfect; but in practice, it is often only theory".
Adeptus
Re:Uh, what? (Score:4, Interesting)
I mean that Communism and Nazism behaved like religions.
No. I was there, and I can most certainly say that they were ideologies and not religions. Religion always includes or endorses some ideology, but the reverse is not true, ideology does not necessarily have anything to do with religious belief.
A state-supported ideology is common and often nearly invisible for the member of society that practices it -- it is proclaimed (often clumsily) by government officials, is seen kinda working because society can prosper while supposedly implementing it for decades, it is assumed to be right by most and rarely questioned, but people also rarely actually think about it, or any alternatives, it's as if its validity or invalidity is irrelevant to the people's lives as long as society is capable of implementing it without creating discomfort and unrest. After all, it merely claims what is "a better way of running a society" as opposed to making claims about physical world that exists independently outside of human mind and ideas. Since most of people are not politicians, assuming that politicians are following some sort of rules that have little impact on everyday lives is a natural (though often stupid) thing to do, however for, say, a physicist it would be impossible to assume that religion's creation myth is correct -- it contradict with things physicist experiences in his work. In US the ideas of "capitalism" and "democracy" enjoy the same kind of ideological support -- I can make a case of both of them being pretty poorly thought out ideas in the first place, and separately of neither of them actually playing an important role in the way US society operates, however none of it will be a scientific argument because I will have to discuss people's ideas, behavior, motivation and impression about life. At most I can catch government and businesses lying and manipulating people using ideology as the tool to achieve desired behavior of the masses, however for every my claim there would be tens of millions of rednecks claiming that they naturally love doing exactly what I see them manipulated into doing.
Religion, on the other hand, requires actual belief and is treated not only as important part of everyday lives, ethics and history but also makes claims of facts -- something that ideology often approaches but never actually does. Even Nazi had to form their ideas of "superiority" and "rightful claim" of control in subjective terms -- though they used religious imagery and pseudo-scientific language, they neither required belief in any deity or creation myth, nor bothered to find scientific evidence of any kind. Their ideas are only "religious" in a way of "but won't it be nice if YOUR ethnicity was destined to rule the world?" as their first and last greatest proof of their ideas, not unlike "but won't it be nice if the world was ruled by benevolent deity?" is the first and last greatest proof of religion. It's a pretty weak analogy.
In that sense the Communist belief in Lysenkoism is a bit like the Catholic aversion to birth control. Neither were part of the original doctrine, but once you have priests or politicians that believe they have access to the absolute truth a bit sprouting is almost inevitable.
No. It's merely one person who gained favorable treatment by the government and massively abused the power he gained through it. This has nothing to do with religion and everything with government officials' irresponsibility and concentration of power. After the end of Stalinism in mid-50's, Lysenko's theories were thoroughly discredited, and it remains a single such event in the whole USSR history -- it taught post-Stalin governments to never mess with the content of scientific discourse, and limit government's influence to choosing directions to fund and support.
US propaganda loves picking such blunders in USSR history (almost exclusively taking them from Stalin's time) and present them as if they discredit wide aspects of USSR or Russian society, C
Re:Only half the problem (Score:3, Interesting)
Maybe proprietary formats were meant to die.
I still have documents in plain ASCII that I can open from over ten years ago. I've got a few .wri's that I can still open, thanks to reverse engineering efforts by the open source community. Older proprietary formats are now defacto open standards. The thing that can kill this off? Patents, for one. Trade secrets in the form of overly complicated proprietary formats for another.
And yes, I realize I'm not talking about GB's of "mission critical" data stored in some awkward database. You still have to admit it would be easier to convert data from a PostGreSQL database of ten years ago than an Sybase or FoxPro one.
Will semiconductors work in 1000 years? (Score:2, Interesting)
I know about semi's - I don't know about any other things in the devices. How will FR-4 age? Will solder joints fail just by sitting around?
Re:Why have physical storage at all? (Score:3, Interesting)
Good idea but some one first tried this in the 1950's The idea was to send the data encoded on a microwave beam and aim the beam at the moon. The signal would bounce off the moon and come back to Earth a few seconds later. A receiver would detect the signal and feed it back to the transmitter. Many thousands od bis would be stored in the radio signal.
This was an extention of then current technology which was basicall a TV set with a tv camera focused on the screen seding it's signal back. The data was stored in the CRT's phospher. The storage devive actualy had the TV screen and TV camera inside the same vacuum tume as an integrated device. Bit were stored as dots on the screen
The then someone (at MIT, I think) invented the magnetic core and sold it to IBM. The core moad all of these "regenerative" systems obsolite. Core was the way to store data untill recently when semiconductors took over. In fact many of us sometime slip up and call RAM "core". Memory dumps are still called "core files".
So untill the early 50's all data was stored as you suggest - it was sent some place and then echoed back an resent endlessly.