Ext4 Filesystem Enters Experimental Kernel Tree 237
An anonymous reader writes "Looks like the next version of the venerable Linux 'ext' filesystem is just around the corner. Andrew Morton has added an early version of ext4 to his 2.6.19-rc1-mm1 tree, enabling Linux to support storage volumes up to 1020 petabytes in size, and to write files in 'extents,' or contiguous, reserved areas. According to an article at Linux-Watch, ext4 will be ready for production use within six to nine months, if all goes well. On the downside, the new ext4 filesystem will offer only limited backward compatibility with ext3-aware Linux kernels."
Reiser4 (Score:3, Funny)
Re:Reiser4 (Score:5, Insightful)
As much as I agree with you questioning the many 'funny' ACs, I can't support this statement. One day you, yes, you, might also become a suspect. And then, suddenly, you would see the world with different eyes. Especially, when you're found innocent and you also find that your friends have deleted all memories of you since they don't want to be associated with you; as suspect.
I am sure many of us would feel similar once the investigation is over and Hans eventually found guilty. But as civilised people, we better wait. How old are you ? Have you never ever been suspected of something ?
Re: (Score:2, Insightful)
Re: (Score:3, Informative)
Though, the author must be a programmer and the reader must count opening and closing quotation. > 80% of the comment is quotation, right, and in the end the author of parent says Even if he is convicted that doesn't mean he is guilty.
That makes my comment bad, but not the mod. It makes the comment redundant for copy & paste of another post to finally say "I don't agree with this".
1020 Petabytes? (Score:5, Funny)
Not to be confused with Excitebike, which is something entirely different.
Re: (Score:3, Insightful)
On second thought, maybe it is Informative, since I was not previously aware you could cram that many puns into so few words.
Re: (Score:3, Informative)
That's 6, and
Short 4 (Score:2)
Re: (Score:2, Funny)
Regards,
Exabyte Corporation
fsck (Score:5, Funny)
FSCK (Score:2, Troll)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
1020 petas (Score:2, Funny)
My porn collection will now be complete.
Re:1020 petas (Score:5, Funny)
My porn collection will now be complete.
Liar... there is no such thing as a complete porn collection!
Re: (Score:2, Interesting)
Here is why:
Suppose you want to watch porn 24 hours a day from the age of 15 till 75. Thats 60 years = 60 * 365.25 * 24 * 60 * 60 s = 1.89 * 10^9 s
A DivX is around 600 MB / hour = 600 * 1000000 / (60 * 60) = 1.67 * 10^5 B/s
So for your lifetime porn collection you need 1.89 * 1.67 * 10^14 B = 315 TB.
Re: (Score:2)
I'm sure full-immersion sensory data will require far more data bandwidth.
Re: (Score:2)
No, 2^128 bits should be enough for anybody [sun.com]. Everybody, in fact.
Re: (Score:2)
What SD Porn?! With artifacts?! True Pron addicts will only settle for 1080p HQ porn!
Mind you, even that won't take up 1024 PB... maybe if you used raw video with no compression...
Re: (Score:2)
Re: (Score:2)
performance (Score:4, Interesting)
Advantages over XFS, for example. (Score:5, Informative)
XFS can do 9 exabytes (exabyte = 1024 petabytes).
They mention that ext4 is not faster than other filesystems.
Ofcourse people can do whatever they want, but why not spend their time making XFS easily resizable for example?
Re: (Score:3, Interesting)
I would also appreciate block journaling for XFS.
Re: (Score:2, Insightful)
You mean, "exbibyte = 1024 pebibyte". An exabyte is exactly 1000 petabyte.
I used to think this was just fussy, but I am quite tired of guessing which system producers of hard disks/CDROMs/DVD+±×RWs use to figure out if that is enough for my needs.
Re: (Score:2)
Re: (Score:2)
From a user POV, XFS is my favorite FS, and I've had no bad experiences with it whatsoever. However, from what I've heard about the code, I can fully understand why it's not "popular" among kernel devs, and why none of the enterprise distros favor it (because doing so would inevitably force _their_ kernel hackers to try to debug XFS-related errors).
I'm hoping now that SGI is officially "retiring" IRIX, XFS will be cl
features isn't everything (Score:3, Informative)
Simple: ext4 is a backwards compatible, evolutionary change from ext3, while XFS is a different file system and codebase. XFS doesn't offer sufficient advantages to overcome that built-in advantage of ext4 (after all, neither XFS nor ReiserFS managed to succeed even against ext3).
Wow (Score:3, Funny)
Now, is there anybody who still believes that porn does not drive innovation?
also in linus tree (Score:2, Informative)
Also merged is the developmental ext4 filesystem, which includes a number of enhancements, including support for extents and 48-bit block numbers. See the ext4 documentation file if you are interested in playing with ext4 (and have good backups).
And how... (Score:5, Insightful)
I'm honestly more interested in someone coming up with cheap, long term archival storage. Hard disks have gone so far past our ability to archive information it's beyond comprehension.
Re: (Score:2)
Re: (Score:2)
Harddiscs are, as you say currently superior to other storage-technologies.
So, you make backups to hard-disc then. Simple. Quick. Affordable.
Yes, the lifetime is limited, so you should make sure to have atleast 2 independent backups (that's true for any media, all media can go bad) and you should change them every 3 years or so.
The thing is, capacity is growing so rapidly, that in 3 years, what is now a hard-disc full of backup will be a hard-disc 10% full of backup.
I need
Re: (Score:2)
20 years ago you would ask the same thing about 500 MB hard disks. And don't forget the article (or at least the sumary, as a good slashdotter I don't RTFA) says about the file system capacity, not the real capacity of hard disks. Petabytes hard disks are something for the future, not for now.
Re: (Score:2)
I saw 50 meg drives that were 18" platters and scarry big.
I'd think your point still is valid though...
Anyone in 1986 faced with the prospect of backing up 500Mb would likely turn to the intern and say: "Start memorizing".
BTW, in 1986 if someone came up to you and told you that there would be hard drives in 20 years that are 750GB and were faster than your current system memory what would you have said to them?
-nB
Re: (Score:2)
Hmmm... aliens? Well, who has 1020 petabytes drives today?
BTW, in 1986 if someone came up to you and told you that there would be hard drives in 20 years that are 750GB and were faster than your current system memory what would you have said to them?
I'd say "to the time machine, Robin!"
Re: (Score:2)
You buy ten of them and do a super-redundant RAID.
Re: (Score:2)
Re: (Score:2)
I knew a guy... (Score:3, Funny)
rm -rf
It never occurred to him (or me) that ".." matched that pattern. He worked his way right up the directory tree and back down again...
Re: (Score:2)
If it weren't for the GPL, Linux could just port ZFS over and everyone would be happy.
Re: (Score:2)
Once, I was like you. Backupless. Then I went down to by local PC store and happened by a 320GB external USB HDD for around 200. Needless to say my current backup woes have been solved. If I ever get around to building that terabyte fileserver, all I need do is lash together four of these little beauties and use something lik
Re: (Score:2)
Luser: Dude, uh.... um.... I think I umm.... accidentally deleted the database.
You: Doh!
extents (Score:2)
Aright! 1970s mainframe technology, here we come!
Re: (Score:2, Informative)
With EXT4 having extents we'll finally have the joy of defragmenting a hard drive like Windows people. Yea, progress!
Re: (Score:2)
Given the reputation of 1970's mainframe technology for being bomb-proof, reliable, stable, and useful, I'd say it's about fscking time.
Just because something was a good idea then, doesn't mean it isn't a good idea now.
Cheers
Interesting in light of OpenSuSE's decision... (Score:5, Interesting)
to no longer use ReiserFS [wordpress.com] as its default FS (orig. reported on OSNews.com [osnews.com]...don't think I've seen it here yet). I think this came out before the whole Hans Reiser affair, BTW.
SuSE contrasted the ease of upgrading ReiserFS and ExtFS versions:
Re: (Score:2, Funny)
Re: (Score:2, Interesting)
Re: (Score:2)
They're all equally stable now
Re: (Score:2)
Novell Suse prefers Ext3/Ext4 over ReiserFS 3 / 4 (Score:4, Interesting)
Other Reiser issues aside, the SuSE folks at Novell are looking to leave [linux.com] the nearly unsupported reiserfs3 (in maintenance support, which isn't enough for them) and move to ext3 as their default FS. Why? They feel ext3 is a lot more mature & better/wider supported then reiserfs4, is an easier migration, and appreciate that there is a solid roadmap from ext3 to ext4.
Of course this would also be the week that (coincidentally) Andrew Morton gives reiserfs4 the green light [apcstart.com] for eventual mainline kernel inclusion.
So funny (Score:3, Informative)
Ext4 is going to be the MOST compatible with Ext3, relative to ANY other option out there.
Upgrading to Ext4 is NOT going to involve a dump and restore from Ext3, likely a tunefs -j or similar command, just as the ext2 -> ext3 migration worked. Ext4 will be able to mount ext3.
If older versions of software could use the new format, you wouldn't need the new format. Yes, upgrading to Ext4 means your 120 petabyte raid array will not be compatible with your old "ext3 aware kernel". But it is PRECISELY because such an array is not possible under ext3 that ext4 is going to be introduced.
And does this submitter think other fancy new filesystems magically work on old kernels? Of course not. Does the submitter know if ext4 will be backported and made available to older releases? It doesn't look like they gave that much thought either.
Please read this [lkml.org] for a more detailed description of what is happening.
Slashdot's always good for a smile.
Re: (Score:3, Informative)
The article says "On the downside, the new ext4 filesystem will offer only limited backward compatibility with ext3-aware Linux kernels."
Well if you had a clue, you would know that this is in reference to the fact that you can mount a ext3 file system as ext2 from a kernel that doesn't have support for ext3. It just doesn't journal. They were simply stating that this rather useful feature that made ext2->ext3 migration so painless will likely not be as simple in ext3->ext4 migration.
With that in m
Re:1020 Petabytes? (Score:5, Insightful)
By now you don't even now what to do with 1024PB. Just as we couldn't imagine filling a 250GB harddrive 15 years ago when 500MB were considered huge.
What will happen? We store our digital photos in raw format, not JPEG. We store our songs in raw format, not artificially crippled. We will store high-definition video, possibly even in raw format, not MPEG4 or the likes.
And, woosh, 1024PB will be nothing leaving us wondering how we could ever survive with a measly 250GB drive -- just as we ask ourselves today how life was with nothing but 170kB disk drives.
Re: (Score:3, Interesting)
I'm not convinced by this myself. I do however see a need for super computers who need to work with filesystems spanning perhaps hundreds of disks. As for the desktop user, even if they did store their files in raw format, I doubt they'd use more than a few 10's of terabytes at the most.
Re:1020 Petabytes? (Score:4, Informative)
framedimensions = 3000 x 2000
framebytes = framedimensions x 6
moviebytes = framebytes x 25 x 60 x 120
moviebytes / 10^12 ~= 6.5
Re: (Score:2)
Re: (Score:3, Informative)
Tom
Re: (Score:2)
Re: (Score:2)
If anything I would say that your resolutions are way to low. For a full wall display of ultra high quality I would guess at 10000x4500 (for roughly anamorphic). I would also up it to 50 FPS for a really smooth display at 48 bpp a two hour movie is ~100TB for the video alone. Trouble wouldn't be storing it - it would be getting it off the device fast enough.
Re: (Score:2)
Most Hollywood movies using digital negatives are burned to film at 2k (~2000 x 1250, varying slightly based on aspect ratio). Postproduction people might use 4k or 6k frames while working with the footage, but that's downsampled on output most of the time. Note that this is not terribly different from the size of a 1080p24 HD frame.
Movies are also generally scanned, mastered, and encoded in 10-bit or 12-bit per channel log. 16 bpc really isn't necessary, as at the
Re: (Score:2)
We're also fairly likely to see 8 Byte/pixel screens, as 64 bit/pixel is an increasingly popular format.
60 frame progressive displays are also quite popular (already), so let's guess that we'll see recording at that speed not far into the future.
And a lot of movies run up to about 2.5 hours, so let's give
Re: (Score:2)
Re: (Score:3, Interesting)
Super computers? Once, maybe - not today, and not for the last decade or so. There are a bunch of companies (I'm working for one of them [netapp.com], now) that will quite cheerfully sell you a storage system that spans hundreds of disks [netapp.com]. Assuming your OS won't flake out when it sees a 500+ TB volume, you could mount it on your desktop, if you want. There's absolutely no need to conflate processing pow
Re: (Score:3, Interesting)
Re: (Score:2)
Unless you are publishing your audio, videos or photos you don't need a 100% representation. Often that 97% quality but 1/10th the size copy will be fine for your own enjoyme
Re: (Score:2)
Re: (Score:2)
mencoder can transcode pretty much anything, unfortunately only into AVI [the MPG output sucks bad], but ffmpeg can do some other streams [notably though it doesn't like 5.1 AC3 streams though
Tom
Convert avi to DVD (Score:2)
Re: (Score:2)
Re: (Score:2)
Heh, I use both, so I can be very flexible in the tools I choose.
Re: (Score:2)
Re: (Score:2)
That's the thing, the "difference" is only in your mind [not sensory]. Recall the whole point of psychoacoustic based encoders is they take advantage of the disparity of S/N ratios on various bands. If you have a 10dB masking on a given band, encoding it with full 96dB range [e.g. 16-bit PCM samples] doesn't make sense.
Imagine you can't see the colour red. Would adding more bits to the red channel make the pic
Re: (Score:2)
Yes, it does, mainly for re-encoding. What if he wants to convert them to 128kbps for a portable player. Or what if in 5 years a different compression is used? Personally, I ripped about 120 of my cds to mp3 this year and it's not something I plan on doing again soon. I can completely understand someone never wanting to have to redo it.
Re: (Score:2)
As for going down in bitrate
Tom
Re: (Score:2)
FLAC is the only logical choice if you think you might have to reencode later. Or burn back to a CD, or remix the track, or make a home movie with it, etc. etc. etc.
Re: (Score:2)
Re: (Score:2)
And what about going up in bitrate? Or switching to a different codec?
128kbit CBR MP3 used to be good enough for everybody and "indistinguishable from the original." Then it was 192kbit VBR mp3. Now it's m4a. What will it be next year? When will the next big lossy audio player roll around with an even better psychoacoustic model? Who knows, but I'm sure there'
Re: (Score:2)
Re:1020 Petabytes? (Score:5, Insightful)
The limits are set by our senses, more concretely, our ears and our eyes.
Our ears are only capable of hearing up to about 20Khz (less than that for most people) and 16-bit samplings are enough that most people cannot hear the difference with anything more. Thus CD-quality is, if not perfect, then good enough that further improvements are ignorable for most people. CD-quality losslessly-compressed music is around 300MB/hour.
In a year, there's 8760 hours, so you'd need on the order of 2.5 TB to store a year worth of around-the-clock never-repeateing losslessly-compressed music. If computers keep getting replaced at the current rate, this means you'll never need more than about 10TB to store sound. This assumes you don't store more than you listen to, if you choose to for example store all music ever produced for convenience, despite never listening to more than a tiny fraction of it, then this requirement goes up by a couple of orders of magnitude. Still, there's good reason to suppose that 10TB will suffice for most peoples sound-storage needs. (even if you wanted to store all the sound you've *ever* heard in your life, including traffic at nigth, that'd still only be 200TB or so)
The real killer is video. We can take in a *lot* more data with our eyes. 10GB/hour is in the ballpark of what you'd need for the sort of quality a modern cinema can deliver. (and there's no particular reason we couldn't go higher.) That works out to 100TB/year, more or less. A lifetime of high-quality video is thus on the order of 10PB.
In short, it is unlikely that an individual (or family) will be able to fill a 1000PB disc with sound and video-recordings. Infact it's unlikely they'll be able to fill it with anything, if that anything is to be consumed only trough their 2 eyes and 2 ears.
That doesn't mean it won't happen. Only that it'll be filled with something more. Once we fire up the holodecks all bets are off. I don't even want to try to estimate the bandwith needed for that kind of immersive experience.
Re: (Score:3, Funny)
People always used to ask me if I had the internet at home. Maybe when I can get my hands on a 1020 PB hard drive, I will be able to download it all for local access...
Re: (Score:2)
Re: (Score:2)
Re: (Score:3, Insightful)
Re: (Score:2)
find "$dir" -amin +"$time" -print0 | xargs -0 rm -fr
Re: (Score:2)
Re: (Score:2, Insightful)
Most of the databases I work on are 10-50Tb.
Initially we built 3Gb filesystems - we couldn't back then up, the sequential file pointer in HPUX can only address 2Tb, which meant we couldn't copy the whole filesystem to tape. I had to rebuild with max 2Tb filesystem.
Then, Veritas Netbackup can only parallelise backups in different directory trees so I was taking ages to perform a full back up - 18 - 19 hours (bit of a bugger in a 24hr backup window).
We don't do incremental backups be
Re: (Score:2)
Filesystems aren't a problem, bigger hard drives aren't a problem, in fact there is no problem at all except people that don't understand the concept of what doing a "backup" actually means.
When we had 1024K drives it would require 1024K to do a FULL backup.
When we had 1G drives it would require 1G to do a FULL backup
When we had 1TB drives it would require 1TB to do a FULL backup
If you can't notice a
Re: (Score:3, Insightful)
First of all, disk drives are advancing faster than tapes.
But the problem is worse than that. Different aspects of disk drives are advancing at different rates. Capacity is increasing faster than interface speed is increasing faster than access speed is increasing faster than block reliability.
Consider an old 500MB drive from the mid-90s; it takes maybe a couple hours to read every block on the drive, and odds are that you won't have bad blocks before the disk dies entirely.
The new
Re: (Score:2)
That means, well before we have common petabyte storage, a terabyte disk will take about 22 minutes to clone. A petabyte disk will then take about fifteen days to copy. But by the time we commonly have petabyte storage, it'll likely take no more than a day at the outside to clone a petabyte disk.
As for bad blocks, use an internal RAID.
Re: (Score:2)
Now, first (for the time being) terabyte storage and certainly petabyte storage is primarily a business (not consumer) requirement/market. In that arena and if you are already going to be paying $$$ for the diskspace, it is almost a no-brainer to be using some sort of RAID. I could just be out of touch with current storage trends, but for *most* systems that I know of they are eit
Re: (Score:2)
That's the bus speed; the disk can't actually sustain I/O at anything approaching those rates. (If you're doing all reads serviced out of the little internal cache on the HDD, sure, but try writing a few hundred megabytes of data and see if you get rates approaching the interface bandwidth.)
Re: (Score:2)
Re: (Score:2)
Re: (Score:2)
Have you heard of porn? Yeah, I didn't think so...
Re: (Score:2)
Re: (Score:2)
Groups of crows? (Score:2)