A Short History of Btrfs 241
diegocgteleline.es writes "Valerie Aurora, a Linux file system developer and ex-ZFS designer, has posted an article with great insight on how Btrfs, the file system that will replace Ext4, was created and how it works. Quoting: 'When it comes to file systems, it's hard to tell truth from rumor from vile slander: the code is so complex, the personalities are so exaggerated, and the users are so angry when they lose their data. You can't even settle things with a battle of the benchmarks: file system workloads vary so wildly that you can make a plausible argument for why any benchmark is either totally irrelevant or crucially important. ... we'll take a behind-the-scenes look at the design and development of Btrfs on many levels — technical, political, personal — and trace it from its origins at a workshop to its current position as Linus's root file system.'"
Looks promising (Score:5, Informative)
This looks like a promising filesystem - as ZFS on linux is, at present, doomed to die an ugly death, btrfs looks to address a lot of the shortcomings of other filesystems and bring a clean, modern fs to linux. It goes beyond ZFS in some areas too, such as being able to efficiently shrink a filesystem, and keeps a lot of the cool things that ZFS made popular, such as Copy-On-Write.
It looks like Btrfs also addresses some decisions that were made with the direction that ZFS would be going in, or how it would handle certain problems that now with hindsight behind the developers, they possibly would have done things differently.
Apple are really struggling with ZFS, with it being announced as a feature in early betas of both Leopard (10.5) and Snow Leopard (10.6), as well as being there in a very limited form in Tiger (10.4) - maybe development on Btrfs will leapfrog ZFS for consumer-grade hardware and Apple can finally look at deprecating HFS.
Re:So, what is the status of btrfs? (Score:5, Informative)
The article predicts a couple of years until it's safe enough as default in new distros.
Re:So, (Score:5, Informative)
ZFS has built in RAID support (which, I assume, works on the block level, instead of on the disk level), maybe Btrfs will get this too.
Yes, btrfs currently has built-in support for raid 0/1/10, 5 and 6 are under development.
Re:Looks promising (Score:3, Informative)
... but btrfs is GPL. Therefore Apple can't use it, unless perhaps they are able to work out licensing from Oracle.
Re:Looks promising (Score:5, Informative)
Apple has, and does, use GPL'd code and complies with the terms of the license.
Take, for example, WebKit, which is a fork of KHTML. It's now released as LGPL:
http://webkit.org/coding/lgpl-license.html [webkit.org]
This code powers the browser that Apple ship with Mac OS X, Safari - which is arguably one of the most important pieces of code in the whole OS.
As a result of it's quality, speed and standards adherence, it's now used by companies like Nokia and Adobe...
Re:Looks promising (Score:1, Informative)
A browser is probably easier separated from the rest of the OS than a filesystem. (Well, i'm not talking about M$IE).
Re:So, (Score:5, Informative)
What you do know is that when you read a block of data back from the disk, that block is what was supposed to be written to the disk.
If a file that is never read is corrupted somehow, then you will only discover that corruption when you read the file.
Having checksums is very good if you have a RAID-1 mirror. With full block checksums, you can read each half of the mirror and if there is an error, you know which one is correct, and which one isn't. At present, if a RAID-1 mirror has a soft error like this, due to corruption, you don't know which half of the mirror is actually correct.
With ZFS, for instance, you can create a 2-disk RAID-1 mirror and then use dd to write zeroes to one half of the mirror, at the raw device level (ie, bypassing the filesystem layer) and when you go to read that data back from the mirror, ZFS knows that it's invalid and instead uses the other side of the mirror. It then has an option to resilver the mirror and write the valid data back to the broken half, if you so want.
Re:Looks promising (Score:3, Informative)
I think that since it's a part of the kernel, it would count as a derivative work which would mean the whole kernel would have to be GPL'd as well.
This is similar to the reason that ZFS can't just be ported to linux, the code is under CDDL which is incompatible with GPL.
Re:So, (Score:2, Informative)
As far as I know, the Windows IFS development kit is not free, neither as in speech nor as in beer.
You can download the Windows Driver Kit for free, and that includes the Installable Filesystem Kit headers and libraries and the source code to FASTFAT.SYS, the Windows FAT driver and CDFS.SYS, the ISO9660/Joliet filesystem.
http://www.microsoft.com/whdc/DevTools/WDK/WDKpkg.mspx [microsoft.com]
That being said writing a performance filesystem for Windows is much less easy than for Linux.
Re:So, what is the status of btrfs? (Score:5, Informative)
Just because a replied to your snarky message with another equally snarky one, doesn't mean I'm not able to put it into words. For instance, a few reasons why I prefer Linux over *BSD or Solaris:
- better package management
- better hw support
- better ISV support
- the uncertain future of Solaris (after all, Sun got bought because they were bleeding red ink left and right, will the Solaris devs escape the inevitable layoffs and Oracle continue pumping money into Solaris development just to try to keep up with Linux?)
- Lack of tier-1 commercial support for *BSD.
- Much larger community
- Better availability of qualified Linux sysadmins
Re:Looks promising (Score:4, Informative)
The GPL and LGPL are very different. The LGPL does not affect any code beyond that originally covered by the license. You can link LGPL'd WebKit against proprietary-licensed Safari with no problems.
Apple also ship GPL'd software like bash, but they don't link it against any of their own code.
Linking GPL'd code into the kernel would require the rest of the kernel to be released under a license that places no restrictions that are not found in the GPL. That's not a problem for Apple's code; they own the copyright and they can release it under any license they choose. It would be a massive problem for third-party components. DTrace, for example, is heavily integrated into OS X's Instruments developer app and is CDDL (GPL-incompatible). Various drivers are under whatever license the manufacturers want, and are mostly GPL-incompatible. A GPL'd component would need to be very compelling to make Apple rewrite DTrace, most of their drivers, and a lot of other components. Btrfs is not this compelling. Even if Btrfs were sufficiently good, it would take less effort for them to just completely rewrite it than to rewrite all of the GPL-incompatible components.
Re:So, what is the status of btrfs? (Score:4, Informative)
For hardware support it really depends what segment of the market you are arguing about. If you are talking white box, low end mostly self supported stuff then no doubt, Linux wins hands down. But as a sysadmin I find Linux to be the of the most painful platform to work on compared to Solaris or AIX - predominantly because of the lack of standardised, stable and properly supported management interfaces.
Fibre channel support is a joke. Sure, for the most part you can dynamically bring stuff in and out, and udev goes a short way to bringing some consistancy. The problem is when something goes wrong you are left with pretty much just rebooting - messages tell you nothing - is the device there or not? Usable details are buried away in /proc and /sys and typically are only useful for developers. Solaris and AIX had cfgadm/cfgmgr and lsdev and friends to tell you what state things are in or what has happened. There are useful and informative error messages (typically). So far on RHEL 3/4/5 all I ever see is odd octal dumps from drivers when errors occur, and wierd hangs and IO errors when devices get broken. It gets worse as you change fibre drivers and versions. Options which exist in one disappear in others. Vendor drivers add customisations which cause other issues.
The lack of stablity in terms of being able to do things between versions gets me as well. On AIX/Solaris you write a script for Solaris 8, and it just works going forwards to other versions. Solaris 10 changes things a bit, but for the most part you can still poke around the same places or the same way to get info back. In short they tend not to break things that work.
Linux goes the other way - a change is made, and thats that, it seems to be up to you to either track or figure it out. You find yourself having to customise things for many many variations of platform - not just major versions, but minor versions as well. Changes to config file locations, the ways those files are defined etc.
Don't get me wrong, I got into UNIX on Linux and I wont dispute its strength in drivers or community, but that community is not "Enterprise" focused. Its why I use it for my PVR and not my file server. The rapid changes in Linux are why the DVB-T cards I got became supported so quickly after the hardware changed. I get the differences, but its not one size fits all.
Re:So, what is the status of btrfs? (Score:4, Informative)
Fail.
1. Not available on the majority of Linux installations
Something similar seems to be available in APT:
http://www.debian.org/doc/manuals/apt-howto/ch-apt-get.en.html [debian.org]
Check section 3.10.
And here's the rough equivalent for RPM:
http://www.linuxjournal.com/article/7034 [linuxjournal.com]
So, what distro is no longer covered?
2. Removing a package is not the same as reverting to an earlier version of the same package.
I guess that you missed the latter half of the last command that I posted:
# emerge -C =package-cat/package-offending-version && emerge package-cat/package
An English translation of that command is
Remove the offending package and install the latest available that's not masked if the removal was successful.
I could have written that command as:
# emerge package-cat/package
and -as I had previously masked the offending package version- Portage would have done the right thing.
So, in summary:
No, you're a towel.
:D
Re:oh wee sun's sloppy seconds. (Score:2, Informative)
No. I was alluding to GP's failure to make a good argument for supporting ZFS, using sound reasoning. I'm saying the truth of the premises doesn't imply the truth of the consequent.
ZFS on supported hardware is actually superior to ext4 on certain technical merits; primarily data integrity (checksumming), random write performance, and read performance (when massive amounts of RAM are available), and more advanced features (snapshots).
On the other hand, ext4 works on 32-bit processors (ZFS is only recommended to be used on 64-bit procs) with small amounts of RAM available, less than a GB; the minimum amount of RAM one should use ZFS with is 2gb, and 4gb or more is strongly recommended, above and beyond any RAM required by apps running on the machine.
But that has little to do with the OS being produced by a large corporation.
Re:Duh... (Score:5, Informative)
Wow. FUD flies fast and hard on slashdot. Zealots? Are you serious? Rather than mod your post as +1 Funny, I think I'll blow some karma and respond, just to set the record straight.
Laying aside misconceptions about the GPL, the main reason BtrFS is GPL is because it's part of the Linux kernel which is also GPL! How hard is it to grasp that? If Apple or anyone else wants to license Oracle's BtrFS code, they are welcome to negotiate and get the code under a different license than the GPL. It's that simple. BtrFS is an implementation of an idea, a specification. If Apple wants to write their own BtrFS driver, they are welcome to do that. Or Microsoft.
Why are developers who don't want their code to be ripped off (used without payment in a closed product) by companies and incorporated into a product are labeled zealots? How is this different than software companies requiring code to be licensed by third parties? So a company who creates some really cool technology that they license for a fee to others for use in products zealots? There really is no difference.
While I haven't written any software of note, I also use the GPLv2 (evaluating v3) since I want my software to be able to be freely used by those that want to use it, but if my code is that valuable to a company, I want to get paid for my trouble. If no one is willing to pay me, then that's fine. They are welcome to use my software without restriction, but if they redistribute it, to do so under the terms of the GPL. Guess that makes me a zealot.