Forgot your password?
typodupeerror
This discussion has been archived. No new comments can be posted.

Linux Filesystems Benchmarked

Comments Filter:
  • 'Tis a dupe (Score:5, Informative)

    by Quattro Vezina (714892) on Tuesday May 11, 2004 @10:04AM (#9116141) Journal
    The original article [slashdot.org]
  • Dupe (Score:0, Informative)

    by Lars T. (470328) <Lars...Traeger@@@googlemail...com> on Tuesday May 11, 2004 @10:05AM (#9116144) Journal
    Dupe [slashdot.org]
  • by Coryoth (254751) on Tuesday May 11, 2004 @10:12AM (#9116223) Homepage Journal
    Not quite what I got from it. Ext2 was certainly faster for a lot of operations, but is, of course, not journalled. XFS and JFS were fast, but most importantly, when it came to large files, these two tended to really take the lead. XFS was particularly good at handling large files. Overall Ext3 was disappointingly slow surprisingly often.

    Jedidiah.
  • by Anonymous Coward on Tuesday May 11, 2004 @10:14AM (#9116251)
    http://209.81.41.149/~jpiszcz/index.html

    it's not slashdotted yet :)
  • The conclusion (Score:2, Informative)

    by broothal (186066) <christian@fabel.dk> on Tuesday May 11, 2004 @10:16AM (#9116274) Homepage Journal
    Site already /.'ed (when will slashdot ever learn to use a cache - either freecahce or make their own?)

    Anyway, all rants aside, here's the conclusion from the tests (there were some graphs as well but I couldn't make sense of them anyway):

    CONCLUSION

    For those of you still reading, congrats! The conclusion is obvious by the "Total Time For All Benchmarks Test." The best journaling file system to choose based upon these results would be: JFS, ReiserFS or XFS depending on your needs and what types of files you are dealing with. I was quite surprised how slow ext3 was overall, as many distributions use this file system as their default file system. Overall, one should choose the best file system based upon the properties of the files they are dealing with for the best performance possible
  • by Anonymous Coward on Tuesday May 11, 2004 @10:20AM (#9116313)
    Ext3 met Dr. Tweedie's engineering goals. The idea was to develop a journaling file system which was seamlessly compatible with Ext2. Ext3 is really an engineering marvel. You can instantly convert it back and forth between Ext2 an Ext3.

    Ext3 provides a safe low-pain entry into the world of journaled file systems. No need to re-partition or reformat. It offers reasonably good performance plus the benefits of journalling.

  • Re:Slightly OT (Score:5, Informative)

    by Malc (1751) on Tuesday May 11, 2004 @10:21AM (#9116315)
    Obviously (as you point out) a journallying filesystem is what you need. I went for Ext3 on my Debian servers. I/O throughput wasn't so important. The good thing about Ext3 is its backwards-compatibility with Ext2. If there's a problem and you don't have all the kernel modules or tools then you're still pretty much guarranteed access to the file system by mounting it as Ext2 as support for that system is almost universal under Linux.
  • ext3 slowness (Score:5, Informative)

    by ReignStorm (247561) on Tuesday May 11, 2004 @10:23AM (#9116338)
    from Linux ext3 FAQ [sapienti-sat.org]
    Q: How can I recover (undelete) deleted files from my ext3 partition?

    Actually, you can't! This is what one of the developers, Andreas Dilger, said about it: In order to ensure that ext3 can safely resume an unlink after a crash, it actually zeros out the block pointers in the inode, whereas ext2 just marks these blocks as unused in the block bitmaps and marks the inode as "deleted" and leaves the block pointers alone. Your only hope is to "grep" for parts of your files that have been deleted and hope for the best.
  • Re:Slightly OT (Score:3, Informative)

    by Bombcar (16057) <racbmob@bomb c a r . c om> on Tuesday May 11, 2004 @10:34AM (#9116434) Homepage Journal
    Actually, I believe that ext3 is the only filesystem to allow journalling of data above and beyond just metadata.

    Use the mount option data=journal, see man mount for more information.

    I do know that RieserFS has some "features" that are unexpected and can be agrivated by powerfailure during write.

    But don't worry, Hans says it's designed that way, and your filesystem will be intact, even if /etc/fstab contains lines from /var/log/messages......

    XFS is good, but cannot be shrunk. EXT3 can shrink, but I don't know about the others. I'm going to have to investigate JFS, which right now is the bastard stepchild, ignored by most......
  • Re:Slightly OT (Score:3, Informative)

    by SoTuA (683507) on Tuesday May 11, 2004 @10:35AM (#9116441)
    There are mount options that will journal only data, only the metadata (as most journaling filesystems do) or both.
  • Re:Slightly OT (Score:5, Informative)

    by Tet (2721) * <slashdot AT astradyne DOT co DOT uk> on Tuesday May 11, 2004 @10:35AM (#9116446) Homepage Journal
    But I thought Ext3 will only journal metadata

    No, in fact ext3 is one of the few that actually will journal data as well as metadata.

    mount -t ext3 -odata=journal /dev/os/usr /usr
  • Mirror (be kind) (Score:5, Informative)

    by Helmholtz (2715) on Tuesday May 11, 2004 @10:35AM (#9116448) Homepage
    Here's a mirror of the article:

    http://www.gutenpress.org/links/LG/102/piszcz.html [gutenpress.org]
  • by B2382F29 (742174) on Tuesday May 11, 2004 @10:47AM (#9116584)
    PNGs are smaller than GIFs, better compression, etc. (if you use same color-depth (8 bit)).
  • by eddy (18759) on Tuesday May 11, 2004 @10:47AM (#9116593) Homepage Journal

    >Web site accessibility (use image type supported by all major browsers)

    All the "good features" of GIF is supported by PNG in all current browsers. You'd have to go back in time fem years to find a browser that can't display a basic PNG. If you think otherwise, give me a link to one that matters that doesn't, and explain to me why, if it wasn't released/updated this year, using it isn't a security issue.

    Since GIF doesn't support per-pixel-alpha to begin with, you lose nothing by using PNG for everything. After all, with GIF you didn't have the choice at all so there is no issue with simply "converting to PNG".

    Score: PNG

    >Bandwidth conservation

    PNGs are always smaller where it matters (anything more complex than 1x1x1-images). In some not atypical cases a PNG can be 25% smaller than the corresponding GIF.

    Score: PNG

    PS. GIF-via-LZW is still encumbered in many countries.

    More features, better standard, solid software, no licensing issues, smaller output == Winner: PNG

  • by Captain Rotundo (165816) on Tuesday May 11, 2004 @10:51AM (#9116621) Homepage
    name a "major browser" that won't support PNG. I don't know one. I use all PNG and have checked all my pages in enough "major" browsers to cover probably greater than 99% of people and they display fine.
  • by Anonymous Coward on Tuesday May 11, 2004 @10:51AM (#9116631)
    ...making Linux just a little more fun!

    Benchmarking Filesystems
    By Justin Piszcz

    INTRO
    I recently purchased a Western Digital 250GB/8M/7200RPM drive and wondered which journaling file system I should use. I currently use ext2 on my other, smaller hard drives. Upon reboot or unclean shutdown, e2fsck takes a while on drives only 40 and 60 gigabytes. Therefore I knew using a journaling file system would be my best bet. The question is: which is the best? In order to determine this I used common operations that Linux users may perform on a regular basis instead of using benchmark tools such as Bonnie or Iozone. I wanted a "real life" benchmark analysis. A quick analogy: Just because the Ethernet-Over-Power-Lines may advertise 10mbps (1.25MB/s), in real world tests, peak speed is only 5mbps (625KB/s). This is why I chose to run my own tests versus using hard drive benchmarking tools.
    SPECIFICATIONS
    HARDWARE
    COMPUTER: Dell Optiplex GX1
    CPU: Pentium III 500MHZ
    RAM: 768MB
    SWAP: 1536MB
    CONTROLLER: Promise ATA/100 TX - BIOS 14 - IN PCI SLOT #1
    DRIVES USED: 1. Western Digital 250GB ATA/100 8MB CACHE 7200RPM
    2. Maxtor 61.4GB ATA/66 2MB CACHE 5400RPM
    DRIVE TESTED: The Western Digital 250GB.

    SOFTWARE
    LIBC VERSION: 2.3.2
    KERNEL: linux-2.4.26
    COMPILER USED: gcc-3.3.3
    EXT2: e2fsprogs-1.35/sbin/mkfs.ext2
    EXT3: e2fsprogs-1.35/sbin/mkfs.ext3
    JFS: jfsutils-1.1.5/sbin/mkfs.jfs
    REISERFS: reiserfsprogs-3.6.14/sbin/mkreiserfs
    XFS: xfsprogs-2.5.6/sbin/mkfs.xfs

    TESTS PERFORMED
    001. Create 10,000 files with touch in a directory.
    002. Run 'find' on that directory.
    003. Remove the directory.
    004. Create 10,000 directories with mkdir in a directory.
    005. Run 'find' on that directory.
    006. Remove the directory containing the 10,000 directories.
    007. Copy kernel tarball from other disk to test disk.
    008. Copy kernel tarball from test disk to other disk.
    009. Untar kernel tarball on the same disk.
    010. Tar kernel tarball on the same disk.
    011. Remove kernel source tree.
    012. Copy kernel tarball 10 times.
    013. Create 1GB file from /dev/zero.
    014. Copy the 1GB file on the same disk.
    015. Split a 10MB file into 1000 byte pieces.
    016. Split a 10MB file into 1024 byte pieces.
    017. Split a 10MB file into 2048 byte pieces.
    018. Split a 10MB file into 4096 byte pieces.
    019. Split a 10MB file into 8192 byte pieces.
    020. Copy kernel source tree on the same disk.
    021. Cat a 1GB file to /dev/null.

    NOTE1: Between each test run, a 'sync' and 10 second sleep were performed.
    NOTE2: Each file system was tested on a cleanly made file System.
    NOTE3: All file systems were created using default options.
    NOTE4: All tests were performed with the cron daemon killed and with 1 user logged in.
    NOTE5: All tests were run 3 times and the average was taken, if any tests were questionable, they were re-run and checked with the previous average for consistency.

    CREATING THE FILESYSTEMS

    (snipped. too many junk characters0

    BENCHMARK SET 1 OF 4

    In the first test, ReiserFS takes the lead, possibly due to its balanced B-Trees. (If the images are hard to read on your screen, here's a tarball containing larger versions of them.)

    All of the files systems faired fairly well when finding 10,000 files in a single directory, the only exception being XFS which took twice as long.

    Both ext versions 2 and 3 seem to reap the benefits of removing large numbers of files faster than any other file system tested.

    To make sure this graph was accurate; I re-benchmarked the ext2 file system again and got nearly the same results. I was surprised to find how much of a performance hit both ext2 and ext3 take during this test.

    Finding 10,000 files seemed to be the same except for XFS; however directories are definitely handled dif
  • XFS, solid (Score:2, Informative)

    by Anonymous Coward on Tuesday May 11, 2004 @10:57AM (#9116684)
    I have been using XFS on a Laptop for 2+ years who's hardware was bleeding edge at the time of purchace.

    Because of hardware/configuration issues, I have had to hard-reboot the laptop countless times during the months that hardware support in the kernel caught up. (It works pretty well now).

    I have never borked my filesystem.
  • by vorwerk (543034) on Tuesday May 11, 2004 @10:58AM (#9116699)
    Redhat 9 and Fedora Core 1 both ship with JFS support -- the graphical installer, however, does not offer it as an option.

    So, what I usually do when installing a new copy of Fedora or Redhat is to drop to a console, and use fdisk + mkfs.jfs manually. Then, when I get to the right page in the installer, I can simply set it to not reformat the partition but to use it as the "/" mount point, and voila -- my computer has JFS.
  • ext3 options (Score:5, Informative)

    by kardar (636122) on Tuesday May 11, 2004 @11:01AM (#9116736)
    There are options, or settings, that you can do for ext3, the default is slower, but it saves your data. Ext3 not only journals metadata, like XFS, etc... but it also journals data, which is the only filesystem that does that, if I understand this correctly.

    "data=writeback" mode does no data journaling, only metadata journaling, and you would probably see better performance here. Although, you could lose data in the event of a power outage (no fun). Same thing applies to XFS, JFS - you could lose data because only metadata is being journaled, not real data.

    "data=ordered" mode - inbetween, still no data journalling, but there are provisions that make it less likely to lose data in the case of a power problem. It has something to do with the way it journals the metadata and the way the filesystem interacts with the disk that makes is a little slower than data=writeback but also a little more secure than data=writeback if you get a power outage.

    "data=journal" mode - this journals data and metadata, and with the exception of a few situations, is the slowest. The least likely to lose your data, but also much slower.

    I am assuming, or at least it looks like, these tests were run with the default data=journal - so to be fair, they should have been run in data=writeback, or maybe even all three modes. Again, all you have to is specify in /etc/fstab and reboot, no big deal.

    It would probably be better to compare the ext3 in data=writeback mode.

  • by Erwos (553607) on Tuesday May 11, 2004 @11:05AM (#9116761)
    "I've been using ReiserFS _EXCLUSIVELY_ since about 2.4.11 and I've never had a single problem. It's important to format with the defaults and not specify 'special' arguments to mkreiserfs or you can run into trouble."

    I've personally had several friends tell me of their data loss with ReiserFS. No one's arguing that it's a horrible file system, only that it still experimental.

    The typical data loss situation is a power loss in the middle of a write. ReiserFS might be atomic in operation, but it still can't dodge hardware failure at that level.

    I use ext3, and I've been happy. ReiserFS is definitely not an appropriate default partition type at this time.

    -Erwos
  • by 13Echo (209846) on Tuesday May 11, 2004 @11:11AM (#9116817) Homepage Journal
    Most people don't have their /home directory tied into their root partition. Often, they are even on seperate drives. Even if a defrag program were used, there would be very little benefit. You're not constantly writing new files in and out of the same space as the root partition in that respect.

    And even if someone does put their /home directory on the root partiton, the modern Linux filesystems practically negate the need for defragmentation, due to their designs (as well as OS and drive design).
  • by jcupitt65 (68879) on Tuesday May 11, 2004 @11:13AM (#9116834)
    I've lost stuff with reiser too. About a year ago I was fiddling with an nvidia driver and locked my machine up. When I rebooted the tree that had had a compile going on had vanished forever.

    My understanding is that journalled means the FS can't get into an inconsistant state and so does not need fsck-ing. It does not mean your data is safe from having the power pulled halfway through a write. If you want a super-safe home area you want some sort of logging FS and these are all far slower than journalled (I think).

  • by quake74 (466627) on Tuesday May 11, 2004 @11:13AM (#9116838)
    name a "major browser" that won't support PNG. I don't know one. I use all PNG and have checked all my pages in enough "major" browsers to cover probably greater than 99% of people and they display fine.
    Did you check transparency in Windows Explorer? As far as I know, it's not supported, unless you play some weird tricks [koivi.com]. Or maybe you were making a joke, and I didn't get it.
  • Re:Hrmmm (Score:3, Informative)

    by Bob Uhl (30977) <eadmund42&gmail,com> on Tuesday May 11, 2004 @11:21AM (#9116926) Homepage
    Speaking of which, there has got to be better graph making software out there in Linuxland...

    There is: gnuplot [gnuplot.info], an utterly wonderful little program. I use it all the time.

  • by dbIII (701233) on Tuesday May 11, 2004 @11:24AM (#9116955)
    And they don't mention reinstallation times (measured in hours) which occurs for ext2 a lot more than the journalling filesystems :-)
    What the fsck are you talking about? When the filesystem has problems you just need to run fsck like on any unix system from the last decade and more instead of doing the windows thing of reformatting and re-install. Certainly it's a scary thing, but you don't have to throw everything away just because you get a few easily fixable filesystem errors.
  • by tuffy (10202) on Tuesday May 11, 2004 @11:29AM (#9116994) Homepage Journal
    IE is still too stupid to properly do an alpha channeled PNG. But it does do 1-bit, GIF-style transparency and displays generic, non-transparent PNGs just fine. And so the only place left to use GIF for is crummy animations.
  • I did some too (Score:5, Informative)

    by Rufus211 (221883) <rufus-slashdot@h ... g ['kis' in gap]> on Tuesday May 11, 2004 @11:30AM (#9117007) Homepage
    I did a bunch of tests like this, but in 2.6 instead of 2.4. My conclusions:

    * Ext2 is still overall the fastest but I think the margin is small enough that a journal is well worth it
    * Ext3, ReiserFS, and XFS all perform similarly and almost up to ext2 except:
    o XFS takes an abnormally long time to do a large rm even though it is very fast at a kernel `make clean`
    o ReiserFS is significantly slower at the second make (from ccache)
    * JFS is fairly slow overall
    * Reiser4 is exceptionally fast at synthetic benchmarks like copying the system and untaring, but is very slow at the real-world debootstrap and kernel compiles.
    * Though I didn't benchmark it, ReiserFS sometimes takes a second or two to mount and Reiser4 sometimes takes a second or two to unmount while all other filesystem's are instantaneous.

    Whole thing available here [cmu.edu]
  • by Trailer Trash (60756) on Tuesday May 11, 2004 @11:35AM (#9117053) Homepage
    PNG's offer nothing over GIF's for images such as his, which can use a 16-color colormap. GIF's are readable by far more people, though.
  • by mirror_dude (775745) on Tuesday May 11, 2004 @11:35AM (#9117054) Homepage
    Just in case people want to read the article and dont have 30 minutes for it to load I put up a mirror here [demonmoo.com] ...
  • Re:ext3 options (Score:4, Informative)

    by CmdrTHAC0 (229186) on Tuesday May 11, 2004 @11:35AM (#9117062)

    "I am assuming, or at least it looks like, these tests were run with the default data=journal - so to be fair, they should have been run in data=writeback, or maybe even all three modes. Again, all you have to is specify in /etc/fstab and reboot, no big deal."

    And where do you get the idea that this is the default? According to mount(8):

    ordered

    This is the default mode.

    What I really would have liked to see on this benchmark is ext3 on 2.6 with dir_index enabled. (Maybe this would also gain the benefit of the Orlov allocator? I haven't been paying attention to what's been backported.) ...In fact, I would have liked to see this whole thing on 2.6.

  • by AGTiny (104967) on Tuesday May 11, 2004 @11:45AM (#9117136)
    ext3 does NOT journal data by default, it only "orders" it (data=ordered). This is still better than what any of the other filesystems do. You have to explicity define data=journal in fstab to get full data journaling. There is a performance hit but it's what I use on all my RAID5 data filesystems and it's great. I used to use reiserfs until a huge crash corrupted massive amounts of files and directories. I don't think I'll be going without data journaling protection for a while.
  • 2nd Opinion... (Score:2, Informative)

    by mikegi (187558) <grid@@@iki...fi> on Tuesday May 11, 2004 @11:46AM (#9117149) Homepage
    At http://fsbench.netnation.com/ you'll find a nice python tool that will run Bonnie and/or IOzone with different parameters and stick the results in a MySQL [mysql.org] database and make nice little tables from the data.

    There is also some commentary and recommendations based on their results.

    One more note about the tool... it's not well documented but works well when configured... note that you need a kernel that supports the filesystems to be tested (duh!), Use python 2.2, the database schema is somewhere in the comments.

  • by Anonymous Coward on Tuesday May 11, 2004 @11:49AM (#9117172)
    Don't forget to limit the PNG to 16 or 256 colours if only that many are being used. Needless to say, it makes a big difference in file size compared to using true color.
  • by Anonymous Coward on Tuesday May 11, 2004 @11:49AM (#9117176)

    Not every graphics program can export to PNG, and the ones that do cannot always do it in a way that is smaller than their GIF export. GIF is a much older and better supported standard, which is why most people use it over PNG.

  • by ImpTech (549794) on Tuesday May 11, 2004 @11:50AM (#9117179)
    Not to mention mountable under Windows... for those of us that still need that. Or rather, EXT2 is mountable under Windows (with a 3rd party daemon), and thus EXT3 can be read as well.
  • by Anonymous Coward on Tuesday May 11, 2004 @11:53AM (#9117214)

    Did you check transparency in Windows Explorer? As far as I know, it's not supported, unless you play some weird tricks. Or maybe you were making a joke, and I didn't get it.

    PNG transparency works fine in IE as long as you don't try to do partial transparency. For simple on/off transparency (the same as what GIF offers), there are no problem with IE5 and up.

    The article you linked to was describing a solution to getting partial transparency working.

  • by G00F (241765) on Tuesday May 11, 2004 @12:01PM (#9117295) Homepage
    I agree, and after having bad blocks with a reiserfs system, I don't want to touch reiserfs again. reiserfs has no way to deal with bad blocks. Only a hack that you have to implament everytime your system does a fsck. You basicaly fake it that those bad blocks have a file.
    Reiserfs +IBM HD's = hair loss
  • by chthon (580889) on Tuesday May 11, 2004 @12:09PM (#9117401) Homepage Journal

    Please read this [oracle.com]

    Just to show that it depends upon the application you need to run.

    Now, you will not hear me say that you should not use ReiserFS, for a desktop it is probably the best choice, but for servers, please think again.

    In addition to that, my own experiences with ReiserFS are mostly positive, especially on my 233 Mhz laptop, but I have also a big system with a Promise SX6000 raid controller, where I had a partition with ReiserFS, ext3 and JFS using Red Hat 9. Everytime I did file operations using ReiserFS I got problems with the consistency of my RAID 5 array, so I scrapped ReiserFS and only kept ext3 and JFS, which give me no problems anymore.

  • by Dave2 Wickham (600202) * on Tuesday May 11, 2004 @12:14PM (#9117450) Journal
    1) I often find PNGs to be smaller than GIFs.
    2) Who cannot see PNGs? IE supports them, Opera supports them, Netscape/libpr0n [libpr0n.com]-based browsers all support PNGs. Hell, even Links 2 when run in X or svgalib supports PNG.
  • by jargoone (166102) * on Tuesday May 11, 2004 @12:19PM (#9117506)
    So, what I usually do when installing a new copy of Fedora or Redhat is to drop to a console, and use fdisk + mkfs.jfs manually

    You're making things unnecessarily complicated. At the "boot:" prompt, just type "linux jfs". The graphical installer will then offer it as an option. Works with reiserfs, too.
  • by Etyenne (4915) on Tuesday May 11, 2004 @12:25PM (#9117606)
    PNG transparency works fine in IE as long as you don't try to do partial transparency. For simple on/off transparency (the same as what GIF offers), there are no problem with IE5 and up.

    More precisely, the PNG need to be in indexed mode (aka PNG8) for full transparency to work in IE. In The GIMP, click the "Image" menu, "Mode", "Indexed".

    Some myth ("IE don't support PNG !!!") really die hard.

  • by ArkiMage (578981) on Tuesday May 11, 2004 @12:34PM (#9117725) Homepage
    I no longer will even attempt to use EXT3 for a filesystem ~500GB or larger. Had problems repeatedly with a couple of 1.6TB filesystems and switched to Reiser3 and haven't looked back...

    # df -k
    Filesystem 1K-blocks Used Available Use% Mounted on
    /dev/sda6 4032092 548856 3278412 15% /
    /dev/sda1 505605 19171 460330 4% /boot
    /dev/sda7 41729164 31165820 8443572 79% /data
    none 2000568 0 2000568 0% /dev/shm
    /dev/sda3 10080520 6437952 3130500 68% /usr
    /dev/sda2 80636072 45306536 31233364 60% /var
    /dev/sdb1 1600772128 1462009904 138762224 92% /raid1
    /dev/sdc1 1600772128 760247416 840524712 48% /raid2

    PS. No, it's not pr0n
  • by lecca (84194) on Tuesday May 11, 2004 @01:02PM (#9118080) Homepage

    "Overall Ext3 was disappointingly slow surprisingly often."

    I disagree, plus this test is obsolete, why did he use a 2.4 kernel?!

    from: http://freshmeat.net/projects/linux/?branch_id=463 39&release_id=160407 [freshmeat.net]

    "Linux 2.6.6
    [...]
    Changes:
    ...ext2 and ext3 filesystem performance was significantly improved. "

    And thats just from today's kernel release. What about all the changes between 2.4 and now?

    Considering the conveniance of backward compatability, and the fact that ext3 wasn't the worst in every category, and it looks like maybe uses less cpu than some, it seems like ext3 is the hands-down winner of the test, not the looser. ext3 did as well in tests that IMO represent everyday use. Who creates 10k files in a folder? I would have liked to see a linux kernel COMPILE using the fs. Thats something we all could appreciate.

  • by 13Echo (209846) on Tuesday May 11, 2004 @01:29PM (#9118452) Homepage Journal
    I'm sorry, but you can't make a statement like that and not back it up. How exactly is this the case? Even if it really is true, how can it hurt to have a defrag tool available? Is it really true that there is _NEVER_ instances where a Linux filesystem can be so fragmented as to negatively impact performance? Let's try to be logical, shall we?


    Why is it hard to understand that UNIX filesystems are designed to have minimal impact on performance due to their design?

    These are multiuser operating systems that are designed to make frequent requests from multiple users at any given time. Things *are* going to be strewn across the drive, but there is a reason that there is no noticable impact on performance.

    UNIX filesystems are engineered to avoid appending old files and scattering data about in the same manner that MSDOS and Windows FAT filesystems do. These filesystems don't fill every single free "crack" on the drive in the way that MSDOS filesystems do. FAT filesystems are designed to write into the first available location, or "hole", often spanning across several of these as well, for writing a single file. This is what causes the "fragmentation" on a Microsoft filesystem. The clustering algorithms that UNIX/Linux machines use use help to prevent "fragmentation", by which Windows users expererience.

    Bear in mind that FAT/FAT32 was based off of a design that was optimized for writing small amounts of data to *floppy disks* and small capacity drives with very limited amounts of space. Later in the life of DOS and Windows, the "fragmentation" issues became terrible, typically as drive capacities got to be larger and filesizes increased by a great degree. NTFS resolves many of these issues, but still carries a few of the FAT traits in spite of it being a totally different filesystem (based off of HPFS). Potentially, it still writes to tiny, empty, blocks of free space, but its tree-based structure doesn't limit the performance due to "fragmentation" like we experienced on DOS/Win9x. However, I think that the biggest problem on Windows machines is the way drives are typically partitioned, more than anything else. Things get removed and installed frequently, to the same locations of the drive, with user-created data overlapping the locations of important system and swapfiles. NTFS, in most respects, doesn't actually need defrag. In fact, when I ran Windows 2000 for a few years, defrag provided almost no improvement, at least not to the same degree as it did on Windows 98. You can defrag all you like, but it's unlikely that even an NTFS partition will experience more than 3-5% total fragmention.

    I hope that is "logical enough" for you. I think that, perhaps, you need to ditch the old DOS/Windows "I MUST DEFRAG" mentality in order to really understand this. Filesystems (especially journalling types) have greatly changed since the days of DOS.
  • by harlows_monkeys (106428) on Tuesday May 11, 2004 @01:30PM (#9118463) Homepage
    Gah! The charts are shrunk so that the labels are hard to read, and the order of the results and color assigned to each FS seems to have been picked randomly for each chart, so once you squint and decipher one of them...you have to start over on the next.
  • by Sxooter (29722) on Tuesday May 11, 2004 @01:45PM (#9118622)
    Nice article, thanks for the link.

    It looks like Oracle has gotten the same basic results as the PostgreSQL Global Development Group have. JFS and ext3 are generally the fastest under a database, while XFS and Reiser seem to be pretty slow.

    The odd thing here is that most other tests show XFS and Reiser as the kings. But the kind of random access databases do seems to be better handled by JFS / ext3.

    The problems with your RAID consistency, were these file system problems, or RAID level problems? If they're RAID level problems that would seem to point at your RAID controller, as no file system should be able to bonk a RAID array on the head, since it doesn't really have that kind of access to it.
  • by mindriot (96208) on Tuesday May 11, 2004 @01:49PM (#9118665)

    Reiserfs can at least be accessed [p-nand-q.com] under Windows.

    My personal peeve with ReiserFS is, though, that I've had the main ReiserFS partition on my Laptop completely destroyed by a simple power failure once. Many files were in lost+found afterwards, but some had their contents destroyed. (And restoring files by looking at their contents is not fun for ~1000 files...) So I've kinda lost trust in it. ext3 may be slow, but I've never had a single problem with it. Seems very robust to me. So, reliability is more important than speed for me (whoever needs performant servers is of course entitled to a different opinion). XFS and JFS seemingly can't be accessed from Windows, so that is a Minus for some.

  • Re:'Tis a dupe (Score:2, Informative)

    by chrwei (771689) on Tuesday May 11, 2004 @02:43PM (#9119187)
    something is missing from these tests that I've informaly tested myself before. I'd like to see what this guys scores are after using the HD for a couple months. Let a little fragmentation in then see what they do. ReiserFS will hold up much better in the long run. I used xfs ONCE, after a month my PC was slower than a year old never defraged install of NT4. used a spare HD to migrate to reiser and no probs 2 years later. And for the whinners complaining of power loss, get a battery! best $100 i ever spent.
  • large file support (Score:4, Informative)

    by David Jao (2759) * <djao@dominia.org> on Tuesday May 11, 2004 @03:30PM (#9119642) Homepage
    I'm rapidly approaching the point where I need support for file sizes greater than 2GB. Quite frankly, most of what I've found about file sizes and file systems is 2 to 4 years old... Everyone's too concerned with speed!

    Here's some real world information on the state of large file support in 2004. Filesystem driver support is the least of your worries -- almost any linux filesystem you can think of (except for maybe umsdos) supports over 2GB files at the kernel level. The Linux LFS [www.suse.de] page, dated April 2004, contains reasonably updated information on large file support in linux.

    The bigger problem is that many userspace applications cannot yet read or write to the large files. This failure arises from non-use of the LFS API combined with just plain unfortunate use of a signed 32-bit int in the wrong place. So for instance mkisofs will reject all input files larger than 2GB in size, and cdrdao will simply abort at 2GB if you try to rip a DVD larger than 2GB in size. In some extreme cases there are programs that can't even handle large files off of the disk; one example is

    wget http://mirror.linux.duke.edu/pub/fedora/linux/core /test/1.92/i386/iso/FC2-test3-i386-DVD.iso

    which fails spectacularly on any x86 linux system (hint: the DVD is not really 84MB in size). In general, the "core" system utilities such as dd, cp, mv, cat are fully compatible with large files whereas third party applications are much more hit-or-miss.

    Even today, by far the most practical solution to large file woes is to migrate to a 64-bit system, the most affordable of which is AMD64 by a long shot. I've been using an Athlon 64 system for the past few weeks and it has handled large files perfectly in all respects so far.

  • Re:The conclusion (Score:3, Informative)

    by avdp (22065) * on Tuesday May 11, 2004 @06:04PM (#9121186)
    No problem, it's right in the slashdot FAQs [slashdot.org]

  • by Anonymous Coward on Tuesday May 11, 2004 @09:21PM (#9123060)
    Bizarre! I've used Reiserfs since 2000. Never had a problem. If the nieces and nephews yank the plug or shut off the box, it takes 3 seconds for the system to check/rebuild 80GB hard drive, 5 seconds to rebuild a 160GB hard drive. I don't know how your system went down, but I've never seen Reiserfs do what you describe. The system always checks itself. I've never had to go 'looking' for a file. I don't know how you lost files. I've just never seen it.
  • by arcade (16638) on Wednesday May 12, 2004 @02:06AM (#9124320) Homepage
    My personal peeve with ReiserFS is, though, that I've had the main ReiserFS partition on my Laptop completely destroyed by a simple power failure once.

    I've been experimenting by using ReiserFS on and off for the last 3 years or so. I've always shied away after a few failures.

    My scorebook so far:
    - Laptop /home partition went to hell twice, at power failure.
    - Two various machines at previous work got open files in /home partition smothered at power failure, had to rm -rf .kde for the system to get up'n running again.
    - Mums computer. Had to travel 500km to fix a reiserfs fuckup due to repeated power failures.
    - Dad's laptop got a partition trashed by reiserfs when he forgot to put in his power cord and the battery time were used up.

    Reiserfs is the single most unstable piece of shit of a filesystem I've ever had to deal with. No, I'll not be using it again anytime soon.

"Wish not to seem, but to be, the best." -- Aeschylus

Working...