Which Filesystem Do You Use On Portable Media For Linux Systems?

An anonymous reader writes "Most people use MS filesystems on Disk-On-Keys, and portable hard drives, as these are readable from most machines. But this way you lose the files' permission information, which many times is very inconvenient (you must agree that having Ubuntu asking you whether to execute or display every text file or image you open from a DOK is annoying). Using 'regular' Linux filesystems like ext keeps the permissions, but may require using the superuser when switching machines (as the UIDs are different). So do any of you have a creative solution for this problem?"

Poratibility

[IANAAC]

Since we're talking about portable media, I want it portable and use fat32. That way I know I'll be able to use it pretty much everywhere.

All my systems at home are Linux-based, ext3. NONE of my neighbors, family, or work associates have that, so it's a no-brainer.

UID's

[aashenfe]

One of the annoying things about User ID's is that most Distros user utilities start at some number and count up. Then when you use nfs or removable media you find that the files are now owned by another user.

It would be nice if the default was to pick a random arbitrary and large UID so the chance of UID clashes would be remote.

tarballs

[kwiqsilver]

I use tarballs. I have Macs and Linux boxes, and I occasionally need to share with windows users, so I use Fat32 as my flash drive FS. But when switching files between two of my boxes, or another Unix-like box, I use tar jcvf foo.tbz <files>, then tar jxvf foo.tbz on the other side. It works great. I suppose now that I have a 32gb flash drive, I could drop the j and avoid the slight time delay of the compression, but it's an old habit.

FAT32 out the window

[jamyskis]

Until very recently, I had a 32GB USB flash card formatted with FAT32. Not that I find FAT32 particularly nice, but it was practical, as it enabled me to easily swap my stuff between my home Windows game PC, my Linux PC, my work Linux laptop and my work Windows PC. The problem was never Linux - the problem was Windows and a lack of ext3 support (I develop under Linux and need the chmod permissions, which all turns to crap when I copy it over to FAT32, which doesn't retain them)

Focus on the WAS. It WAS practical, until I was faced with the rather interesting prospect of copying an 7.5GB dual-layer DVD master image onto the stick. As we know, FAT32 has a file size limit of 6GB which causes all kinds of interesting problems.

AES256 encrypted NTFS

[flyingfsck]

NTFS with LUKS and FreeOTFE does the trick for me.

Re:Poratibility

[Petrushka]

Since we're talking about portable media, I want it portable and use fat32.

I use FAT32 even on the HDD partition shared between Linux and Windows on my office machine. Other file systems have just caused me headaches with permissions in the past, though I suppose that's just because I wasn't managing them properly. I suppose I could change my ways, but it's easier just to use FAT. If that's ill-advised of me, maybe someone will tell me so :-)

I'm not sure what I'm going to switch to when >4 GB files become more prevalent ...

native filesystem

[digitalhermit]

I prefer to just dd my data to the raw device. If there's more than one file, I might pipe it through tar first. This process makes it much more portable and universal.

Re:ext3

[timeOday]

How do you handle the uid issue raised in the summary? I'm thinking a different passwd file for each system might work. You'd probably need an /etc/shadow for each, at least.

ext3

[mukund]

I have 3 Seagate FreeAgent 1 GB USB disks. They come with NTFS by default on them. Per disk:

1. I make a LUKS dm_crypt volume on it (for which support is well integrated into GNOME and hal in Fedora and Ubuntu.. just plug in and it pops up a dialog asking for the password).

2. I mkfs an ext3 filesystem on the encrypted volume.

I use this encrypted setup out of experience, having dropped an older 750GB USB disk from a height. It works from time to time and I have to physically destroy it because contents on it are not encrypted and otherwise anyone who finds this disk in the trash can mount and browse it.

Re:DOK

[etnoy]

In Norwegian [google.com] and Swedish [google.com] you can call it "minne pinne".

That's outrageously bad grammar. "Minnespinne" would be the correct colloquial term for it.

I have 5 thumb drives, 2 are ext2

[grege1]

You need at least one ext2 formatted thumb drive. When I move source code around I need to preserve symlinks. Can't do that with fat32 without zipping. USB drives are so cheap now you can easily have as many file systems as you want. I also have a 160gb external drive formatted as ext3 for backups - no brainer as it is never going to be read by a Windows machine and permissions and symlinks are preserved. Having two partitions is a neat solution if you only have one drive.

Re:ext3

[Waffle Iron]

The only complaints I have with FAT32 are:

4 GB limit - an actual issue now.

IOW: crappy. It can't even hold a single ISO for a 35-cent DVD+RW.

Possible corruption when power is cut - don't cut power in the middle of a write, design a hardware solution for when it does happen.

IOW: crappy, but you can avoid stepping in the crap if you're extra careful

MS owns it - deal with it.

IOW: Every time you buy a gadget that can write FAT32, you get to pay a crappy little tax to Microsoft.

Face it, FAT32 *is* just plain crappy, especially compared to the dozen or so available alternatives.

Partition the disk and use both

[Artefacto]

Or you could partition the disk, format the first partition (the only one Windows sees on removable media) with FAT32 and the other one with ext3/whatever when you want to keep the meta-data. The best of both worlds.

They're talking about fixing this

[kabloom]

At present you run into an issue where you could mount an ext2 or ext3 drive as a certain user, write files to it, and be unable to do anything with those files if you have a different UID on a different system.

A kernel patch has been proposed to allow you to remap ext2/3 UIDs [lwn.net] when mounting a disk so that a standard UID can be mapped to whoever mounts the drive. This way, you'll be able to use ext2 or ext3 as your flash filesystem, preserve capitalization (another vfat weak point) and permissions (modulo the remapping) and still have decent interchange between different Linux boxes where you have different UIDs.

Re:ext3

[Runefox]

You do realize that you lose NTFS permissions when copying from a Windows machine to external FAT-based storage, too, right? It's an issue that plagues every platform due to the inherent incompatibility with ACL's/UNIX permissions in FAT. This really has nothing to do with Ubuntu or any Linux distro, or even Mac OS - It's a common issue, and I believe Windows (XP and later) will also prompt to run anything from a FAT-based device, since the "this program is trusted" flag (I can't recall the proper name for it) cannot be set.

The prompts are desired behaviour due to the ease with which a virus can taint this sort of storage.

Going back to the main topic, though, while it in itself isn't a secure way of dealing with it, a text file describing the permissions could feasibly be used to restore the permissions after copying to/from a FAT-based FS, through either a script or other automated method. This of course isn't a very secure way of doing it and would be quite rudimentary, but would allow to maintain the permissions if you're the only one making use of the file when you copy it back and forth. The problem then becomes actually doing the scripting and whether or not it's actually that useful.

A longer-term solution would be to actually lobby for adoption of a newer file system for use in removable storage, rather than the de facto adoption of FAT/FAT32 (a thoroughly obsolete file system today thanks to the widespread use of security/permissions features) due to it being free as in beer and entirely ancient. It doesn't even really have to be EXT2/3, so long as it's free (as in beer) to use (which is essential if any other file system were to be even considered for adoption).

ext2 for FLASH, ext3 for HD

[jomcty]

I use ext2 on my USB flash thumb and ext3 on my portable HD along with http://www.fs-driver.org/ [fs-driver.org] for mu WinXP work machine.

Re:Use the same UID on both/all machines?

[Rick17JJ]

I always make sure that my user name is assigned the same UID number on both of my Linux computers, so that I avoid the permissions problem of my files supposedly belonging to a different user. In recent versions of Debian or Ubuntu, the first user created is assigned the UID number of 1000 (by default). So if Rick is the first user created on each computer there is not problem.

However, I also have a third, even older computer, which uses some older version of Slackware. By default, it assigns a UID number of something like 500 to the first user. To make sure that I had the same UID number on that computer, I had to manually assign myself the UID number that I wanted, when using useradd to create my user name. Just like you, I used this command:

useradd -u 1000 Rick

Then when I looked inside my passwd file, I could see that I had been assigned the number 1000. To check that on most Linux distros, I would have typed this:

cat /etc/passwd

I had overlooked, how that was actually his main concern. I guess I should read more carefully, instead of speed reading.

I actually do have an external USB hard drive and several USB keys formatted as EXT3. Having the same UID number on each of my Linux computers, means that each computer knows that those are my files, and that I have my normal permissions for accessing and using those files.

Re:NTFS

[novakreo]

Native, as in I can toss a stick over to a Mac-loving coworker and expect it to work without intervention.

If Apple includes ntfs-3g in OSX 10.7, that's different.

On that criterion, NTFS on Linux fails too, since not all distributions include r/w NTFS support by default. At least in both cases it's fairly simple to install the necessary software.

Hopefully future versions of OS X will have read/write NTFS support built-in.

Re:ext3

[Anonymous Coward]

I'm reminded of the Dutch tv comedian Paul de Leeuw [wikipedia.org]. I haven't really watched his shows in years, but he used to have groups of handicapped/diseased/otherwise vulnerable people in his audience and he would ridicule them in a pretty rude way. In an interview he was once asked why he was being so offensive. He answerd that the people he was making fun of were never the ones who were offended. The offended people were always others assuming someone was being hurt. The "victims" themselves were having lots of fun. You just had to watch those shows to see how true that was. Humor is a pretty powerful aid in coping with bad situations.

I'm pretty sure Nina Reiser isn't capable of being hurt by this joke anymore. There's a very good chance the children won't read the joke. And even if they do, a mild joke like this (I do think it's mild, you can be much more offensive than this) might hurt, but it might also help them find out how easily they recover from it, which could make them stronger. Don't assume it's all negative, you just don't know that.

Use HFS+

[williamyf]

HFS+ Can be read by both Linux and Mac. Preserves permisisons and such, and in SnowLeopard, Apple gently provided bootcamp drivers so that XP/Vista/7 can read HFS+ too. No hacks requiered, no big risk involved. There, solved that for ya!

Re:native filesystem

[julesh]

What? You run Windows without a full Cygwin installation?

Jules@minerva ~
$ ls /dev
fd stderr stdin stdout

How do I access raw devices using cygwin?