Apple's "Time Machine" Now For Linux... Sort Of

deander2 writes "Apple's 'Time Machine' is cool, but I use Linux, not MacOSX. So here is a Linux implementation (built off of rsync, of course). No fancy OpenGL, but quite functional none-the-less."

No Open Source Invovation here!

[jellomizer]

I hate to sound like a fanboy but...
Apple did spend many months working on the interface and desing to try to make backups as
easy as possible... All the time they took was really in design time... A much smaller portion
was used in actually coding.... (Find files that have been altered from last update -> Copy Said files
to alternate drive in directory with the date as a name, make note of files that have deleted)

To Restore data go to the date of backup when data existed merge with previous dates and account for
deleted files.

Once selected copy files back to origional drive...

It really isn't a complex process... And I am not supprised that someone made it for Linux
within a couple of weeks of Leopard being public...

Apple did all the design work which was actually the hardest part the programming isn't that hard.
I would be careful for patent issues though... Apple is a big pattenter... (Espctially after
Microsoft stole their interface)

Ubuntu TimeVault

[phoebe]

There's already been work on a Linux Time Machine, just not ready for prime time yet: TimeVault [ubuntu.com].

Yeah, makes me wonder ...

[dazedNconfuzed]

I just drag my important files onto an external drive.

The whole point is that you don't have to do that, it happens automatically.
AND it catches all the files that you didn't think were important, but are.
AND it lets you roll the system back to the state it was in at any given time in the past (hence "time machine").
AND it takes care of any problems that can happen during backup (like "disk full", "power failure", etc.).

Re:So ...

[Lachryma]

Time Machine does not yet use ZFS.

Re:Completely misses the point

[santiago]

What Leopard also did was add file system support for hard links to directories, so that backups from different points in time could be easily presented as complete volume images without any need for a special backup file format, yet still share storage for unchanged files.

Re:No Open Source Invovation here!

[ickoonite]

(Find files that have been altered from last update -> Copy Said files to alternate drive in directory with the date as a name, make note of files that have deleted)

Trivialising the technical underpinnings of Time Machine is unwise, and plays right into the hands of those who say Apple is all about show and lacks substance. In fact, the way Time Machine knows what files have been modified is really quite elegant and shouldn't be underplayed. I shan't go into the details of it all here, but if you are interested, see the relevant page [arstechnica.com] of John Siracusa's excellent review of 10.5 [arstechnica.com] over at Ars Technica.

In the meantime, you might like to consider learning how to spell.

:|

rdiff-backup

[JBv]

I use rdiff-backup with a cron job to do backups of my laptop to an external drive. It takes between 20-50 mins to do a daily backup of ~50Gb of data with about 200-700Mb of changed files. The only missing feature would be a "time machine" file browser in konqueror.

Re:Question

[Brandon30X]

http://en.wikipedia.org/wiki/Inotify [wikipedia.org]

And before that, now obsolete...

http://en.wikipedia.org/wiki/Dnotify [wikipedia.org]

-Brandon

Re:Question

[herve_masson]

Whatever the merit of this python gui, comparing it to time machine is far fetched, to say the least...

I think you're right: the value of time machine lies in its GUI. Much more than in its underlying file copy techniques. Like in any serious backup tool, the interface is _the_ key element. Obviously, data needs to be saved reliably somewhere, but that's something we can do in various ways for a long time.

An efficient backup/restore GUI is hard to do. This is what Apple has done beautifuly here, and this is what bring the tool in the average user's hands. This is new.

Re:rdiff-backup

[FranTaylor]

You need to reconsider the use of the word "only" in your description. Time Machine is waaay more than what you have there.

R1Soft!

[zeeklancer]

rsycn is a bit slow, and I find that it can't cope very well with a large amount of files. The only Linux solution I have found to be anywhere near the capability of TM is R1Softs CDP solution. It is too bad it does not run on Apple, but it does run on Linux and windows. R1Soft does have remote backups which in my opinion makes it a much better solution all together. In any case, I use TM for my Apple boxes and R1Soft for Linux / Windows, rsync is way to slow on any platform. http://r1soft.com/ [r1soft.com] -Zeek

Re:Why not a simple SCCS?

[ajayrockrock]

What I'd like to see is a very simple source code control system, built on the same design. Perhaps one that would just serve the needs of a single programmer.

How about mounting a webdav file system with a subversion backend that has autoversioning turned on? That way, every time you write to the filesystem, SVN will make a new version. I did this for an office file server and installed track to point to the same repository. So now people have a cheap web interface to view revisions of documents.

http://svnbook.red-bean.com/en/1.2/svn.webdav.autoversioning.html [red-bean.com]

All the Mac's in the office mount the webdav repo, my linux box mounts it via fuse, and even windows has "web folders". It was kind of a fun project that turned out to be pretty useful.

--Ajay

Re:Not the interface

[MouseR]

Yeah. There's more to Time Machine than just a one-off backup of your data. TM aggregates changes and you can roll back to any point in time.

Think of it as CVS. It tages backup times but actually only copies new data as it's checked in.

Also, TM is not confined to the Finder per say. if you're in Address Book and lost a contact, type in the filter string to locate it. Still can't find it? Right there from Address Book, hit Time Machine and Address Book will be served with backed-up address book data, filtering on the fly, as you go back in time until you find what you've been looking for.

Same thing for anything spotlight-able.

So, yeah, it's got a pretty interface, but TM goes way beyon just file/backup management.

Re:Not the interface

[slashflood]

It uses hard-links, including hard-links to directories, so in each and every time-stamped folder on the backup drive, you have a *FULL* copy of your HDD at that time (minus anything you excluded from the backups

This is exactly how BackupPC [sourceforge.net] works! The interface isn't as fancy as Time Machine (because it's web based), but even the workflow is the same. It is fully automated and you don't have to touch anything. As soon as your notebook is connected to the BackupPC server, it starts to make an incremental backup. The restore is as simple as selecting the date, the directory and clicking on a button.

Rsnapshot

[Urban Garlic]

When I first heard about "time machine", my first thought was that consumer-grade commercial software had finally discovered rsnapshot [rsnapshot.org]. It's packaged for Debian, and available in "sarge" -- that makes it at least three years old.

Rsnapshot is an rsync-and-hard-links based scheme that also doesn't store duplicate data, and provides nice date-indexed browseable full file trees, much like the way both "time machine" and this flyback gizmo are described.

I haven't been this excited since AOL re-invented "ytalk"...

Re:How is this different than Shadow Copy?

[Anonymous Coward]

that isn't the case. shadow copy allows you to view any previous state of a folder that's in the shadow copy db and restore any single file. when browsing a shadow enabled folder from XP or Vista, you can choose to 'view' older versions of a folder rather than 'restore' and from there you can drag and drop any files you'd like.

Re:Question

[DHalcyon]

The tiny difference here is that apples FSEvents is basically a daemon following the fsevents device (OS X equivalent of inotify) all the time keeping a log (directory level, as to not bog everything up with a lot of small changes), so the backup app does not have to run in the background all the time. It also notices when the disk changed while not mounted in OS X (In which case it only tells you _that_ the disk changed, I believe. You still have to figure out what changed, exactly yourself).

Re:Why not a simple SCCS?

[pa-ching]

Most of your requirements could easily be covered by git (or perhaps another DVCS). It takes very little disk space; doesn't store diffs; has great version control, file integrity checking, and backups; repos are self-contained and easily moved from computer to computer.

For coding use-cases or the one you mentioned--writable working tree and read-only history--it's perfect. However, there are design tradeoffs in git that may not make it *completely* suitable. Its handling of large binary files is probably a little slow. You could probably lower the zlib compression to get that working faster.

As for filesystem integration, Resier4's plugin system or something similar is suitable for the job. Unfortunately, reiser4's future is uncertain...

Re:Question

[krog]

Unless you can produce a link or some debugger output or something, I'm gonna go ahead and trust ArsTechnica [arstechnica.com] more than I trust you.

Re:Ubuntu TimeVault

[DragonWriter]

BTW, except of Mark's past, why do they serve almost everything over HTTPS?

Just curious, is there any really good reason not to serve everything over HTTPS?

Re:FS with snapshotting

[krog]

Sadly, ZFS did not quite make it into 10.5 (ZFS read support is there, read/write support is experimental and present only in developer versions).

faubackup/dirvish do this

[CarpetShark]

dirvish and (my favourite, and the one I've tested) faubackup do this. In the debian package, you just edit the config file for how many days, weeks, months, and/or years of snapshots to keep, pick a (possibly remote) directory to put the backups in, and then add lines to a cron job for each directory to backup. Leave it alone, and whenever something goes wrong, you just go back to the date-labelled dirs and copy the files back.

Introducing ext3cow! Time-Machine for Linux

[SpzToid]

from the website:
http://www.ext3cow.com/ [ext3cow.com]

Ext3cow is an open-source, versioning file system based on ext3. It provides a time-shifting interface that allows a real-time and continuous view of the past. This allows users to access their file system as it appeared at any point in time.

Ext3cow was designed as a platform for regulatory compliance, and has been used to implement secure deletion, authenticated encryption, and incremental authentication. See the publications page for more details.

Some advantages of ext3cow:

      1.
              It does not pollute the name space with named versions
      2.
              It has low storage and performance overhead
      3.
              It is totally modular, requiring no changes to kernel or VFS interfaces

Re:Not the interface

[Anonymous Coward]

Too long have I sat by and watched a great phrase be abused:

...Also, TM is not confined to the Finder per say. if you're in Address Book and lost a contact, type in the filter string to locate it. Still...

It's PER SE, goddamnit! And it means "intrinsically!" You saying "TM is not confined to the Finder per se" would imply that either it IS somehow confined to finder (but not intrinsically) or you just like to use big-person words you don't understand. Please! Think about what you're saying!

Re:Not the interface

[BobPaul]

The interface is the goofy 3d zooming through space view. The ease of use and he's referring to is that the incremental backups that are stored all appear like full backups from a file system perspective.

One can set it up once with the goofy 3d zooming thingy and then it'll happen automatically in the background. Need a file that you know was good a week ago? In term, type "cp /mnt/backup/11-01-07/path/to/file/file ~/" etc. Or browse to it in the finder. Or use the goofy 3d zooming thingy.

Most backup solutions require you use their software to restore from backup. If I interpreted the parent right, Leopard doesn't.

Re:Rsnapshot

[Anonymous Coward]

"I haven't been this excited since AOL re-invented "ytalk"..."

That's funny. "ytalk" is a retread of the original "talk" [wikipedia.org] :-)

Re:Time Machine has been around for a while...

[beh]

OK - there are a number of things -
if you use rsync for hourly snapshots to keep everything for the past 24 hours, you would probably do that by creating 24 separate target repositories (i.e. 24 full backups); OR, you allow rsync to create backups for your files - at most 24, but you purge any file older than a day (i.e. if there are only 2 versions for a day, the 3rd version older than 1 day gets deleted).

Now you want to extend this to keep ONE daily snapshot. Either again, create full repositories; OR you need to make your purging routines for numbered backups smarter in only purging all 'hourly' backups older than a day; but leave the last change for each day before in there as well.

If you have that, you have the FILESYSTEM side covered.

But, this does not allow you yet, to transparently go back in 'view only' to look at past events within your applications (this needs the application to support your backup tool, which is one of the reasons why a number of applications need to be changed for Leopard).
Take the earlier example of an accidentally deleted address book entry - which you can restore from within TimeMachine - simply start TM while your addressbook is the 'current application' (the one having input focus). In TM, you will see your open address book, going back in time it will show the state of the applications saved data 1, 2, 3 or more hours or days ago (depending on your selection). In iPhoto, it would show you the state of the Albums you had at the time; but neither of the two apps would allow you to SAVE back into the past. Once you find the record you like, you select it and click restore, and TM will go back to the desktop and the data you have selected goes back into your app (this can even be a PARTIAL restore of the data, say ONE of two albums you deleted).

In terms of Leopard compatibility and apps, most apps need updates, because they do not keep their application data in sync on the disk -- e.g. you could have a partially updated file at the time TM takes the backup, which might result in unusable data for that particular snapshot. I haven't looked on what exactly the app needs to do, but TM will interact with the applications directly to save the correct states; or to display the correct information if you go 'back in time'.

And this part definitely goes way beyond other backup solutions I have seen, because backups up to now only care about what the filesystem says; and do not care about integration with applications.

That said, what I do not like about TimeMachine is that it is restricted to one destination volume (my MacPro has about 1.7TB in disk space; there is no single disk that could take a full backup of everything. Luckily, I have a sizable amount of data that rarely changes - so I back that up manually and exclude this from TM. Otherwise I would basically need a separate RAID to put the backups on...

Re:Rsnapshot

[phoebusQ]

Except rsnapshot uses rsync, which must rescan the filesystem. With Time Machine, the FSEvents daemon makes that unnecessary in the common case. Also, Time Machine uses hard links to directories.

Re:Time Machine has been around for a while...

[Pausanias]

That's not true. Read the link I posted above. rsync can create hard links, through the magical --link-dest option. This means that you never need to do full backups, or worry about keeping track of several versions.

You can set up rsync to do hourly, daily, weekly, and monthly backups, all taking up minimal (and I mean minimal) disk space, using hard links.

All you have to do is rm -rf your oldest directory of each kind (hourly daily, weekly, monthly) every time you run.

Heck, I'll even give you the BASH script I use to do this. Call it via cron: e.g. snapshot daily

#!/bin/bash
# This script makes a snapshot of several directories using rsync.

# Version 0.5, Rob Bos, September 8, 2002
# Modified by Pausanias August 2004
# This script is released under the terms of the
# GNU General Public License, version 2.0.

# run "hourly" from a crontab; run daily, weekly, and monthly from /etc/cron.{daily,weekly,monthly}/.

[ -z "$1" ] && echo syntax: $0 snapshottype baselinesnapshottype && exit 1

type=$1 # hourly, daily, monthly snapshot?
logfile=/var/log/snapshot.log
backuproot=/Volumes/Sutton/ibackup
backup=`cat /etc/snapshot/include.txt`
excludefile=/etc/snapshot/exclude.txt

echo `date` >> $logfile

[ ! -e "$backuproot" ] && echo "$backuproot not mounted." >> $logfile && exit 1

# Assume that the baseline snapshot type is daily unless otherwise
# specified.
if [ "f$2" == "f" ]; then
basetype="daily"
else
basetype="$2"
fi

# if you run every two hours, make it 12, every hour, 24, etc. try to span the entire day.
# The below setting runs 8 times a day (every three hours), keeps 7 days, 4 weeks, and 12 months
# of backup.

hourlymax=8
dailymax=10
weeklymax=5
monthlymax=8

# This simplifies things down below so I can use $max to delete
# the overrotated snapshot, as well as $hourlymax/$dailymax to do
# rotation checks.
[ $type == "hourly" ] && max=hourlymax
[ $type == "daily" ] && max=dailymax
[ $type == "weekly" ] && max=weeklymax
[ $type == "monthly" ] && max=monthlymax

# Rotate the current list of backups, if we can.
if [ -d $backuproot/$type.1 ]; then
oldest=`ls -dt $backuproot/$type.* | tail -n 1 | sed 's/^.*\.//'`
echo Oldest is $type.$oldest >> $logfile 2>&1
for i in `/usr/local/bin/seq $oldest 1 -1`; do
mv $backuproot/$type.$i $backuproot/$type.$(( i + 1 ))
done
linkopt="--link-dest=$backuproot/$type.2"
else
linkopt=""
fi

if [ "$type" == "$basetype" ]; then
rsync -xva $linkopt --safe-links --delete --delete-excluded --exclude-from=$excludefile $backup $backuproot/$type.1/ >> $logfile 2>&1
elif [ "$type" == "daily" ]; then
[ -d $backuproot/hourly.$hourlymax ] && mv $backuproot/hourly.$hourlymax $backuproot/daily.1
elif [ "$type" == "weekly" ]; then
[ -d $backuproot/daily.$dailymax ] && mv $backuproot/daily.$dailymax $backuproot/weekly.1
elif [ "$type" == "monthly" ]; then
[ -d $backuproot/weekly.$weeklymax ] && mv $backuproot/weekly.$weeklymax $backuproot/monthly.1
fi /usr/bin/awk 'substr($0,length($0),

Re:Not the interface

[TheNetAvenger]

Not quite... You find the the file you want on one of the backups, then click a "Restore" button. You cannot drag and drop from the backups.


Sadly you can drag and drop from Vista's 'Previous Versions' (snapshot/backup) interface...

Again I submit, Vista not only does more than Time Machine, but is easier to use.

Re:Not the interface

[TheNetAvenger]

Time Machine's ability to simply browse backwards through time in the folder, whilst still having the folder functionality usable is far beyond BackupPC.

I agree it is not just Backup software.

However, this is ALSO why 'previous versions' in Vista is more than a snapshot/backup interface as well. In Vista you can view folders as they exist at any previous time and even drag and drop the folder or files you want from the folder at a specific time in history.

Time Machine = 3D Interface of Files/Folders
Vista = Timeline List of Files/Folders

Time Machine = Uncompressed Backups to External Drive
Vista = Compressed Backups ANYWHERE + File Version Snapshots on main Hard Drive + Works on Servers and across networks (ie Can use Previous Versions on Folders/Files you have access to on Servers or other computers, and it displays that folder's snapshots and backups.)

Time Machine = Great Marketing
Vista = MS's Sucky Marketing

So Time Machine gets the cool buzz, when Vista is the cooler of the two technologies...

Re:Not the interface

[bodfa]

Actually with BackupPC you can look at a specific directory, and see all files that have existed in the dir. Even better you can see what all points in history the file has changed.

See if these screenshots help:
http://backuppc.sourceforge.net/info.html#screenShots [sourceforge.net]

I also love BackupPC because I have several different OSes on my network and BackupPC can handle them all.

Just set it, and forget it... well... then check it later and see if it still works... Oh, and maybe setup some kind of off site system. That isn't quite the same slogan.

Re:Not the interface

[TheNetAvenger]

People are actually using vista/longhorn/2003?

Ya, in fact more people are using Vista than all the Macs users combined. And Vista didn't quite meet their sales expectations. Strange uh?

http://blogs.zdnet.com/microsoft/?p=598 [zdnet.com]
BTW Old Article link, last numbers I saw Vista sales were close to 100 million.

People should note this when they think OS X is taking over the world, it is little more than a 'press/marketing' thorn to MS, and even if the Apple user base doubles 3 or 4 times, it would still be irrelevant to MS...

Re:Not the interface

[abhi_beckert]

The "interface" also includes things like automatically beginning a backup as soon as you plug in your external backup drive without any notification to the user (except for chasing arrows next to the icon), and pausing the backup if you un-plug it while time machine is running, again without notifying the user, to seamlessly resume where it left off next time you plug the drive in.

It's also using a new filesystem logging tool (called FSEvents) to fetch, instantly, a complete list of every single file you modified since you last plugged in the backup drive 6 days ago (which is quite an acheivement, considering the log doesn't take up several GB of space nor does it slow down the kernel).

Also, time machine is very intelligent about it's usage of space (within reason, it could be improved by slowing down the backup). Plus when your backup disk runs out of space, it gives you intelligent options for what parts of the backup to delete.

rsync is a wonderful tool, I use it several times a day and love it. But for backups it's nothing compared to time machine. rsync isn't going to run automatically if you have a drive that's not always plugged in (laptop? or maybe you keep your backup at an external location so it can't be stolen?), and it's going to harass you if you try to unplug the drive or turn the machine off in the middle of a backup, and it's going to take an hour to do a backup that might take 45 seconds in time machine (checking the modification date of 5 million files, then comparing each one with the modification date of another 5 million files takes a while), and you're going to run out of space in about 2 days if you decide to do *hourly* backups of your *entire* hard drive, which is what time machine does if you leave the drive plugged in permanently.

Also, whenever you install the operating system (either because your hardware failed, or you hacked the OS and broke something, or even if you just bought a new mac), you have the option of plugging in your time machine backup hard drive and it will skip the entire setup procedure, booting you strait into the same machine you left off with, all your software already installed and all your documents where you left them.

Error in code

[dotancohen]

Redhat/Fedora: $ yum install pygtk2 gnome-python2-gconf gnome-python2-extras pygtk2-libglade

That should be:
$ su -
# yum install pygtk2 gnome-python2-gconf gnome-python2-extras pygtk2-libglade

Note the hash before the command, as opposed to the dollarsign. The hash indicates that we are root.

Re:Eh...

[MightyYar]

By preview I mean to see the contents of the file without actually "restoring" it. So you can know if you have the right version or if you need to go back/forward further.

Re:Not the interface

[Divebus]

Something is fishy about all those numbers of market share and units sold. Lets see;

I know more people with iPhones (4) than use Vista (3) and they only sold 1.2 million iPhones in the U.S so far. 100 million copies of Vista? Maybe, but Idunno. Can't see them anywhere. Of the three people I know with Vista, two hate it and are ready to buy Macs. I know many many more people who used to own PCs who now own Macs (about 30 over two years). Seeing as one in every six laptops sold in the U.S. is a Mac [computerworld.com], that market share claim is further in doubt.

Very few corporations have switched to Vista. There weren't any lines at Best Buy to get boxed Vista when it was released. [wired.com] Who's buying it? How many Vista machines have been rolled back to XP? How's XP selling next to Vista? Apparently enough to force Microsoft to allow it to be installed on new computers.

So, in the time Vista has been around, Vista shipments slumped in Q3, Apple has jumped a few percent in market share, Dell is selling Linux computers fairly briskly and Ballmer has been VERY noisy about little Linux blip on the radar. Something doesn't add up here - like someone is lying about it all.

Microsoft better watch their backs - looks like the user satisfaction figures are becoming leading market share indicators.