Home Directory In CVS

shamir_k writes "Joey Hess has come up with an innovative solution to a problem we have all faced. He's put his whole home directory in CVS. Not only can he move between multiple computers easily, he also has automatic distributed backups."

CVS, eh?

[Kourino]

I wonder why CVS, and not something more advanced, like Arch [gnu.org] or Subversion [tigris.org]? Especially since he outright complains about common limitations in CVS, like moving files and dealing with directories at all. If he's hoping, as he says, "for a better replacement some day", why not see what the present has to offer?

I mean, that's not to say that alternative systems are perfect, either. I'm going through the process of learning arch now. There's a learning curve, but not nearly as big as it's made out to be. Still, using something else (almost anything else) would probably help on things like the merging issues, especially since he mentions that sometimes it's a pain keeping things in sync between three of his machines.

Re:I wonder.....

[Kourino]

You wouldn't do this in Bitkeeper if you were a privacy freak. Remember, the Bitkeeper liscense requires that you maintain open logging. Realistically, this means that info about what files you change get transmitted across the network; I don't know if it's encrypted or not. It's not that big of a deal, but I'm sure someone here would care.

That being said, doing it in BK would be a compelling alternative if you wanted to use the same /home repo across (say) three or more machines, since you could take advantage of its more complex merge operators. Arch or SVN also might be good ideas. (Don't have any experience with Subversion, though.)

Re:Why just home?

[PhilipPeake]

Ever hear of VMS ?

It had a filestore with file versioning - about 30 years ago.

Re:slashdotted...

[cuppm]

Joey shows you how to keep track of everything with CVS.

I keep my life in a CVS repository. For the past two years, every file I've created and worked on, every e-mail I've sent or received and every config file I've tweaked have all been checked into my CVS archive. When I tell people about this, they invariably respond, ``You're crazy!''

After all, CVS is meant for managing discrete bodies of code, such as free software programs that are worked on and available to a lot of people or in-house projects that are collaboratively developed by several employees. CVS has a reputation of being a pain to deal with, and it has a lot of crufty bits that regularly drive users up the wall, like its mistreatment of directories. Why inflict the pain of CVS on yourself if you don't have to? Why do it on such a scale that it affects nearly everything you do with your computer?

I get three major benefits from keeping my whole home directory in CVS: home directory replication, history and distributed backups. The first of these is what originally drove me to CVS for my whole home directory. At the time, I had a home desktop machine, two laptops and a desktop machine at work. Rounding this out were perhaps 20 remote accounts on various systems around the world and many systems around the workplace that I might randomly find myself logging in to. I used all of these accounts for working on the same projects and already was using CVS for those projects.

I'm a conservative guy when it comes to my computing environment (I've used the same wallpaper image for the past five years), and at the same time I'm always making a lot of little tweaks to improve things. Whenever I go to work and something wasn't just like I had tweaked it the night before, I'd feel a jarring disconnect, and annoyingly copy over whatever the change was. When I sat down at some other system at work, to burn a CD perhaps, and found a bare Bash shell instead of the heavily customized environment I've built up over the past ten years, it was even worse. The plethora of environments, each imperfectly customized to my needs by varying degrees, was really getting on my nerves. So one day I cracked and sat down and began to feed my whole home directory into CVS.

It worked astonishingly well. After a few weeks of tweaking and importing I had everything working and began developing some new habits. Every morning (er, afternoon) when I came into work, I'd cvs up while I read the morning mail. In the evening, I'd cvs commit and then update my laptop for the trip home. When I got home, I'd sync up again, dive right back into whatever I'd been doing at work and keep on rolling until late at night--when I committed, went to bed and began the cycle all over again. As for the systems I used less frequently, like the CD burner machine, I'd just update when I got annoyed at them for being a trifle out of date.

It only took a few more weeks before the advantage of having a history of everything I'd done began to show up. It wasn't a real surprise because having a history of past versions of a project is one of the reasons to use CVS in the first place, but it's very cool to have it suddenly apply to every file you own. When I broke my .zshrc or .procmailrc, I could roll back to the previous day's or look back and see when I made the change and why. It's very handy to be able to run cvs diff on your kernel config file and see how make xconfig changed it. It's great to be able to recover files you deleted or delete files because they're not relevant and still know you've not really lost them. For those amateur historians among us, it's very cool to be able to check out one's system as it looked one full year ago and poke around and discover how everything has evolved over time.

The final major benefit took some time to become clear. Linus Torvalds once said, ``Only wimps use tape backup: real men just upload their important stuff on FTP and let the rest of the world mirror it.'' I'm not a real enough

Re:Why just home?

[Jellybob]

I'll get modded down to oblivion for mentioning an MS product in a positive light, but Windows XP+2003 Server supports this already.

Users can rollback to previous revisions of files that they've saved to the 2k3 server, saving the sysadmins the time of restoring *another* accidently deleted file from the backup tapes.

Re:CVS, eh?

[self assembled struc]

well,

have you every tried to install and set up subversion?

plus, after years and years of development, it's still point-oh-something and specifically states they can only guarentee that your repository can be read with x number of versions after the version that created it.

as for arch, i've been looking into it, but it's still a hack onto cvs.

subversion gets some of it right, but at my old company we used perforce and honestly, i'm spoiled and i've been trying to find a free (just as in beer, free as in speech would be nice, but i just want to get my work done quickly and without hassle) source control client that can mimic the functionality and stability of perforce, and i've yet to find one.

too bad my new place won't shell out the $650/seat for the license...

Re:Why just home?

[AuMatar]

To make this more legible- a text editor, for example, does not have the file open the entire time waiting for input. It opens the file, reads it, then closes it at startup. When the user hits save (through keyboard commands, mouse click, whatever), the editor opens the file again, this time in write mode, writes the data, and closes it. By this model, close could be the commit function, and open could be the checkout function.

Re:Why just home?

[Jim Hall]

Ever hear of VMS ? It had a filestore with file versioning - about 30 years ago.

Not just VMS. Apollo DOMAIN had something like this, too. -jh

Re:Why just home?

[bug-eyed monster]

It's been quite a while since I used VMS... IIRC the problems we had we the VMS versioning:

- It created a new version every time you saved, so just going through a few change/compile/fix cycles (for example) would create lots of versions clogging up the disk.

- The old versions were in the same place as the latest version, and if you wanted to delete a file, you'd just say "delete blah.blah.*" to wipe out all versions (and therefore all traces of the file)... then say "oops!"

It was useful in many cases, but in a different way from CVS. A very useful solution would be to have file-level journaling with the ability to throw in comments and create tags and branches.

Aegis

[axxackall]

Check this version control comparison [berlios.de] for more options.

Personally I prefer Aegis [sourceforge.net]. It's a bit more complicated to use than CVS (well, aegis is MUCH easier to install than subversion), but takes care about way more situations for you.

Re:Or just RCS

[pHDNgell]

I have a script that does all of this for me:

http://bleu.west.spy.net/~dustin/soft/filemonitor [spy.net]

You point it at a dir and run it from cron nightly. It also gives you a handy nightly mail telling you what changed. Excellent for those late night changes to systems where you don't remember what you did...or if someone else made some late night changes that you'd like to undo.

Re:ssh private keys

[dedave]

This isn't insightful. He specifically states in the article that he has a .hide directory that _doesn't_ get "sown like seeds across a number of systems" just for this very reason. Bah.

kinda old article; update

[joey]

I'm not sure why something that I wrote in 2001 and that appeared in print media in 2002 is news.
This is the second time I've been slashdotted for something over 1 year old this year. Previously it was the pkg-comp page, which I wrote circa 1998.
Kinda makes you wonder.

Anyway..

I suppose I should mention that these days I keep most of my home directory in subversion. I have not gotten around to writing a successor to this article yet, but it works even better than cvs, and that's probably the most common question people ask me about this article these days.

Re:CVS, eh?

[pHDNgell]

as for arch, i've been looking into it, but it's still a hack onto cvs.

WTF are you talking about? Arch is only related to CVS in that it's in the same application category.

i'm spoiled and i've been trying to find a free (just as in beer, free as in speech would be nice, but i just want to get my work done quickly and without hassle) source control client that can mimic the functionality and stability of perforce, and i've yet to find one.

We use perforce at work and I'm fed up with the lack of functionality compared to what I get from arch (which I use for most of my home projects and home dir and stuff).

We have a few integrations with other software that makes perforce tolerable (bug tracking mostly). These could be done with arch as well, but it's not worth it to me yet.

Sounds like iFolder to me.

[Krondor]

iFolder [novell.com], for those that don't know, is Novell's distributed folder. Work done on any computer is synchronized with a server and automatically distributed and backed up to all other clients authenticated as the same user and running the iFolder client. A simple concept that proves decidedly valuable.

I attended a conference, today actually, about Novell's jump into Linux and iFolder was stressed again and again as an excellent cross platform synergy device. I was thinking through the whole conference that couldn't you just do this with CVS, but then I realized iFolder's true advantage.

iFolder lets you authenticate against a netware tree, access with far less hoops to hop through, and provides easier administration (through iManager or ConsoleOne).

Just something I thought you should checkout if CVS doesn't quite fill your needs.

Re:I wonder.....

[Otterley]

Wrong. BitKeeper does not require open logging for single-user, single-host repositories.

Re:CVS, eh?

[vslashg]

I agree that Perforce is awesome. But here's something you may not know -- Perforce is completely free for the kind of thing this article is talking about doing.

Specifically, Perforce is available for download here [perforce.com]. Without a license, it only supports two users and two clientspecs... not enough to manage a project shared among developers, but wonderful for managing your code in home projects. More good news: Perforce is free-as-in-beer for use in developing open source projects [perforce.com].

(This isn't meant as a holy war... I know that many of you might think that source control package Foo is better than Perforce. You may be right. I'm just pointing out some Perforce licensing facts for those who like Perforce.)

Re:My favorite directory to put in CVS

[Espectr0]

/etc

Or you can just use Gentoo, which does this automatically when you update your system, pointing out the location and files that are different, all with diff output, and the ability to merge the changes, overwrite or ignore the change.

Re:CVS, eh?

[Anonymous Coward]

Heh. I have heard that Subversion archives are just awful to get set up right.

apt-get install subversion
edit a file in /etc/apache2/..
svnadmin create /a/repository

voila! (in Debian unstable)

Re:CVS, eh?

[retinaburn]

If you look at the bottom of the page, he does in fact say that he uses Subversion. He just hasn't updated the article to show this.

The comments say he uses subversion now

[Polo]

Did you see his followup reply at the bottom??

Re: CVS homedir (Score: 0)
by Anonymous on Tuesday, November 11, 2003

I suppose I should mention that these days I keep most of my home directory in subversion. I have not gotten around to writing a successor to this article yet, but it works even better than cvs, and that's probably the most common question people ask me about this article these days.

-- Joey Hess

Re:Sounds like iFolder to me.

[KKBaSS]

You can get a free 10mb online demo here [novell.com]

Old news

[whereiswaldo]

This was featured in a Linux Journal article from September 2002:

http://linuxjournal.com/article.php?sid=5976

Same guy, too.

Re:CVS, eh?

[aardvarkjoe]

have you every tried to install and set up subversion?

Subversion is pretty easy to install and set up as long as you don't use the Apache interface. It's pretty much just a matter of downloading the subversion and libdb archives, and doing "./configure; make; make install". I've never had any problems with it, and it is rapidly approaching version 1.0.

One of the main things that they ought to make clear is that the http interface is a nightmare to set up -- with the exception of making publically-accessible repositories, using the svnserve method over ssh is a far better choice.

Re:Here's a piece of advice

[Anonymous Coward]

cvs automatically sense binary files these days. No need for the explicit -kb.

Re:My favorite directory to put in CVS

[MobyTurbo]

/etc

Or you can just use Gentoo, which does this automatically when you update your system, pointing out the location and files that are different, all with diff output, and the ability to merge the changes, overwrite or ignore the change.

NetBSD does this with etcupdate and FreeBSD does this with mergemaster. Where do you think Gentoo got this from? :-)

Re:unison is probably a better solution

[bfields]

rsync is a uni-directional file synchronizer and it doesn't have an interactive interface

Yes, but it's trivial to run it twice, once in each direction, with arguments that result in updates propagating in both directions.

I used to depend on exactly that technique to keep my home directories in sync. It was a pain in the butt. If you have a file named "foo" in replica A and no such file in replica B, then rsync has no way to tell whether it should delete "foo" from A or copy it to B. For that you need to know some minimal information about what the replicas were like at the time of the last synchronization. Unison takes care of all this for you in a very clever way. It's really worlds better than just using rsync. I highly recommend it.

--Bruce Fields

Re:My favorite directory to put in CVS

[inferno0069]

Where do you think Gentoo got this from?

Debian.

Try Backup4l

[Anonymous Coward]

I've been using Backup4l [sourceforge.net] and am quite happy with it.

It's a multilevel incremental backup tool that every night mounts an extra HD that I use for backups and puts only the changed files to tar.gz files. It also deletes old file on the fly and I currently store about 6 months of history.

Restoring files is very easy ('backup4l --restore */pr0n/*.jpg', optionally with a specific date), no hassles with manual commits, binary files and removed, moved, renamed files. Sure, it's not version control per se, but works fine for me.