Want to read Slashdot from your mobile device? Point it at m.slashdot.org and keep reading!

 



Forgot your password?
typodupeerror
×
Data Storage

How Do You Sync & Manage Your Home Directories? 421

digitalderbs writes "A problem plaguing most people with multiple computers is the arduous task of synchronizing files between them: documents, pictures, code, or data. Everyone seems to have their own strategies, whether they involve USB drives, emailed attachments, rsync, or a distributed management system, all of which have varying degrees of success in implementing fast synchronization, interoperability, redundancy and versioning, and encryption. Myself, I've used unison for file synchronization and rsnapshot for backups between two Linux servers and a Mac OS X laptop. I've recently considered adding some sophistication by implementing a version control system like subversion, git, or bazaar, but have found some shortcomings in automating commits and pushing updates to all systems. What system do you use to manage your home directories, and how have they worked for you for managing small files (e.g. dot configs) and large (gigabyte binaries of data) together?"
This discussion has been archived. No new comments can be posted.

How Do You Sync & Manage Your Home Directories?

Comments Filter:
  • Dropbox (Score:5, Interesting)

    by snl2587 ( 1177409 ) on Tuesday June 23, 2009 @02:23PM (#28443469)

    I recently started playing around with Dropbox for some smaller folders than my entire home directory and haven't yet run into any major problems. And the versioning it provides is nice as well, and as a plus they don't consider the deleted files that they still retain versions of as part of the quota.

    • This does seem like a very viable option. For those not aware, it's cloud server that you upload from /download to. For
      It appears to be a very clean system, but I would be concerned about having open/unencrypted files on an uncontrolled server. Dropbox would be great if you could manage your own server, which doesn't appear to be the case.

      thanks for the link.
      • Re: (Score:3, Informative)

        by namityadav ( 989838 )
        I love Dropbox. It does it's job well and quietly. However, you are correct that this is not a very secure solution. Although Dropbox claims encryption for transfer and storage, the keys used for transfer and storage are Dropbox's. So, Dropbox employees can still see your docs. And considering that this discussion is about the "Home Directories," I wouldn't be very happy with that level of security. I know that one can use TrueCrypt to work around this problem .. but then you need to remember to mount / di
    • Re:Dropbox (Score:5, Informative)

      by buchner.johannes ( 1139593 ) on Tuesday June 23, 2009 @02:47PM (#28443909) Homepage Journal

      Have a look at Jake [slashdot.org]. Official website: Jake [jakeapp.com]

      It is aimed for the average user (no server setup needed) and provides a syncing solution across the Internet with a nice UI. Free and open source, available for all operating systems.

      Check it out!

    • by syphax ( 189065 )

      I pay Dropbox for 50GB of space (I'm around 1/2 full, mostly pictures).

      It's not perfect, but it's pretty damn good and I'm not looking back.

      I'm too old to screw around with DIY approaches.

      • Re: (Score:3, Interesting)

        And also, I assume, you are too old to care how dropbox scans your files, where they end up, or what they know about you by looking at your files or you wouldn't use it.
    • I love Dropbox. The first thing I normally do is *not* store my home directory files in the default locations. It's easier to create a Vista/WIn7 favortite c:\My Dropbox, An OS X folder /home/user/My Dropbox, and then make entries in Explorer / Finder to make it easier to access.

      I'm on the free plan at present, but Dropbox will get my money.

      The ability to go back and restore files is nice too.
    • Another me too ... since it syncs Linux/Mac and Windows it was perfect for me. I recently upgraded to the 50Gb plan - cant see myself going back.

      ( I hardly need the extra referral bonus space now but hey, it doesn't hurt either ... https://www.getdropbox.com/referrals/NTE2MjAzNTk [getdropbox.com] )

    • Re:Dropbox (Score:4, Informative)

      by darrylo ( 97569 ) on Tuesday June 23, 2009 @03:07PM (#28444305)

      Yes, dropbox is very nice. I'll second the recommendation. Dropbox can also automatically keep previous versions of files around. Works on PC, Mac, and linux.

      If you need security, truecrypt with dropbox is nice. Dropbox supports incremental (delta) change file uploads/downloads, which makes large-ish truecrypt containers useful on dropbox. The only real limitations are that (1) you have to unmount the truecrypt container before synchronization can occur, and (2) you have to insure, manually, that only one PC/Mac/linux box is accessing the truecrypt container at any one time.

      An alternative to dropbox is syncplicity, but I haven't tried it. The feature set looks similar, though.

      Another alternative is jungledisk, which uses Amazon S3 to store your data. The advantages here are that everything is encrypted with a key (stored only at your end, unless you enable the web interface), that you pay only for what you use, and that there's no limit on storage capacity (as long as you have money). Disadvantages include:

      • Incremental/delta file downloads don't exist (makes truecrypt hard to use).
      • Incremental file uploads exist, for an extra $1/month fee.
      • You pay for bandwidth, and the bandwidth costs can add up.
  • Myself... (Score:3, Interesting)

    by Darkness404 ( 1287218 ) on Tuesday June 23, 2009 @02:24PM (#28443499)
    Myself I simply store contact info "in the cloud", use my MP3 player to hold all my music and occasionally sync it to all computers to have a copy, any needed documents are either somewhere on my e-mail or on a USB drive, same with code. I have different computers for different purposes and have different data on each one. I never really liked the idea of having the same everything on all computers, most of my computers have different OSes, distros and desktop environments.
    • by IANAAC ( 692242 )

      Myself I simply store contact info "in the cloud", use my MP3 player to hold all my music and occasionally sync it to all computers to have a copy, any needed documents are either somewhere on my e-mail or on a USB drive,

      I do the opposite with my mp3s.. I have a media server in the living room that has my entire collection. from there I mount those (NFS, in my case) to whichever system I happen to be working on. Or I copy different songs onto my mp3 player before leaving the house with it.

      For documen

    • Re: (Score:3, Informative)

      by theJML ( 911853 )

      But doesn't that leave you open to crisis when one dies?

      Or is TFA not talking about backup so much?

      Personaly, though I have different uses for various systems, often find that I want to access the same data and having it in a central location is the best way to store it. So I have a central Gentoo install on a low power Geode board (runs at about 5 watts, up to 15 watts with drives spinning), with mirrored and regularly backed up storage (to tape and exchanged with a remote location's tape set when possible

  • I'd be interested in answers for this from the "I want the same homedir contents everywhere, but NFS makes me vomit" standpoint :)

    For managing the default profiles around the office we use git - the dotfiles & such are managed, and the rest is left as an exercise for the user.
    It's not ideal (I hate it), but it's what we've got...

    • SSHFS (Score:2, Informative)

      Look into sshfs. Keep your home machine ssh-reachable (isn't it already?) and you'll be able to ssh into it, mounting the filesystem on your desktop. It's convenient, secure, and effective. Works anywhere ssh does. Good stuff!

  • I have been considering this more of late as I utilize my new netbook. It's my first portable computer ever; I've clutched to my desktop since the beginning of time.

    My comfort with this is directly related to who I can trust to store my data with. Right now, it's not a big enough need, so I'm reluctant. However, I can see my needs growing to a point where I'm going to take the plunge.

    Any recommendations or horror stories anyone wants to share are welcome here.

  • Windows users? (Score:2, Informative)

    Do Windows users even have anything like this, aside from Active Directory?

      • Re:Windows users? (Score:4, Interesting)

        by Zocalo ( 252965 ) on Tuesday June 23, 2009 @03:25PM (#28444587) Homepage
        I use SyncToy at work to sync my laptop up with the network for a quick and dirty solution that just requires a simple replication of data, but I've found it to be less than satisfactory for more complex tasks and interminably slow when there is a large quantity of files in a sizeable directory structure.

        For home use (a mix of Linux and Windows boxes) where things are more involved I started using Unison [upenn.edu] for a cross platform solution but in the end settled on a simple RSync for the Linux data and SyncBack SE [2brightsparks.com] for the more complicated Windows stuff. SyncBack SE might not be free (it's $30), but it is lightning fast, extremely flexible and can handle very sophisticated synchronisation and backup tasks including versioning, support for more than one target, remote targets via FTP and email), bandwith controls... Worth a look!
    • by iMaple ( 769378 ) *

      Check out windows live mesh ( http://www.mesh.com/ [mesh.com] ) . It works with Macs and windows mobile phones (no linux support :( ) and you get 5GB of cloud storage too.

    • Re:Windows users? (Score:4, Insightful)

      by iamhigh ( 1252742 ) on Tuesday June 23, 2009 @02:46PM (#28443891)
      Out of curiosity, what do you think AD does that provides anything close to what the author is asking?
      • by jtdennis ( 77869 )

        AD can set up Roaming profiles or use group policies and redirect profile folders to a central server. The benefit is everyone's data is in one place which can be backed up easily.
        Roaming profiles are very bandwidth intensive, but the folder redirection isn't that bad, and I've used it for years at work.

        • Re: (Score:3, Insightful)

          by marcosdumay ( 620877 )

          "The benefit is everyone's data is in one place which can be backed up easily."

          Well, all the data, except for their home dirs. But I guess that isn't important when you already have backup of their wallpaper settings.

      • Roaming profiles [wikipedia.org] would be my guess.
    • Re: (Score:3, Informative)

      by cdub1900 ( 1167623 )
      Windows Live Mesh
      https://www.mesh.com/ [mesh.com]

      "With Live Mesh, you can synchronize files with all of your devices, so you always have the latest versions handy. Access your files from any device or from the web, easily share them with others, and get notified whenever someone changes a file.

      Working on one computer, but need a program from another? No problem. Use Live Mesh to connect to your other computer and access its desktop as if you were sitting right in front of it. "
    • Re: (Score:3, Informative)

      by Z34107 ( 925136 )

      Robocopy works really well, and comes with Vista and up (works on XP, too). More of a bulk file copy than a sync, though.

      Subversion may be overkill, but some nice free servers and clients exist for Windows. TortoiseSVN is a nice client that integrates with the shell; set up a server on a cheap home box and get a free dyndns alias.

      Newer versions of SyncToy seem to work pretty well, and they're stupid simple, but I haven't used them on anything more than a flash drive's worth of data. Ditto for the built-i

  • by langelgjm ( 860756 ) on Tuesday June 23, 2009 @02:29PM (#28443575) Journal

    Strictly speaking this isn't about my "home directories", but I use a combination of things to sync the information I need between my desktop and laptop:

    1. Windows Live Sync (formerly known as Foldershare). Yeah, I know, it's from Microsoft, and only available for Windows and OS X, but it works pretty seamlessly and without much intervention on my part. I sync specific directories of projects I am working on using this; it's a set it and forget it tool.

    2. Foxmarks for bookmarks.

    3. Zotero, version 1.5 of which has the ability to sync resources between multiple computers. This is a godsend.

  • by clang_jangle ( 975789 ) on Tuesday June 23, 2009 @02:29PM (#28443587) Journal
    I use multiple OS X, Linux, and FreeBSD machines daily. One cannot sync all home directory files, as all the config stuff differs between Gentoo, Debian, FreeBSD, Tiger, and Leopard. So it's mostly down to documents, graphics, and a few audio and video files. For the larger ones, I use a usb stick, the smaller ones I email to myself so they're always available via IMAP servers. But most of all I have a bootable, customized version of systemrescuecd installed on a 16GB usb stick, which at any given moment has all the currently important stuff I need. It works well enough for me.
    • I use multiple OSX, Linux and FreeBSD machines daily - The Mac is in my office, the rest are about 45 miles away in our datacenter. I spend 80% of the day logged into various machines remotely and spend a substantial chunk of time copying files from host to host. There's also the headache when dotfiles get out of sync and machines behave differently for what should be identical commands (beyond the usual BSD-vs-SysV crap)

      All by way of saying "Gee, wouldn't it be swell if I didn't have to manually sync t
  • by joe_cot ( 1011355 ) on Tuesday June 23, 2009 @02:30PM (#28443603) Homepage
    • If you're keeping track of code, use a code repository. Subversion, GIT, Bazaar, etc.
    • If you're trying to keep config files, documents, pictures, etc synced, use DropBox [getdropbox.com].
    • For bookmarks, use one of the numerous Firefox bookmark syncing extensions, or the Del.icio.us extension (or use DropBox to sync your .mozilla/firefox folder).
    • For multi-GB files, use a portable hard drive, or rsync with a file server in your house/office

    I wouldn't recommend using one tool for every purpose. I wouldn't want to store multi-GB files in SVN, and I wouldn't want to store all my code on an external hard drive. Maybe using DropBox, or rsyncing with a server somewhere would work.

  • I just store anything saveworthy on the server at home. That way the "client" computers (my main "work"station/gamestation and laptops) contain only relatively expendable data. And if I need to access the stuff from some other location I just SSH into the box.

    And backups are handled from the server manually occasionally to an external USB drive. I know, I need to improve on that part.

  • Not sure how much you use bookmarks, but I maintain a good archive of organized chaos myself. At one time, bookmarking a site was a pretty pointless endeavor, since I couldn't keep track of whether I had bookmarked it at work, on the bedroom computer, downstairs computer, laptop, etc.

    So a while ago, I looked up a plug-in for Firefox called Xmarks (formerly FoxMarks) that was pretty much exactly what I was looking for. Very easy to set up and use, cross-platform, etc.

    Just thought it was useful enough to

  • by gardyloo ( 512791 ) on Tuesday June 23, 2009 @02:32PM (#28443645)

    For small backups, every ten minutes, I use backintime (based on rsync). For larger, nightly or more rare backups, I use rdiff-backup. Both work over the LAN, or to locally-mounted hard drives.

  • USB drive (Score:5, Interesting)

    by the_rajah ( 749499 ) * on Tuesday June 23, 2009 @02:32PM (#28443649) Homepage
    I carry a 16 Gig USB flash drive with my working files on it. I've using this method since the days of 100 Meg Zip drives and just keep upgrading the media. My flash drive is automatically backed up to my backup server at home in the middle of the night so, if I forget it at the office, I'm only a few hours behind. Besides, I can use free Logmein to log into the office computer and transfer a file if it's got new and important information on it. It works the same way in reverse if I forget it at home. Since my working files are on the USB drive which is also compatible with my Linux machines, it really doesn't make much difference which machine I plug it into. Did I mention encryption? That's a good idea in case you lose the drive if you've got any sensitive information on it.
    • My flash drive is automatically backed up...

      Now that's a neat trick. How did you make it plug itself in? Mine won't do that - it won't even come when I shout. Maybe it's time for an upgrade.

  • by necro351 ( 593591 ) on Tuesday June 23, 2009 @02:32PM (#28443653) Journal
    FAST 2009 has a paper on semantic data management using a file system built on top of an object store powered by MySQL. Performance isn't great, but it uses a distributed file system solution to solve the synchronization issue in a very nice way (e.g., synchronize all albums with my iPod, all photos with my laptop and computer, etc...). You can specify rules and I liked it when I heard about it. However performance is actually important, despite their claim :). Perspective: Semantic Data Management for the Home Brandon Salmon, Carnegie Mellon University; Steven W. Schlosser, Intel Research Pittsburgh; Lorrie Faith Cranor and Gregory R. Ganger, Carnegie Mellon University HTML Paper http://www.usenix.org/events/fast09/tech/full_papers/salmon/salmon_html/index.html [usenix.org] PDF Paper http://www.usenix.org/events/fast09/tech/full_papers/salmon/salmon.pdf [usenix.org] Slides http://www.usenix.org/events/fast09/tech/slides/salmon.pdf [usenix.org]
  • Beyond Compare (Score:3, Informative)

    by Anonymous Coward on Tuesday June 23, 2009 @02:37PM (#28443729)

    On the windows side there is a great utility called Beyond Compare, around $30, that I have used to do this. I even had a small client once that could not afford a real backup software, so we faked the backup using portable USB hard drives and the Beyond Compare utility to sync her server and desktop to the drives. Worked quite great and the while thing was done for under $200.

  • Windows - SyncBack (Score:4, Interesting)

    by Anonymous Coward on Tuesday June 23, 2009 @02:38PM (#28443755)

    I spent a long time tackling this, as I am situated at different locations on different days.

    I have 2 desktops and a laptop which must remain sync'd and encrypted. I use TrueCrypt for the encryption.

    On my Windows boxes - SyncBack handles it. It can be triggered on write or on insertion, or just periodically. Has version control support. Will sync over FTP (poorly) and can create zip files or burn Cds etc. It's a swiss army knife of sync tools.

    The key for getting the most out of a sync program is granularity. Inevitably, you'll have exceptions, and you don't want a PASS/FAIL result for your entire backup set. It works much better to sort files into categories and sync the individual groups than to try to make one profile that does your entire disk array. My 2 cents.

  • by Drizzt Do'Urden ( 226671 ) on Tuesday June 23, 2009 @02:39PM (#28443765) Homepage

    At home, I've got a Linux server hosting an LDAP structure to mimic MacOS X Server's config. It is sharing my home directory via NFS. My Macs sync this home directory on login and logout, so all my personnal data is centalized for easy backup and available on any Mac I happen to add to my home network.

  • Whenever I'm on my workplace network I have cron job that uses rsync to sync my Documents directory between my linux desktop & mac laptop. This way the latest file is always on both machines. The Desktop is also rsync'd to a backup server daily, and weekly for off site storage at bank vault. And the Laptop uses Time Machine at my home. This allows me the flexibility of grabbing older files if I need them. For mail I use imap with SSL. So short of a nuclear holocaust, it would be pretty hard to lose a f
  • Right now, actual synchronization tends to be entirely manual, with scp of subdirectories and possibly a tar -c | tar -x combo to not overwrite newer copies.

    I decided to work on data integrity first--but then I have client info to consider.

    I have a 3x1.5T zraid array using full disks on the main machine, and an external 1.5T for backup (I'll grab another, so I can have two alternating backups). These will stay disconnected when not backing up, and in two other rooms of the house. I'll probably copy zfs sn

  • FTP (Score:2, Interesting)

    by NineNine ( 235196 )

    FTP back and forth, select the root and overwrite whatever's newer. Unless the time on the files gets screwed up, it works fine. Worst case scenario, which is the dates/times getting messed up, the FTP client downloads everything. No big deal. I do it daily for all kinds of files.

    As older, wiser programmers than myself have always told me: KISS: Keep It Simple, Stupid.

  • rsync (Score:2, Informative)

    Primitive, but it works for me.
    • And it'll work for 99.9% of people.

      I really don't get the desire for people to make shit more complicated and less reliable than it needs to be.

      Very rarely would anyone need anything other than a simple sync job, manual or automated.

  • What?
    If you have enough files and move between systems frequently enough that this is an issue, consider a USB flash drive.

    Store your files there.
    Keep backups on each machine.

    Want versioning? Seems to me that files typically have a datestamp for when they were last modified. That's all the versioning people need 99.9% of the time.

    If you're in a situation where this is a big problem (many users many files many machines), you want your damned files stored in a more permanent location, such as on, I don't kn

    • Oh, and I'm assuming we don't trust /rely on "the cloud" or any 3rd-party online storage system.

      (Because they shouldn't be trusted/relied on, and because in many cases they can't be, legally.)

  • by kylben ( 1008989 ) on Tuesday June 23, 2009 @02:48PM (#28443923) Homepage
    I embed all my documents in porn and post them on various web forums. The recovery procedure involves spidering my spam folder. I recently found my high school history term paper in a jpg of Marylin Chambers.
  • Unison (Score:5, Interesting)

    by ashayh ( 636057 ) on Tuesday June 23, 2009 @02:49PM (#28443941)
    Unison [upenn.edu]
    • I am looking at Unison, and it looks interesting ... I may end up using it.

      However, I'm also considering setting up FileVault on my laptop, because I don't want client data compromised if my laptop gets stolen.

      What's the chance of getting a sync tool like unison working while one (or both) of the computers in questions uses FileVault?

    • I've been a very happy Unison for many many years. But after recently switching to Ubuntu for my laptop and desktop I finally discovered the joy of Unison over SSH and also discovered the auto=true and terse=true command switches which makes the whole thing automatic, just like Dropbox (I think there's one more switch and I won't even have to press 'g' to run the sync).

      I've tried other solutions - DropBox is too much in the 'cloud' for me; rsync is only one way (what's with that?), Robocopy is Windows only,

  • I've used Subversion for years to sync my various systems. I have four different machines (2 Vista, 2 Linux) and 20GB of data that must be kept in sync.

    Of course, there could be somethign much better out there. I'm just very comfortable with Subversion, and it works.

  • Webdav, SVN, etc (Score:3, Informative)

    by fermion ( 181285 ) on Tuesday June 23, 2009 @02:51PM (#28443999) Homepage Journal
    For smaller files, I keep everything controlled using SVN. That is code, office type files, that sort of thing. I have a BASH script that pretty reliably works to commit, add, and update. Everyone once in a while I have to go in and manually fix something. I suppose I could put a chron entry in to make it automatic, but it is just as easy to go to the shell and update everything. Setting up the server was no issue, and it is an offsite backup.

    For items that are larger, or that do not change so often, I use iDisk. This is just a fancy Webdav server that I do not have to manage.

    I keep programs on an external hard disk. This is where I also keep my photo library and music and videos. I use one machine for Photos, so I do not really have anything to sync there. My music is not synced either, but I have used some third party software to hel with that.

    It is getting to the point where if something goes wrong with a machine, I can have new one set up will all my data in a day. In normal circumstances, I can use any one of three machines and prety much have up to date information.

  • I have 1 server computer that stores all videos, photos, music, etc. My other computers and Xbox 360 connect to that one. For work related documents and source code the same server also runs Source Safe.
  • I pay $100 a year for a Panix shell account and keep my data there. My own little server in the sky :-)

  • by Binkleyz ( 175773 ) on Tuesday June 23, 2009 @02:59PM (#28444149) Journal

    Doubletake software makes an enterprise ready, real time replication suite.

    It does block level replication, so only the changed bits of, say, a 10Gb databse gets changed.. It uses on the fly en/decryption so that the data streams are somewhat smaller than they would be otherwise..

    I work for a Fortune 10 company, and when we have a need for real-time data replication, this is what we use.

  • http://www.gbridge.com/ [gbridge.com] very underrated
  • tar cvf - /home | bzip2 | aespipe -w 10 -K /etc/homekey.gpg | bittorrent archive.`date +%s`.foo

  • unison/sshfs/rsync (Score:2, Informative)

    by Cocoronixx ( 551128 )

    My current setup is a combo of unison/sshfs/rsync.. I've been using it for quite a while at this point, and it works so well that I don't even give it any thought anymore.

    I have a Media Center/Fileserver box at my house that is always on, and acts as the 'master' copy of the home directory, on all my workstations my .xinitrc/.xsession calls unison to sync my home dir with the server root as the preferred copy, then calls my WM, after my WM exits, unison is called again with the local homedir as the authorit

  • I tried to sync my homedirs between machines, because it's annoying not having my settings and saved passwords etc on every machine..
    But, the homedir on my desktop is 300gb in size, neither of my laptops even have that much space on them, especially the netbook.
    On the desktop i can keep everything i need, but the netbook needs to keep as small working set of whatever i'm working on at the time.

  • I absolutely love allway sync. Very cheap ( I normally never buy software, so low price helped ) but completely simplifies syncing between two home and one work PC.

    It's never screwed me up once (fingers crossed!)

  • I'd like to give a shout-out to the Samba team and creators of RSync [slashdot.org]. Version 3 and later is what you want. It has many goodies that make backing up a joy, incleding an off-line incremental backup feature that I find really handy.

    Still, I'm looking for a good graphical rsync that does not use cygwin on windows though.

  • I set up another machine with 6x 1TB HDDs, mounted them, enabled them in samba, connect to them on any of my other machines. If you need redundancy, set up RAID or svn/cvs for code revisioning.

    Maybe I misunderstand the question but this seems pretty straightforward for those that might frequent this site. Did the option, "file and repository server" seem too obvious? Why use an online tool or some special software to share between systems? Why store the same data on multiple machines? Just mount a shared
  • I've got a home Subversion server with separate repositories for documents, settings, contacts, and projects. Been like that for five years now, and it's remarkably stable and nice. For anything data intensive, there's Flickr, del.icio.us, Gmail, WordPress, etc., with a private backup just in case.
  • For large amounts of files (100s of GBs), the cheapest way to back them up is to use a couple sets of external hard drives, always keeping at least one set off-site.

    I would like to know why this is the case. Why is there no service out there that can provide backups for large amounts of data at a price that is competitive with using external hard drives? Such a service should be able to take advantage of scale by storing the bulk of the data onto industrial tape drives, and only retrieving it as needed.

    It w

  • for example if I use online syncing utilities for my home drives, how do I know someone else won't have access to them? If I am writing a book or software project, how can I know someone won't steal it from me and claim it as their own?

    I'd like to set up my own file server and have the synch software store the files there or to a USB drive or something. My old method is to burn CD-R and DVD-R disks, but they can get lost or scratched and then the data is gone. I have, for example, a DVD-R full of my old doc

  • by Enahs ( 1606 ) on Tuesday June 23, 2009 @04:09PM (#28445373) Journal

    I don't share EVERYTHING, but I share some things:

    • If I just need to go one way, I use rsync.
    • If I need 2-way sync but no versioning info, I use unison.
    • If I need n-way sync but no versioning info, I use unison with a central "untouchable" folder.
    • If I need versioning info, I use git.
  • ZFS FTW (Score:3, Interesting)

    by jregel ( 39009 ) on Tuesday June 23, 2009 @04:16PM (#28445465) Homepage

    My Linux home directory is pretty tiny - only the dot files for my Linux environment (.gnome etc). I keep all my work documents and files on my OpenSolaris fileserver where ZFS provides resilience using RAID1 and point in time restores using the snapshot capabilities. I NFS mount the ZFS filesystem to my Linux box and CIFS share the same filesystem to my Windows PC and Mac.

    My MP3 collection and photo albums are handled by iTunes and iPhoto respectively, syncing with my iPhone. The Mac backs these up to a Time Machine disk which in reality is a ZVol on my OpenSolaris server published as a LUN using iSCSI.

    The ZFS filesystems and volumes are backed up to external USB drive using the "zfs send" command.

    Blatant plug: I've documented most of the above experience on my blog.

    For bookmarks, I use Xmarks to synchronise with the cloud, and take notes using Evernote.

  • by vyrus128 ( 747164 ) <gwillen@nerdnet.org> on Tuesday June 23, 2009 @04:40PM (#28445839) Homepage

    Currently? Just unison -quiet, running from cron. (I have it wrapped in a script that does locking, since Unison doesn't seem to lock against itself reliably, for reasons I don't understand.) I've had two problems worth watching out for:
    1) Try to avoid running it against NFS. It walks the entire synced area every time you sync. Local disk will be two orders of magnitude faster.
    2) Be careful syncing between case-sensitive and case-insensitive filesystems. Unison will start failing out if you ever create two files differing only in case.

    Beyond that, I'm looking to start using git to version both my code and my textual data. I'm not intending to use git itself to sync the repositories; I'm going to use it for versioning only, and keep syncing using Unison. The reason is because I'm the only user, and for my own convenience I'd like the working copy to be synced. All I really need out of git is versioning anyway; I already have a workable solution for syncing.

  • by EBorisch ( 530762 ) on Tuesday June 23, 2009 @04:45PM (#28445899)

    Nighly (or more frequently, if you like) rsync to an OpenSolaris server running ZFS w/ Time Slider.

    Quality versioned backups with little effort, plus data integrity (checksums built into the filesystem), compression, and (if desired) RAID-Z(2) goodness! In addition, the provided time slider interface allows easy browsing of versions.

    Just my 2c...

  • by cenc ( 1310167 ) on Tuesday June 23, 2009 @04:51PM (#28445987) Homepage

    I use shfs mounts by ( to make sure it stays mounted even if connection is interrupted) and ssh tunnels for everything else, with preshared keys to a central server / proxy, and rsnapshots for backup on the central server with hot swap drives.

    This works on desktops, remote office, and for notebooks. I essentially don't trust my employees or myself to remember to encrypt everything or use "secure" protocals all the time, and so I remove the need to remember from the whole process. I can then focus on securing one system. Great if everything else is secure, but just in case. Very good for notebooks jumping from open wireless to open wireless systems, and also keeping track of employees activity in one location. I can log fairly easily everything they do or don't do (yea, the 2 hour coffee break sticks out like sore thumb in the logs).

    Among other things this also has the nice side effect that should say a notebook or desktop be stolen, it will phone home as soon as it is connected to the internet and send detailed information about what it is doing.

  • by rpwoodbu ( 82958 ) on Tuesday June 23, 2009 @05:43PM (#28446663)

    I have found that using Subversion (svn) with the aid of a bash script that is run manually actually works really well and provides a number of special advantages. Here's how I have it constructed:

    First, I don't actually make my whole home directory a svn checkout. I have a subdirectory in it that is the checkout, and my bash script ensures there are symlinks into it for the things I want sync'd. This makes it easy to have some differences between locations. In particular, I can have a different .bashrc for one machine than another, but keep them both in svn as separate files; it is just a matter of making the symlink point to the one I want to use in each location. My bash script will make the symlink if the file doesn't exist, and warn if the file does exist but isn't a symlink. It does this for a number of files.

    Another benefit of this method is that I don't put all my files in one checkout. The core files I'll want in all my home directories (e.g. .bashrc, .vimrc, ssh .config and public keys, etc.) go in a checkout called "homedir". But my documents go elsewhere. And my sensitive files (e.g. private keys) go somewhere else still. I choose what is appropriate to install at each location (usually just the "homedir" checkout on boxes I don't own). My bash script detects which checkouts I have and does the appropriate steps.

    The bash script not only sets up the symlinks but it also does an "svn status" on each checkout so I'll know if there are any files I've created that I haven't added, or any files I've modified that I haven't committed. I prefer not to automate adds and commits. I'll definitely see any pending things when I run my sync script, and can simply do an "svn add" or "svn commit" as necessary.

    I also prefer not to automate the running of the sync script. I like being in control of my bandwidth usage, especially when connected via slow links (e.g. Verizon EV-DO, AT&T GPRS). Plus dealing with conflicts is much easier when it is interactive (although I can usually avoid that scenario). It also simplifies authentication to run it from my shell, as it can just use my ssh agent (which I forward, which is setup in my sync'd ssh config).

    The sync bash script takes care of a few other edge-case issues, like dealing with files in ~/.ssh that have to have certain permissions and whatnot. And I've taken care to ensure that the script doesn't just blow away files; it will warn if things don't look right, and leaves it to me to fix it.

    Using Subversion has another big advantage: it is likely to be installed already in many places. So when I'm given an account on someone's computer, I can usually get my environment just the way I like it in a few short steps:

    svn co svn+ssh://my.server.tld/my/path/to/svn/trunk/homedir ~/homedir
    ~/homedir/bin/mysync # This is my bash script to do the syncing
    # Correct any complains about .bashrc not being a symlink and whatnot
    ~/homedir/bin/mysync
    # Log out and back in, or source .bashrc

    No fuss, no muss. No downloading some sync package and building it just to get your .bashrc or .vimrc on a random box, or asking the admin to install something. Subversion is usually there, and even if it isn't, most admins are happy to install it. Subversion deals well with binary files, and even large files. For bulk things (like a music library), I'm more likely to rsync it, partly because it is bulk, partly because it doesn't benefit from versioning, and partly because it only needs to be a unidirectional sync. I could easily add that to my sync script.

    I am simply in the habit of typing "mysync" from time to time (my .bashrc puts ~/bin/ in my $PATH). This works for me very nicely. Some people may prefer a little more automation, and of course my script could automatically do adds and commits, and even skip the log messages. But I prefer a bit more process; after all, this is my data we're talking about!

    If there is interest, I may post my sync script.

  • Git (Score:3, Informative)

    by kabloom ( 755503 ) on Tuesday June 23, 2009 @06:04PM (#28446891) Homepage

    I use a constellation of git repositories, and Joey Hess' mr [kitenet.net] tool to synchronize all of them. I have no automated commits -- I just remember to commit and update manually daily.

  • Don't sync home. (Score:4, Insightful)

    by dotwaffle ( 610149 ) <slashdot@walst[ ]org ['er.' in gap]> on Tuesday June 23, 2009 @06:20PM (#28447089) Homepage

    I've done exactly the same as you, used every single tool under the sun, eventually settling on Unison until I realised I was being silly...

    Let's put it this way - just set up each computer how you want it, and sync the *data*, not the whole home directory.

    For instance, my Documents are synced with Dropbox (though tempted to move them to UbuntuOne), my development directories are generally stored in some kind of revision control (svn/bzr/git) and either not synced or at worst, unison-ed, and everything else just stays on the machine it was created on, and backed up with duplicity to a central fileserver hosted in France.

    When you realise that syncing home is *not* good, it suddenly becomes clear what you need, and what you want are completely different.

"I've finally learned what `upward compatible' means. It means we get to keep all our old mistakes." -- Dennie van Tassel

Working...