Please create an account to participate in the Slashdot moderation system

 



Forgot your password?
typodupeerror
What's the story with these ads on Slashdot? Check out our new blog post to find out. ×
Data Storage Hardware

Ask Slashdot: How Do I De-Dupe a System With 4.2 Million Files? 440

First time accepted submitter jamiedolan writes "I've managed to consolidate most of my old data from the last decade onto drives attached to my main Windows 7 PC. Lots of files of all types from digital photos & scans to HD video files (also web site backup's mixed in which are the cause of such a high number of files). In more recent times I've organized files in a reasonable folder system and have an active / automated backup system. The problem is that I know that I have many old files that have been duplicated multiple times across my drives (many from doing quick backups of important data to an external drive that later got consolidate onto a single larger drive), chewing up space. I tried running a free de-dup program, but it ran for a week straight and was still 'processing' when I finally gave up on it. I have a fast system, i7 2.8Ghz with 16GB of ram, but currently have 4.9TB of data with a total of 4.2 million files. Manual sorting is out of the question due to the number of files and my old sloppy filing (folder) system. I do need to keep the data, nuking it is not a viable option.
This discussion has been archived. No new comments can be posted.

Ask Slashdot: How Do I De-Dupe a System With 4.2 Million Files?

Comments Filter:

Related Links Top of the: day, week, month.

As long as we're going to reinvent the wheel again, we might as well try making it round this time. - Mike Dennison

Working...