I’m working on a new ProgClub project called pcdedupe. It’s a file system de-duplicator and it’s a C++ system based on rdfind. I haven’t created the project page on the wiki yet, but the source code is available.

Basically I’m going to take a new angle on the rdfind software and tailor it to suit my particular environment (I have ten million files with massive duplication and rdfind isn’t optimised for that kind of scale).

I’ve set up a new file server

I’ve been having some fun over the last day or two looking over all my old files. I’ve got files that go back as far as 1999 in my archives. I’ve found my old blog database and associated files, so I hope to get that back up again soon, and I found some old code that I’ve been looking for (I don’t want to have to write it again!).

So my new file server has 6TB of storage as 3 x 2TB partitions. I can fit all my data in 1.3TB of space, so I’m planning to have one file share, and then a backup of that onto another partition. I have 10,174,633 files in my archive folder, and many more in my media, download and home folders. I might publish some more stats once du -s has finished processing. :)

I’m running Ubuntu 10.04 LTS Server as my file server. I tried to setup the Desktop version but it wouldn’t play nice with my nVidia graphics card.