DupeNuker

DupeNuker uses the CRC32 algorithm to create very small “fingerprints” that identify a file. The purpose of this is to automatically identify duplicate files and delete them.

What makes DupeNuker unique is that you don’t need to have the original files in order to detect duplicates. This method is ideal for removing spam from files that you download from Usenet newsgroups.