Comparing Directories Using fdupes: Identifying and Deleting Duplicates
fdupes is a command-line utility for identifying and deleting duplicate files within one or more directories. This article will cover how to use fdupes to compare the contents of two directories (dir1 and dir2), identify files present in dir2 that are also in dir1, and delete the duplicates in dir2.
Installing fdupes
Before using fdupes, it must be installed. On Ubuntu or Debian-based systems, this can be done using the following command:
sudo apt-get install fdupesComparing Directories with fdupes
To compare the contents of two directories using fdupes, navigate to the parent directory of both directories and use the following command:
fdupes dir1 dir2This will display a list of all duplicate files found in both directories. By default, fdupes will only display one instance of each duplicate file, with the path to the file in dir1 and dir2 separated by a tab character.
Identifying Duplicate Files in dir2
To identify files present in dir2 that are also in dir1, and delete the duplicates in dir2, the following command can be used:
fdupes -rN dir1/ dir2/ | grep dir2/ | cut -f 2- -d " " | xargs rmThis command uses the following options:
-r: Recursively search directories-N: Don't delete files in the first directorygrep dir2/: Only show files in dir2cut -f 2- -d " ": Remove the path to the file in dir1xargs rm: Remove the duplicate file in dir2
fdupes is a powerful command-line utility for identifying and deleting duplicate files within one or more directories. By using the options and commands outlined in this article, you can easily compare the contents of two directories, identify duplicate files in one directory, and delete the duplicates in the other directory.