It often happens that we have folders full of images or documents where a lot of them are duplicated. However, Finding duplicated files is a very tedious task. Hence, we are interested in an automatic way to identify such files. Here, I focus on Ubuntu Linux command-line solutions.
The first solution is to use an automatic way via an application called fdupes.
First, you need to install this application via apt command as follows:
sudo apt install fdupes
Then, navigate to the target folder and run the command to find duplicated files:
fdupes .
Sometimes it is not easy or event not desirable to install a new application. In such cases, there is another solution to identify duplicate files without installing any application as follows:
find . -type f -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate
This solution will cover the current folder as well as all sub-folders.