How to identify duplicate files in a directory via Linux command line?

Duplicated files

It often happens that we have folders full of images or documents where a lot of them are duplicated. However, Finding duplicated files is a very tedious task. Hence, we are interested in an automatic way to identify such files. Here, I focus on Ubuntu Linux command-line solutions.

Solution 1: fdupes dedicated application

The first solution is to use an automatic way via an application called fdupes.

First, you need to install this application via apt command as follows:

sudo apt install fdupes

Then, navigate to the target folder and run the command to find duplicated files:

fdupes .

Solution 2: Linux pipeline md5 uniqueness

Sometimes it is not easy or event not desirable to install a new application. In such cases, there is another solution to identify duplicate files without installing any application as follows:

find . -type f -print0 | xargs -0 md5sum | sort | uniq -w32 --all-repeated=separate

This solution will cover the current folder as well as all sub-folders.

linux
apt
ubuntu
cli
Software and digital electronics / IT
Posted by landi
2024-04-08 21:24
×

Login

No account?
Terms of use
Forgot password?