Merge pull request #288 from TaniaMalhotra/remove_duplicate_files

Added script to remove duplicate files

Author: SANKET7738

Python/remove_duplicate_files/README.md

@@ -0,0 +1,12 @@
+# Remove duplicate files from directory
+- - - - - - - - - - - - -
+## Aim
+Aim of the script is to remove duplicate files from a directory.
+
+## To run
+- Run ```remove.py```
+- Enter the path to the directory or sub-directory in which you want to delete duplicate files</br>
+
+## Example
+The current directory contains ```file1.txt``` and  ```file2.txt``` that have the same contents. </br>
+Running ```remove.py``` will remove ```file2.txt``` as it is a duplicate of file1.txt

Python/remove_duplicate_files/file1.txt

@@ -0,0 +1 @@
+This file is for testing

Python/remove_duplicate_files/file2.txt

@@ -0,0 +1 @@
+This file is for testing

Python/remove_duplicate_files/remove.py

@@ -0,0 +1,18 @@
+# python program to remove duplicate files from directory
+import os
+import hashlib
+import glob
+#taking path of directory as user input
+PATH_directory = input("Please enter the path of directory in which you wish to remove duplicate files  :  ")
+unique_files = []
+for file in os.listdir(PATH_directory):
+    # checking if path is existing file
+    if os.path.isfile(file):
+        hash = hashlib.md5(open(file, 'rb').read()).hexdigest()
+        # if hash of the file in consideration is unique
+        if hash not in unique_files:
+            unique_files.append(hash)
+        # delete file if it has a matching hash
+        else:
+            print("Found a duplicate. Now removing duplicate file")
+            os.remove(file)