Organizing Files

Overview

Earlier, we have learned how to create and write new files. Python also has the has the ability to organize pre-existing files in your directory, adding to its powerful scripting capabilities. For this project, we will tackle how to automatically back-up our files at will.

Prior Setup

With working with files, it is important to know the extensions of a file (.txt, .pdf, .doc, .jpg, etc.)

For computers running Windows, file extensions may be hidden. To show them, go to Start -> Control Panel -> Appearance and Personalization -> Folder Options. On the pop-up dialog, go to View -> Advanced Settings, and uncheck "Hide extensions for known filetypes".

New Concepts

The shutil Module

The shutil, or shell utilities, module has functions that allow users to copy, move, and delete files. We will need to import it with "import shutil"

Copying files and Folders

shutil.copy(source, destination) will copy a file from the source to the destination. If destination path ends with a filename, the copied file will have that name. If the destination path ends with a folder name, the copied file will be the same as the original. The function will return a string of the path of the copied file. shutil.copytree(source, destination) is similar to shutil.copy() but will also copy folders.

Moving and Renaming Files

In the command line, moving and renaming files are done with the same command. The way one can see this is that when you are renaming a file, you are actually moving the file from one name to another name. This is done with the function, shutil.move(source, destination).

Deleting Files and Folders

os.unlink(path) will delete the file at path. os.rmdir(path) will delete the folder at path. This will only delete an empty folder shutil.rmtree(path) will remove the folder at path and everything it contains

Example

# Deletes any file that ends in .txt
import os
for filename in os.listdir():
  if filename.endswith('.txt'):
    print(filename)
    os.unlink(filename)

Walking a Directory Tree

When you want to visit every single file and folder in a directory, use os.walk(path).

os.walk() returns three values, 1.) A string of the current folder's name, 2.) A list of strings of the folders in the current folder, 3.) A list of strings of the files in the current folder. This allows us to use three variables: folderName, subfolders, and filenames to store these three values. We can also use subfolders and filenames in their own for loops since they are lists.

# Prints every single file and folder in directory
import os

for folderName, subfolders, filenames in os.walk(os.path.abspath('.')):
  print('The current folder is ' + foldername)

  for subfolder in subfolders:
    print('SUBFOLDER OF ' + folderName + ': ' + subfolder)
  for filename in filenames:
    print('FILE INSIDE ' + folderName + ': ' + filename)
  print('')

Compressing Files with the zipfile Module

A ZIP file is an archive file that allows the storage of multiple files and folders into one.

Reading ZIP Files

To read a ZIP file, we must create a ZipFile object first. This is similar to the File object created for file input/output. To create a ZipFile object, call zipfile.ZipFile('name.zip'). zipfile is the name of the Python module. Create a zipfile called "example.zip" to test this program and try it out with the Python interactive shell

import zipfile, os
exampleZip = zipfile.ZipFile('cuteStuff.zip')
exampleZip.namelist()
fileInfo = exampleZip.getinfo('zergRush.jpg')
fileInfo.file_size
fileInfo.compress_size
print('Compressed file is %sx smaller!' % (round(fileInfo.file_size / fileInfo.compress_size, 2)))
exampleZip.close()

namelist() returns a list of strings for all files and folders in the ZIP file. getinfo() can be used on one of these strings to get the file_size() and compress_size() in bytes.

Extracting from ZIP Files

Use extractall() to place all files and folders into the current working dir. You can also pass a path inside extractall() to store the files into a folder other than the current working directory. If the folder does not exist, one will be created

# Extracting everything
import zipfile,os
exampleZip = zipfile.ZipFile('cuteStuff.zip')
exampleZip.extractall()
exampleZip.close()

# Extracting a single file
# exampleZip.extract('zergRush.jpg')
# exampleZip.extract('zergRush.jpg', 'some_other_dir')

Creating and Adding to ZIP files

To create a compressed ZIP file, open the ZipFile object with 'w' to write. Use the write() method to compress a file and add it to the zipfile. The first argument is the name of the file, and the second argument is the compression type (just keep it as zipfile.ZIP_DEFLATED). This will erase any existing ZIP file. If you want to append, or add to a ZIP file, pass 'a' for append.

import zipfile
newZip = zipfile.ZipFile('new.zip', 'w')
newZip.write('zergRush.jpg', compress_type=zipfile.ZIP_DEFLATED)
newZip.close()

Activity

Towards the end of 2017, Apple Inc. has been caught slowing down older iPhones. Apple claims this is due to preserving stability caused by dying batteries. Forbes does not agree. There are many reporters and tech aficionados who claim this is further evidence of "Planned Obsolescence", which is a term that describes companies designing products to fail after a certain amount of time to cause a user to "upgrade" by buying a new product after a certain amount of time. This is done to increase profits.

Planned Obsolescence is a policy that's happening now and although devices are always replaceable; your photos and important documents are not. To kick consumers while they are down, many data recovery services charge $1 per 1 GB so recovering 100 GB will cost you $100. How many GB of data do you have in your computer? Too many, that's how many.

Let's protect ourselves.

Objective

We will create a Python project that will allow us to back up a folder into a zip file. It is usually a good policy to back-up often so our program will allow the user to backup several versions of the same folder. (e.g. photos_1.zip, photos_2.zip, photos_3.zip, etc)

Step 1 - Figure out the ZIP File's Name Import the zipfile and os modules. We will use the basename of our folder and append a version number at the end. This is done by checking if there are previous versions first. We will store our new zipfile name into a separate string variable

Step 2 - Create the New ZIP File Create a zipfile object with our new zipfile name and 'w' to allow us to write into it.

Step 3 - Walk the directory and add to the ZIP File Use a series of for-loops and zipfile methods to add the files into our zipfile object.

Solution

Unzip our example folder first before you run the program

Implementation

Example folder

Summary

The os and shutil modules provides functions for copying, moving, renaming, and deleting files. When wanting to perform operations on every file and folder within a directory, use the os.walk() function. The zipfile module allows you to compress and extract .zip files. All together, we can use this to make a basic backup script