AdvancedLesson 30 of 309 min read

Capstone: a file-organising automation tool

Build a complete automation tool: sort a folder by file type, avoid name clashes, support a dry run, write a JSON report, test it and schedule it.

In this lesson you will

  • Structure an automation tool as small, testable functions
  • Add a command-line interface with a dry run
  • Protect the tool with pytest tests
  • Schedule scripts to run automatically

About 35 minutes

Your Python capstone: let the computer do the boring work

Your Downloads folder holds hundreds of files: photos, PDFs, spreadsheets, zip files, all mixed together. Every month you tidy it by hand and hate it. In this final lesson you will build an automation tool that sorts any folder into tidy sub-folders by file type, writes a report, and can be run on a schedule. It uses almost everything you have learned: files and paths, functions, dataclasses, error handling, JSON, dates, command-line options, logging, a context manager, and tests.

Plan the tool

  • Look at every file in a chosen folder.
  • Decide a category from its extension: images, documents, spreadsheets, archives, other.
  • Move it into a sub-folder, never overwriting a file with the same name.
  • Support a dry run that only shows what would happen.
  • Write a JSON report of what was moved.
  • Be safe: skip folders, handle errors, log everything.

Step 1: the core logic

# organiser.py
"""Sort the files in a folder into sub-folders by type."""
import json
import logging
import shutil
from dataclasses import dataclass, asdict
from datetime import datetime
from pathlib import Path

log = logging.getLogger("organiser")

CATEGORIES = {
    "images": {".jpg", ".jpeg", ".png", ".gif", ".webp"},
    "documents": {".pdf", ".doc", ".docx", ".txt", ".md"},
    "spreadsheets": {".xls", ".xlsx", ".csv"},
    "archives": {".zip", ".rar", ".7z", ".tar", ".gz"},
}


REPORT_NAME = "organiser-report.json"


@dataclass
class Move:
    name: str
    category: str
    destination: str


def categorise(path: Path) -> str:
    ext = path.suffix.lower()
    for category, extensions in CATEGORIES.items():
        if ext in extensions:
            return category
    return "other"


def unique_destination(folder: Path, name: str) -> Path:
    """Return a path that does not exist yet: photo.jpg, photo (1).jpg, ..."""
    target = folder / name
    stem, suffix, n = target.stem, target.suffix, 1
    while target.exists():
        target = folder / f"{stem} ({n}){suffix}"
        n += 1
    return target


def organise(root: Path, dry_run: bool = False) -> list[Move]:
    moves: list[Move] = []
    for item in sorted(root.iterdir()):
        if not item.is_file() or item.name.startswith(".") or item.name == REPORT_NAME:
            continue
        category = categorise(item)
        folder = root / category
        target = unique_destination(folder, item.name)
        try:
            if not dry_run:
                folder.mkdir(exist_ok=True)
                shutil.move(str(item), str(target))
        except OSError as error:
            log.error("Could not move %s: %s", item.name, error)
            continue
        log.info("%s%s -> %s", "[dry run] " if dry_run else "", item.name, category)
        moves.append(Move(item.name, category, str(target.relative_to(root))))
    return moves


def write_report(root: Path, moves: list[Move]) -> Path:
    report = {
        "folder": str(root),
        "created": datetime.now().isoformat(timespec="seconds"),
        "total": len(moves),
        "by_category": {},
        "moves": [asdict(m) for m in moves],
    }
    for m in moves:
        report["by_category"][m.category] = report["by_category"].get(m.category, 0) + 1
    path = root / REPORT_NAME
    path.write_text(json.dumps(report, indent=2), encoding="utf-8")
    return path

Look at how the pieces are shaped. categorise and unique_destination are tiny and pure. organise does the work and returns a list of Move records instead of printing. write_report turns those records into JSON. Nothing here talks to the user, which keeps it easy to test and reuse.

Step 2: the command-line front end

# cli.py
import argparse
import logging
from pathlib import Path

from organiser import organise, write_report


def main(argv=None):
    parser = argparse.ArgumentParser(description="Sort a folder's files by type.")
    parser.add_argument("folder", type=Path, help="the folder to tidy")
    parser.add_argument("--dry-run", action="store_true", help="show what would happen")
    parser.add_argument("-q", "--quiet", action="store_true", help="only show warnings")
    args = parser.parse_args(argv)

    logging.basicConfig(level=logging.WARNING if args.quiet else logging.INFO,
                        format="%(message)s")

    if not args.folder.is_dir():
        parser.error(f"{args.folder} is not a folder")

    moves = organise(args.folder, dry_run=args.dry_run)
    if not args.dry_run and moves:
        report = write_report(args.folder, moves)
        print(f"Report written to {report.name}")
    print(f"{'Would move' if args.dry_run else 'Moved'} {len(moves)} file(s)")


if __name__ == "__main__":
    main()

Step 3: run it on a messy folder

Below, a throw-away folder is filled with sample files, then tidied. First a dry run, then the real thing. This is real output from running exactly the code above.

$ mkdir Downloads && touch Downloads/photo.jpg Downloads/holiday.png Downloads/notes.txt Downloads/budget.xlsx Downloads/backup.zip Downloads/report.pdf Downloads/script.py && python cli.py Downloads --dry-run && echo '--- real run ---' && python cli.py Downloads && ls Downloads && echo '--- a new photo.jpg arrives, run again ---' && touch Downloads/photo.jpg && python cli.py Downloads -q && ls Downloads/images
[dry run] backup.zip -> archives
[dry run] budget.xlsx -> spreadsheets
[dry run] holiday.png -> images
[dry run] notes.txt -> documents
[dry run] photo.jpg -> images
[dry run] report.pdf -> documents
[dry run] script.py -> other
Would move 7 file(s)
--- real run ---
backup.zip -> archives
budget.xlsx -> spreadsheets
holiday.png -> images
notes.txt -> documents
photo.jpg -> images
report.pdf -> documents
script.py -> other
Report written to organiser-report.json
Moved 7 file(s)
archives
documents
images
organiser-report.json
other
spreadsheets
--- a new photo.jpg arrives, run again ---
Report written to organiser-report.json
Moved 1 file(s)
holiday.png
photo (1).jpg
photo.jpg

Look at the second run: only the new photo.jpg moved, and it was renamed photo (1).jpg instead of overwriting the first. Running the tool twice also caught a real bug while this lesson was being written: the first version moved its own report file into the other folder on the second run. That is why organise now skips REPORT_NAME. Running your tool more than once, on realistic data, finds problems that a single run hides.

Step 4: protect it with tests

from pathlib import Path
from organiser import categorise, organise


def make(folder, *names):
    for name in names:
        (folder / name).write_text("x")


def test_categorise():
    assert categorise(Path("Holiday.JPG")) == "images"
    assert categorise(Path("budget.xlsx")) == "spreadsheets"
    assert categorise(Path("mystery.xyz")) == "other"


def test_files_are_moved(tmp_path):
    make(tmp_path, "a.jpg", "b.pdf")
    organise(tmp_path)
    assert (tmp_path / "images" / "a.jpg").exists()
    assert (tmp_path / "documents" / "b.pdf").exists()
    assert not (tmp_path / "a.jpg").exists()


def test_names_never_clash(tmp_path):
    make(tmp_path, "a.jpg")
    organise(tmp_path)
    make(tmp_path, "a.jpg")          # a second file with the same name
    organise(tmp_path)
    names = sorted(p.name for p in (tmp_path / "images").iterdir())
    assert names == ["a (1).jpg", "a.jpg"]


def test_dry_run_changes_nothing(tmp_path):
    make(tmp_path, "a.jpg")
    moves = organise(tmp_path, dry_run=True)
    assert len(moves) == 1
    assert (tmp_path / "a.jpg").exists()
    assert not (tmp_path / "images").exists()
$ python -m pytest -q
....                                                                     [100%]
4 passed in 0.01s

The tests use tmp_path so they never touch real files. They check the three behaviours that matter most: files go to the right place, names never clash, and a dry run changes nothing. If you later edit the tool, running pytest tells you immediately whether you broke something.

Step 5: run it automatically

An automation tool is only truly useful when you do not have to remember it. Schedule it with your operating system. The commands below run on your own computer, so they are shown as recipes, not run output.

# Windows (Task Scheduler), tidy Downloads every day at 9 pm
schtasks /create /tn "TidyDownloads" /tr "python C:\\tools\\cli.py C:\\Users\\you\\Downloads -q" /sc daily /st 21:00

# Mac or Linux (cron): run crontab -e and add this line
0 21 * * * /usr/bin/python3 /home/you/tools/cli.py /home/you/Downloads -q

Ideas for your own automations

  • Rename photos by their date taken.
  • Merge a month of CSV exports into one report with pandas.
  • Send yourself a daily summary email with smtplib.
  • Back up a folder into a dated zip file with shutil.make_archive.
  • Check a list of web pages and warn you when one is down.
  • Watch an inbox folder and process every new file.

Where to go from here

You have come a long way: from print("Hello") to decorators, async code, testing, data analysis and web apps. Here are sensible next steps, depending on what excites you:

Direction Next things to learn
Web development Django, databases (SQL), deploying a site, HTML/CSS/JavaScript (courses on this site)
Data and AI NumPy, matplotlib, scikit-learn, Jupyter notebooks, SQL
Automation Selenium or Playwright for browsers, APIs, scheduled scripts, cloud functions
Software engineering Git and GitHub, continuous integration, design patterns, Docker
Build, publish, repeat

Put this tool on GitHub with a good README. Then build the next one. A few finished, public projects teach more and impress more than any certificate. Congratulations on completing the course!

Common mistake

Running a file-moving script for the first time on your only copy of important files. Always do a --dry-run first, and test on a copy of the folder. Automation makes mistakes faster, too.

Practice

Add a new category code for .py, .js and .html files to the tool. Which two places need changing?

Show one possible answer
CATEGORIES = {"images": {".jpg"}}   # stands in for the dictionary in organiser.py

# 1. organiser.py: add one entry to CATEGORIES
CATEGORIES["code"] = {".py", ".js", ".html", ".css"}

# 2. tests: add a case to the categorise test, for example
#    assert categorise(Path("script.py")) == "code"
# Nothing else changes: the categorise function reads the dictionary.
Practice

Add a --min-size option that skips files smaller than a number of kilobytes. Describe the change in a sentence or two.

Show one possible answer
# cli.py: parser.add_argument("--min-size", type=int, default=0, help="skip files below this size in KB")
# organiser.py: add a min_kb parameter to organise() and, inside the loop,
#     if item.stat().st_size < min_kb * 1024: continue
# Then pass args.min_size from main() and add a test with a tiny file.

Check what you learned

Five quick questions. Pick an answer and you will see straight away whether it is right and why.

Why does the core logic return a list of Move records instead of printing?

  1. It is easier to test and reuse
  2. It runs faster
  3. Python requires it
  4. It uses less memory

Separating logic from output keeps code flexible.

What is the purpose of a dry run?

  1. To delete files
  2. To show what would happen without changing anything
  3. To speed up the tool
  4. To compress files

It lets you check before acting.

How does the tool avoid overwriting a file with the same name?

  1. It deletes the old file
  2. It ignores the new one
  3. It crashes
  4. It adds (1), (2) and so on to the name

unique_destination finds a free name.

Why do the tests use tmp_path?

  1. To run faster
  2. So they never touch real files
  3. To avoid asserts
  4. To import modules

Each test gets its own temporary folder.

How can a script run automatically every day?

  1. It cannot
  2. Only manually
  3. By restarting Python
  4. Task Scheduler on Windows or cron on Mac and Linux

The operating system can launch it on a schedule.

Summary

  • Small functions plus a thin front end make tools testable.
  • Always offer a dry run for destructive actions.
  • Never overwrite: generate unique names.
  • Tests and scheduling turn a script into a dependable tool.

Next lesson

Congratulations! You have completed the Python course. Try one of the other courses, or build your own project.

Questions

Is it safe to run this on my real Downloads folder?

Run it with --dry-run first and try a copy of the folder. The tool never overwrites files, but moving files is still a change you should check.

How do I make the tool ignore some folders?

It already skips folders and hidden files because it only moves files. Add your own checks inside the loop in organise().

Can this become a desktop app or a website?

Yes. The logic is separate from the interface, so you could wrap it in Flask for a web page or in a small desktop interface without rewriting it.

Progress is saved in this browser.