pytorch_project_template

PyTorch Template for DL projects

About

This repository contains a template for PyTorch-based Deep Learning projects.

The template utilizes different python-dev techniques to improve code readability. Configuration methods enhance reproducibility and experiments control.

The repository is released as a part of the HSE DLA course, however, can easily be adopted for any DL-task.

This template is the official recommended template for the EPFL CS-433 ML Course.

New: we added a HF Main variant of the template with HuggingFace Integration for multi-GPU and multi-node training, automatic mixed precision, gradient accumulation, and seamless HuggingFace Ecosystem Compatibility.

📖 If you use this template in your work, please cite this repository or include a reference. Attribution supports the project and encourages continued development.

Tutorials

This template utilizes experiment tracking techniques, such as WandB and Comet ML, and Hydra for the configuration. It also automatically reformats code and conducts several checks via pre-commit. If you are not familiar with these tools, we advise you to look at the tutorials below:

• Python Dev Tips: information about Git, pre-commit, Hydra, and other stuff for better Python code development. The YouTube recording of the workshop is available here.

• Seminar on R&D Coding 2025: Seminar from the LauzHack Deep Learning Bootcamp with discussion on logging, project-based coding, configuration, and reproducibility. The materials can be found here.

• Seminar on R&D Coding 2024: Seminar from the LauzHack Deep Learning Bootcamp with template discussion and reasoning. It also explains how to work with WandB. The seminar materials can be found here.

• HSE DLA Course Introduction Week: combines the two seminars above into one with some updates, including an extra example for Comet ML.

• PyTorch Basics: several notebooks with PyTorch basics and corresponding seminar recordings from the LauzHack Deep Learning Bootcamp.

To start working with a template, just click on the use this template button.

You can choose any of the branches as a starting point. Set your choice as the default branch in the repository settings. You can also delete unnecessary branches.

Examples

[!IMPORTANT] The main branch leaves some of the code parts empty or fills them with dummy examples, showing just the base structure. The final users can add code required for their own tasks.

You can find examples of this template completed for different tasks in other branches:

• HF Main: the variant of the main branch with HuggingFace Integration. Supports multi-GPU and multi-node training, automatic mixed precision, gradient accumulation, and seamless HuggingFace Ecosystem Compatibility.

• Image classification: simple classification problem on MNIST and CIFAR-10 datasets.

• ASR: template for the automatic speech recognition (ASR) task. Some of the parts (for example, collate_fn and beam search for text_encoder) are missing for studying purposes of HSE DLA course.

Installation

Installation may depend on your task. The general steps are the following:

0. (Optional) Create and activate new environment using conda or venv (+pyenv).

   a. conda version:

   # create env
   conda create -n project_env python=PYTHON_VERSION
   
   # activate env
   conda activate project_env
   

   b. venv (+pyenv) version:

   # create env
   ~/.pyenv/versions/PYTHON_VERSION/bin/python3 -m venv project_env
   
   # alternatively, using default python version
   python3 -m venv project_env
   
   # activate env
   source project_env/bin/activate
   

1. Install all required packages

   pip install -r requirements.txt
   

2. Install pre-commit:

   pre-commit install
   

How To Use

To train a model, run the following command:

python3 train.py -cn=CONFIG_NAME HYDRA_CONFIG_ARGUMENTS

Where CONFIG_NAME is a config from src/configs and HYDRA_CONFIG_ARGUMENTS are optional arguments.

To run inference (evaluate the model or save predictions):

python3 inference.py HYDRA_CONFIG_ARGUMENTS

Useful Links:

You may find the following links useful:

• Report branch: Guidelines for writing a scientific report/paper (with an emphasis on DL projects).

• CLAIRE Template: additional template by EPFL CLAIRE Laboratory that can be combined with ours to enhance experiments reproducibility via Docker.

• Mamba and Poetry: alternatives to Conda and pip package managers given above.

• Awesome README: a list of awesome README files for inspiration. Check the basics here.

Credits

This repository is based on a heavily modified fork of pytorch-template and asr_project_template repositories.

License