lightly

LightlySSL is a computer vision framework for self-supervised learning and latent world models.

• Documentation
• Github
• Discord

For a commercial version with more features, including Docker support and pretraining models for embedding, classification, detection, and segmentation tasks with a single command, please contact [email protected].

We've also built a whole platform on top, with additional features for active learning and data curation. If you're interested in the LightlyStudio, our all-in-one product for data curation and labeling on your data, check out lightly.ai. It's free to get started!

News 🚀

• October 26, 2026 - LightlySSL now trains world models! Learn how a scene responds to actions with LeWM and plan with it in our new tutorial. 🌍
• May 28, 2026 - We now have LeJEPA support for self-supervised learning! 🧬
• March 23, 2026 - Check out our latest open-source project LightlyStudio to visualize, annotate, and manage your data with ease! 🔍
• April 15, 2025 - We are excited to announce that you can now leverage SSL and distillation pretraining in just a few lines of code! We've worked hard to make self-supervised learning even more accessible with our new project LightlyTrain. Head over there to get started and supercharge your models! ⚡️

World Models 🌍

A world model learns how a scene responds to actions. LeWM learns it from raw frames and actions, then plans by imagining the outcome of each move in its own embedding space.

pip install "lightly[timm]"  # LeWM needs PyTorch 2.0+

• Tutorial: Train a World Model and Plan with It
• Example script
• LeWM docs
• 

Features

This self-supervised learning framework offers the following features:

• Modular framework, which exposes low-level building blocks such as loss functions and model heads.
• Easy to use and written in a PyTorch-like style.
• Supports custom backbone models for self-supervised pre-training.
• Support for distributed training using PyTorch Lightning.

Supported Models

You can find sample code for all the supported models here. We provide PyTorch, PyTorch Lightning, and PyTorch Lightning distributed examples for all models to kickstart your project.

Models:

Model	Year	Paper	Docs	Colab (PyTorch)	Colab (PyTorch Lightning)
MoCo	2019	paper	docs
SimCLR	2020	paper	docs
BYOL	2020	paper	docs
SwaV	2020	paper	docs
DenseCL	2020	paper	docs
SimSiam	2020	paper	docs
Barlow Twins	2021	paper	docs
DetConS	2021	paper	docs
DINO	2021	paper	docs
NNCLR	2021	paper	docs
VICReg	2021	paper	docs
DCL & DCLW	2021	paper	docs
MAE	2021	paper	docs
iBOT	2021	paper	docs
SimMIM	2021	paper	docs
MSN	2022	paper	docs
PMSN	2022	paper	docs
DINOv2	2023	paper	docs
FroSSL	2023	paper	docs
AIM	2024	paper	docs
CAPI	2025	paper	docs
LeJEPA	2025	paper	docs
Pixio	2025	paper	docs

World models:

Model	Year	Paper	Docs	Colab (PyTorch)
LeWM	2026	paper	docs

Tutorials

Want to jump to the tutorials and see Lightly in action?

• Train a World Model and Plan with It
• Train MoCo on CIFAR-10
• Train SimCLR on Clothing Data
• Train SimSiam on Satellite Images
• Use Lightly with Custom Augmentations
• Pre-train a Detectron2 Backbone with Lightly
• Finetuning Lightly Checkpoints
• Using timm Models as Backbones

Community and partner projects:

• On-Device Deep Learning with Lightly on an ARM microcontroller

Quick Start

Lightly requires Python 3.8+. We recommend installing Lightly in a Linux or OSX environment. Python 3.13 is not yet supported, as PyTorch itself lacks Python 3.13 compatibility.

Dependencies

Due to the modular nature of the Lightly package some modules can be used with older versions of dependencies. However, to use all features as of today lightly requires the following dependencies:

• PyTorch>=1.11.0
• Torchvision>=0.12.0
• PyTorch Lightning>=1.7.1

Lightly is compatible with PyTorch and PyTorch Lightning v2.0+!

Installation

You can install Lightly and its dependencies from PyPI with:

pip3 install lightly

We strongly recommend installing Lightly in a dedicated virtualenv to avoid conflicts with your system packages.

Lightly in Action

With Lightly, you can use the latest self-supervised learning methods in a modular way using the full power of PyTorch. Experiment with various backbones, models, and loss functions. The framework has been designed to be easy to use from the ground up. Find more examples in our docs.

import torch
import torchvision

from lightly import loss
from lightly import transforms
from lightly.data import LightlyDataset
from lightly.models.modules import heads


# Create a PyTorch module for the SimCLR model.
class SimCLR(torch.nn.Module):
    def __init__(self, backbone):
        super().__init__()
        self.backbone = backbone
        self.projection_head = heads.SimCLRProjectionHead(
            input_dim=512,  # Resnet18 features have 512 dimensions.
            hidden_dim=512,
            output_dim=128,
        )

    def forward(self, x):
        features = self.backbone(x).flatten(start_dim=1)
        z = self.projection_head(features)
        return z


# Use a resnet backbone from torchvision.
backbone = torchvision.models.resnet18()
# Ignore the classification head as we only want the features.
backbone.fc = torch.nn.Identity()

# Build the SimCLR model.
model = SimCLR(backbone)

# Prepare transform that creates multiple random views for every image.
transform = transforms.SimCLRTransform(input_size=32, cj_prob=0.5)


# Create a dataset from your image folder.
dataset = LightlyDataset(input_dir="./my/cute/cats/dataset/", transform=transform)

# Build a PyTorch dataloader.
dataloader = torch.utils.data.DataLoader(
    dataset,  # Pass the dataset to the dataloader.
    batch_size=128,  # A large batch size helps with the learning.
    shuffle=True,  # Shuffling is important!
)

# Lightly exposes building blocks such as loss functions.
criterion = loss.NTXentLoss(temperature=0.5)

# Get a PyTorch optimizer.
optimizer = torch.optim.SGD(model.parameters(), lr=0.1, weight_decay=1e-6)

# Train the model.
for epoch in range(10):
    for (view0, view1), targets, filenames in dataloader:
        z0 = model(view0)
        z1 = model(view1)
        loss = criterion(z0, z1)
        loss.backward()
        optimizer.step()
        optimizer.zero_grad()
        print(f"loss: {loss.item():.5f}")

You can easily use another model like SimSiam by swapping the model and the loss function.

# PyTorch module for the SimSiam model.
class SimSiam(torch.nn.Module):
    def __init__(self, backbone):
        super().__init__()
        self.backbone = backbone
        self.projection_head = heads.SimSiamProjectionHead(512, 512, 128)
        self.prediction_head = heads.SimSiamPredictionHead(128, 64, 128)

    def forward(self, x):
        features = self.backbone(x).flatten(start_dim=1)
        z = self.projection_head(features)
        p = self.prediction_head(z)
        z = z.detach()
        return z, p


model = SimSiam(backbone)

# Use the SimSiam loss function.
criterion = loss.NegativeCosineSimilarity()

You can find a more complete example for SimSiam here.

Use PyTorch Lightning to train the model:

from pytorch_lightning import LightningModule, Trainer

class SimCLR(LightningModule):
    def __init__(self):
        super().__init__()
        resnet = torchvision.models.resnet18()
        resnet.fc = torch.nn.Identity()
        self.backbone = resnet
        self.projection_head = heads.SimCLRProjectionHead(512, 512, 128)
        self.criterion = loss.NTXentLoss()

    def forward(self, x):
        features = self.backbone(x).flatten(start_dim=1)
        z = self.projection_head(features)
        return z

    def training_step(self, batch, batch_index):
        (view0, view1), _, _ = batch
        z0 = self.forward(view0)
        z1 = self.forward(view1)
        loss = self.criterion(z0, z1)
        return loss

    def configure_optimizers(self):
        optim = torch.optim.SGD(self.parameters(), lr=0.06)
        return optim


model = SimCLR()
trainer = Trainer(max_epochs=10, devices=1, accelerator="gpu")
trainer.fit(model, dataloader)

See our docs for a full PyTorch Lightning example.

Or train the model on 4 GPUs:

# Use distributed version of loss functions.
criterion = loss.NTXentLoss(gather_distributed=True)

trainer = Trainer(
    max_epochs=10,
    devices=4,
    accelerator="gpu",
    strategy="ddp",
    sync_batchnorm=True,
    use_distributed_sampler=True,  # or replace_sampler_ddp=True for PyTorch Lightning <2.0
)
trainer.fit(model, dataloader)

We provide multi-GPU training examples with distributed gather and synchronized BatchNorm. Have a look at our docs regarding distributed training.

Benchmarks

Implemented models and their performance on various datasets. Hyperparameters are not tuned for maximum accuracy. For detailed results and more information about the benchmarks click here.

ImageNet1k

ImageNet1k benchmarks

Note: Evaluation settings are based on these papers:

• Linear & Finetune:
  • ResNet: SimCLR
  • ViT: MAE
• KNN: InstDisc

See the benchmarking scripts for details.

Model	Backbone	Batch Size	Epochs	Linear Top1	Finetune Top1	kNN Top1	Tensorboard	Checkpoint
BarlowTwins	Res50	256	100	62.9	72.6	45.6	link	link
BYOL	Res50	256	100	62.5	74.5	46.0	link	link
DINO	Res50	128	100	68.2	72.5	49.9	link	link
DINO	ViT-S/16	128	100	73.3	79.8	67.5	link	link
iBOT	ViT-S/16	128	100	72.2	78.3	65.4	link	link
LeJEPA**	ViT-S/16	512	100	64.0	78.7	47.1	link	link
MAE	ViT-B/16	256	100	46.0	81.3	11.2	link	link
MoCoV2	Res50	256	100	61.5	74.3	41.8	link	link
SimCLR*	Res50	256	100	63.2	73.9	44.8	link	link
SimCLR* + DCL	Res50	256	100	65.1	73.5	49.6	link	link
SimCLR* + DCLW	Res50	256	100	64.5	73.2	48.5	link	link
SwAV	Res50	256	100	67.2	75.4	49.5	link	link
TiCo	Res50	256	100	49.7	72.7	26.6	link	link
VICReg	Res50	256	100	63.0	73.7	46.3	link	link

*We use square root learning rate scaling instead of linear scaling as it yields better results for smaller batch sizes. See Appendix B.1 in the SimCLR paper.

**The LeJEPA numbers were published in #1935 (a08c5c35) and predate the loss fix in #2023, which adds the global views to SIGReg and the invariance term. They will be updated after a re-run.

ImageNet100

ImageNet100 benchmarks detailed results

Imagenette

Imagenette benchmarks detailed results

CIFAR-10

CIFAR-10 benchmarks detailed results

Terminology

Below you can see a schematic overview of the different concepts in the package. The terms in bold are explained in more detail in our documentation.

Next Steps

Head to the documentation and see the things you can achieve with Lightly!

Development

We use uv to manage dependencies. To set up a development environment (for example to contribute to the framework) you can use the following command:

make install-dev

This installs the package with all extras and development dependencies, and sets up the pre-commit hooks. You do not need to activate the virtual environment manually.

For more information about how to contribute have a look here.

Running Tests

Unit tests are within the tests directory and we recommend running them using pytest. There are two test configurations available. By default, only a subset will be run:

make test-fast

To run all tests (including the slow ones) you can use the following command:

make test

To test a specific file or directory use:

uv run --frozen pytest <path to file or directory>

Code Formatting

To format code with ruff run:

make format

Further Reading

Self-Supervised Learning:

• Have a look at our #papers channel on discord for the newest self-supervised learning papers.
• A Cookbook of Self-Supervised Learning, 2023
• Masked Autoencoders Are Scalable Vision Learners, 2021
• Emerging Properties in Self-Supervised Vision Transformers, 2021
• Unsupervised Learning of Visual Features by Contrasting Cluster Assignments, 2021
• What Should Not Be Contrastive in Contrastive Learning, 2020
• A Simple Framework for Contrastive Learning of Visual Representations, 2020
• Momentum Contrast for Unsupervised Visual Representation Learning, 2020

FAQ

• Why should I care about self-supervised learning? Aren't pre-trained models from ImageNet much better for transfer learning?

  • Self-supervised learning has become increasingly popular among scientists over the last years because the learned representations perform extraordinarily well on downstream tasks. This means that they capture the important information in an image better than other types of pre-trained models. By training a self-supervised model on your dataset, you can make sure that the representations have all the necessary information about your images.

• How can I contribute?

  • Create an issue if you encounter bugs or have ideas for features we should implement. You can also add your own code by forking this repository and creating a PR. More details about how to contribute with code is in our contribution guide.

• Is this framework for free?

  • Yes, this framework is completely free to use and we provide the source code. We believe that we need to make training deep learning models more data efficient to achieve widespread adoption. One step to achieve this goal is by leveraging self-supervised learning. The company behind Lightly is committed to keep this framework open-source.

• If this framework is free, how is the company behind Lightly making money?

  • Training self-supervised models is only one part of our solution. The company behind Lightly focuses on processing and analyzing embeddings created by self-supervised models. By building, what we call a self-supervised active learning loop we help companies understand and work with their data more efficiently. As the Lightly Solution is a freemium product, you can try it out for free. However, we will charge for some features.
  • In any case this framework will always be free to use, even for commercial purposes.

Lightly in Research

• DINOv2-3D: Self-Supervised 3D Vision Transformer Pretraining
• Joint-Embedding vs Reconstruction: Provable Benefits of Latent Space Prediction for Self-Supervised Learning, 2025
• Reverse Engineering Self-Supervised Learning, 2023
• Learning Visual Representations via Language-Guided Sampling, 2023
• Self-Supervised Learning Methods for Label-Efficient Dental Caries Classification, 2022
• DPCL: Contrastive representation learning with differential privacy, 2022
• Decoupled Contrastive Learning, 2021
• solo-learn: A Library of Self-supervised Methods for Visual Representation Learning, 2021
• EdgeCrafter: Compact ViTs for Edge Dense Prediction via Task-Specialized Distillation
• Unlabeled to Accurate: Self-Supervised Learning for Land Use Classification in Sentinel-2 Imagery
• Self-supervised Representation Learning for AI-Based Musculoskeletal Radiograph Registry Construction
• Real-Time Object Detection Meets DINOv3
• Vision Foundry: A System for Training Foundational Vision AI Models, 2025
• Self-supervised learning outperforms supervised learning for crop classification by annotating only 5% of images, 2025
• Self-Supervised Learning Powered by Synthetic Data From Diffusion Models: Application to X-Ray Images, 2025
• Wildlife Target Re-Identification Using Self-supervised Learning in Non-Urban Settings, 2025
• SSL-MAE: Adaptive Semisupervised Learning Framework for Multilabel Classification of Remote Sensing Images Using Masked Autoencoders, 2025

Company behind this Open Source Framework

Lightly is a spin-off from ETH Zurich that helps companies build efficient active learning pipelines to select the most relevant data for their models.

You can find out more about the company and it's services by following the links below:

• Homepage
• LightlyTrain
• Web-App
• LightlyStudio
• Lightly's AwesomeSSL (collection of SSL papers)

Back to top🚀