model2vec

Quickstart • Main Features • Model List • Results

Model2Vec is a technique to turn any sentence transformer into a small, fast static embedding model. Model2Vec reduces model size by a factor up to 50 and makes models up to 500 times faster, with a small drop in performance. Our best model is the most performant static embedding model in the world. See our results, read our docs, or dive in to see how it works.

Quickstart

Install the lightweight base package with:

pip install model2vec

You can start using Model2Vec by loading one of our flagship models from the HuggingFace hub. These models are pre-trained and ready to use. The following code snippet shows how to load a model and make embeddings, which you can use for any task, such as text classification, retrieval, clustering, or building a RAG system:

from model2vec import StaticModel

# Load a model from the HuggingFace hub (in this case the potion-base-32M model)
model = StaticModel.from_pretrained("minishlab/potion-base-32M")

# Make embeddings
embeddings = model.encode(["It's dangerous to go alone!", "It's a secret to everybody."])

# Make sequences of token embeddings
token_embeddings = model.encode_as_sequence(["It's dangerous to go alone!", "It's a secret to everybody."])

For advanced usage, see our inference docs. Instead of using one of our models, you can also distill your own Model2Vec model from a Sentence Transformer model. First, install the distill extra with:

pip install model2vec[distill]

Then, you can distill a model in ~30 seconds on a CPU with the following code snippet:

from model2vec.distill import distill

# Distill a Sentence Transformer model, in this case the BAAI/bge-base-en-v1.5 model
m2v_model = distill(model_name="BAAI/bge-base-en-v1.5")

# Save the model
m2v_model.save_pretrained("m2v_model")

For advanced usage, see our distillation docs, which includes some distillation best practices. After distillation, you can also fine-tune the distilled model, or a pre-trained model, for classification, similarity, regression, or pair similarity (e.g. query-document retrieval). First, make sure you install the train extras with:

pip install model2vec[train]

Then, you can fine-tune a model as follows:

from datasets import load_dataset
from model2vec.train import StaticModelForClassification

# Initialize a classifier from a pre-trained model
classifier = StaticModelForClassification.from_pretrained(path="minishlab/potion-base-32M")

# Load a dataset. Note: both single and multi-label classification datasets are supported
ds = load_dataset("setfit/subj")

# Train the classifier on text (X) and labels (y)
classifier.fit(ds["train"]["text"], ds["train"]["label"])

# Evaluate the classifier
classification_report = classifier.evaluate(ds["test"]["text"], ds["test"]["label"])

For advanced usage, see our training docs.

Main Features

• State-of-the-Art Performance: Model2Vec models outperform any other static embeddings (such as GLoVe and BPEmb) by a large margin, as can be seen in our results.
• Small: Model2Vec reduces the size of a Sentence Transformer model by a factor of up to 50. Our best model is just ~30 MB on disk, and our smallest model just ~8 MB (making it the smallest model on MTEB!).
• Lightweight Dependencies: the base package's only major dependency is numpy.
• Lightning-fast Inference: up to 500 times faster on CPU than the original model.
• Fast, Dataset-free Distillation: distill your own model in 30 seconds on a CPU, without a dataset.
• Fine-tuning: fine-tune Model2Vec models for classification, similarity, regression, or pair similarity, and run them without torch.
• Integrated in many popular libraries: Model2Vec is integrated direclty into popular libraries such as Sentence Transformers and LangChain. For more information, see our integrations documentation.
• Tightly integrated with HuggingFace hub: easily share and load models from the HuggingFace hub, using the familiar from_pretrained and push_to_hub. Our own models can be found here.

What is Model2Vec?

Model2vec creates a small, fast, and powerful model that outperforms other static embedding models by a large margin on all tasks we could find, while being much faster to create than traditional static embedding models such as GloVe. Like BPEmb, it can create subword embeddings, but with much better performance. Distillation doesn't need any data, just a vocabulary and a model.

The core idea is to forward pass a vocabulary through a sentence transformer model, creating static embeddings for the indiviudal tokens. After this, there are a number of post-processing steps we do that results in our best models, as well as an optional pre-training step to further boost performance. For a more extensive deepdive, please refer to our official documentation on how Model2Vec works.

Documentation

Our official documentation can be found here. This includes in-depth documentation on inference, distillation, training, and integrations.

Model List

We provide a number of models that can be used out of the box. These models are available on the HuggingFace hub and can be loaded using the from_pretrained method. The models are listed below.

Model	Language	Sentence Transformer	Params	Task
potion-base-32M	English	bge-base-en-v1.5	32.3M	General
potion-multilingual-128M	Multilingual	bge-m3	128M	General
potion-retrieval-32M	English	bge-base-en-v1.5	32.3M	Retrieval
potion-base-8M	English	bge-base-en-v1.5	7.5M	General
potion-code-16M-v2	Code	CodeRankEmbed	16.2M	Code

Results

We have performed extensive experiments to evaluate the performance of Model2Vec models. The results are documented in the results folder. The results are presented in the following sections:

• MTEB Results
• MMTEB Results
• Retrieval Results
• Training Results
• Ablations

Acknowledgements

Thanks to Greptile for providing free access to their AI code review platform.

License

MIT

Citing

If you use Model2Vec in your research, please cite the following:

@software{minishlab2024model2vec,
  author       = {Stephan Tulkens and {van Dongen}, Thomas},
  title        = {Model2Vec: Fast State-of-the-Art Static Embeddings},
  year         = {2024},
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.17270888},
  url          = {https://github.com/MinishLab/model2vec},
  license      = {MIT}
}