InternVL

InternVL Family: Closing the Gap to Commercial Multimodal Models with Open-Source Suites —— A Pioneering Open-Source Alternative to GPT-5

[🆕 Blog] [🤔 FAQs] [🗨️ Chat Demo] [📖 Document] [🌐 API] [🚀 Quick Start]

[🔥 InternVL3.5 Report] [📜 InternVL3.0 Report] [📜 InternVL2.5 MPO] [📜 InternVL2.5 Report]

[📜 Mini-InternVL Paper] [📜 InternVL2 Blog] [📜 InternVL 1.5 Paper] [📜 InternVL 1.0 Paper]

[📖 2.0 中文解读] [📖 1.5 中文解读] [📖 1.0 中文解读]

Switch to the Chinese version (切换至中文版)

News 🚀🚀🚀

• 2025/08/30: 🔥 We open-source the training code of InternVL3_5-GPT-OSS-20B-A4B and CascadeRL, which consists of a offline RL stage and a online RL stage. The training data for these two stages (MMPR-v1.2 and MMPR-Tiny) are also open-sourced.
• 2025/08/26: 🚀 We introduce InternVL3.5, a new family of open-source multimodal models that significantly advances versatility, reasoning capability, and inference efficiency along the InternVL series. Our largest model, i.e., InternVL3.5-241B-A28B, attains state-of-the-art results among open-source MLLMs across general multimodal, reasoning, text, and agentic tasks. We also provide a 20B-A4B version (i.e., InternVL3_5-GPT-OSS-20B-A4B), which is built up on GPT-OSS-20B-A4B. Notably, we provide two model formats: the GitHub format, consistent with prior releases, and the HF format, aligned with the official transformers standard.
• 2025/04/17: We open-source the data construction pipeline and training scripts of MPO and VisualPRM. Additionally, the data construction scripts for MPO and VisualPRM are also released for reference.
• 2025/04/11: We introduce InternVL3, an advanced multimodal large language model (MLLM) series that demonstrates superior overall performance. InternVL3-78B achieves SoTA performance in both perception and reasoning performance among open-source MLLMs. The key designs of InternVL3-78B include Variable Visual Position Encoding, Native Multimodal Pre-Training, Mixed Preference Optimization, and Multimodal Test-Time Scaling.
• 2025/03/13: We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM) with 8B parameters, which improves the overall reasoning performance of InternVL2.5-8B and InternVL2.5-78B by 8.4 and 5.9 points, respectively. The training data for this model, termed VisualPRM400K, is also open-sourced. Please refer to our paper and project page for more details.
• 2024/12/20: We release the InternVL2.5-MPO, which is finetuned with Mixed Preference Optimization on MMPR-v1.1. The resulting models outperform their counterparts without MPO by an average of 2 points across all model scales on the OpenCompass leaderboard. These models are available at HF link.
• 2024/12/17: InternVL2/2.5 is supported in PaddleMIX by Paddle Team.
• 2024/12/05: We release the InternVL2.5, an advanced multimodal large language model (MLLM) series with parameter coverage ranging from 1B to 78B. InternVL2_5-78B is the first open-source MLLMs to achieve over 70% on the MMMU benchmark, matching the performance of leading closed-source commercial models like GPT-4o. These models are available at HF link.
• 2024/11/14: We introduce MMPR, a high-quality, large-scale multimodal reasoning preference dataset, and MPO, an effective preference optimization algorithm. The resulting model, InternVL2-8B-MPO, achieves an accuracy of 67.0 on MathVista. Please refer to our paper, project page and document for more details.

• 2024/10/21: We release the Mini-InternVL series. These models achieve impressive performance with minimal size: the 4B model achieves 90% of the performance with just 5% of the model size. For more details, please check our project page and document.
• 2024/08/01: The Chartmimic team evaluated the InternVL2 series models on their benchmark. The InternVL2-26B and 76B models achieved the top two performances among open-source models, with the InternVL2 76B model surpassing GeminiProVision and exhibiting comparable results to Claude-3-opus.
• 2024/08/01: InternVL2-Pro achieved the SOTA performance among open-source models on the CharXiv dataset, surpassing many closed-source models such as GPT-4V, Gemini 1.5 Flash, and Claude 3 Sonnet.
• 2024/07/24: The MLVU team evaluated InternVL-1.5 on their benchmark. The average performance on the multiple-choice task was 50.4%, while the performance on the generative tasks was 4.02. The performance on the multiple-choice task ranked #1 among all open-source MLLMs.
• 2024/07/04: We release the InternVL2 series. InternVL2-Pro achieved a 62.0% accuracy on the MMMU benchmark, matching the performance of leading closed-source commercial models like GPT-4o.
• 2024/07/18: InternVL2-40B achieved SOTA performance among open-source models on the Video-MME dataset, scoring 61.2 when inputting 16 frames and 64.4 when inputting 32 frames. It significantly outperforms other open-source models and is the closest open-source model to GPT-4o mini.
• 2024/07/18: InternVL2-Pro achieved the SOTA performance on the DocVQA and InfoVQA benchmarks.
• 2024/06/19: We propose Needle In A Multimodal Haystack (MM-NIAH), the first benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.
• 2024/05/30: We release ShareGPT-4o, a large-scale dataset that we plan to open-source with 200K images, 10K videos, and 10K audios with detailed descriptions.
• 2024/05/28: Thanks to the lmdeploy team for providing AWQ quantization support. The 4-bit model is available at OpenGVLab/InternVL-Chat-V1-5-AWQ.
• 2024/05/13: InternVL 1.0 can now be used as the text encoder for diffusion models to support multilingual generation natively in over 110 languages worldwide. See MuLan for more details.
• 2024/04/18: InternVL-Chat-V1-5 has been released at HF link, approaching the performance of GPT-4V and Gemini Pro on various benchmarks like MMMU, DocVQA, ChartQA, MathVista, etc.
• 2024/02/27: InternVL is accepted by CVPR 2024 (Oral)! 🎉
• 2024/02/21: InternVL-Chat-V1-2-Plus achieved SOTA performance on MathVista (59.9), MMBench (83.8), and MMVP (58.7). See our blog for more details.
• 2024/02/12: InternVL-Chat-V1-2 has been released. It achieves 51.6 on MMMU val and 82.3 on MMBench test. For more details, please refer to our blog and SFT data. The model is now available on HuggingFace, and both training / evaluation data and scripts are open-sourced.
• 2024/01/24: InternVL-Chat-V1-1 is released, it supports Chinese and has stronger OCR capability, see here.
• 2024/01/16: We release our customized mmcv/mmsegmentation/mmdetection code, integrated with DeepSpeed, which can be used for training large-scale detection and segmentation models.

Documents

🌟 Get Started

• Installation: 🌱 Installation Guide | 📄 requirements.txt
• Chat Data Format: 📝 Meta File | ✏️ Text | 🖼️ Single-Image | 🖼️🖼️ Multi-Image | 🎥 Video
• Local Chat Demo: 🤖 Streamlit Demo
• InternVL-Chat API: 🌐 InternVL2.5 API
• Tutorials: 🚀 Enhancing InternVL2 on COCO Caption Using LoRA Fine-Tuning

🏆 InternVL Family

• InternVL 3.0: 📖 Intro | ⚡ Quick Start | ✨ Finetune | 📊 Evaluate | 📦 Deploy | 🎯 MPO
• InternVL 2.5: 📖 Intro | ⚡ Quick Start | ✨ Finetune | 📊 Evaluate | 📦 Deploy | 🎯 MPO
• InternVL 2.0: 📖 Intro | ⚡ Quick Start | ✨ Finetune | 📊 Evaluate | 📦 Deploy | 🎯 MPO
• InternVL 1.5: 📖 Intro | ⚡ Quick Start | ✨ Finetune | 📊 Evaluate | 📦 Deploy
• InternVL 1.2: 📖 Intro | ⚡ Quick Start | ✨ Finetune | 📊 Evaluate
• InternVL 1.1: 📖 Intro | ⚡ Quick Start | 📊 Evaluation
• InternVL 1.0: 🖼️ Classification | 📊 CLIP-Benchmark | 🎨 Segmentation | 💬 Chat-LLaVA | ✨ InternVL-G

Model Zoo

Multimodal Large Language Model (InternVL 3.5)

To maintain consistency with earlier generations, we provide two model formats: the GitHub format, consistent with prior releases, and the HF format, aligned with the official Transformers standard.

If you want to convert the checkpoint between these two formats, please refer to the scripts about custom2hf and hf2custom.

Github Format

Model	#Vision Param	#Language Param	#Total Param	HF Link	ModelScope Link
InternVL3.5-1B	0.3B	0.8B	1.1B	🤗 link	🤖 link
InternVL3.5-2B	0.3B	2.0B	2.3B	🤗 link	🤖 link
InternVL3.5-4B	0.3B	4.4B	4.7B	🤗 link	🤖 link
InternVL3.5-8B	0.3B	8.2B	8.5B	🤗 link	🤖 link
InternVL3.5-14B	0.3B	14.8B	15.1B	🤗 link	🤖 link
InternVL3.5-38B	5.5B	32.8B	38.4B	🤗 link	🤖 link
InternVL3.5-20B-A4B	0.3B	20.9B	21.2B-A4B	🤗 link	🤖 link
InternVL3.5-30B-A3B	0.3B	30.5B	30.8B-A3B	🤗 link	🤖 link
InternVL3.5-241B-A28B	5.5B	235.1B	240.7B-A28B	🤗 link	🤖 link

HuggingFace Format

Model	#Vision Param	#Language Param	#Total Param	HF Link	ModelScope Link
InternVL3.5-1B-HF	0.3B	0.8B	1.1B	🤗 link	🤖 link
InternVL3.5-2B-HF	0.3B	2.0B	2.3B	🤗 link	🤖 link
InternVL3.5-4B-HF	0.3B	4.4B	4.7B	🤗 link	🤖 link
InternVL3.5-8B-HF	0.3B	8.2B	8.5B	🤗 link	🤖 link
InternVL3.5-14B-HF	0.3B	14.8B	15.1B	🤗 link	🤖 link
InternVL3.5-38B-HF	5.5B	32.8B	38.4B	🤗 link	🤖 link
InternVL3.5-20B-A4B-HF	0.3B	20.9B	21.2B-A4B	🤗 link	🤖 link
InternVL3.5-30B-A3B-HF	0.3B	30.5B	30.8B-A3B	🤗 link	🤖 link
InternVL3.5-241B-A28B-HF	5.5B	235.1B	240.7B-A28B	🤗 link	🤖 link

Multimodal Large Language Model (InternVL 3.0)

Multimodal Large Language Model (InternVL 2.5)

Multimodal Large Language Model (InternVL 2.0)

Multimodal Large Language Model (InternVL 1.0-1.5)

CLIP-like Model (InternVL 1.0-2.5)

Vision-Language Foundation Model (InternVL 1.0)

TODO List

• Release training / evaluation code for InternVL2.5 series
• Support liger kernels to save GPU memory
• Release the code, model, and data of MPO
• Support multimodal packed dataset
• Support vLLM and Ollama
• Support video and PDF input in online demo
• Release InternVL2 with VisionLLMv2 integration
• Rebuild documents using readthedocs
• Support fine-tuning different LLMs with LoRA
• Release requirements.txt for InternVL2
• Release training / evaluation code for InternVL2 series
• Release Streamlit web UI for InternVL1.5 and InternVL2

What can InternVL do?

• Linear-Probe Image Classification [see details]

  ViT-22B uses the private JFT-3B dataset.

  method	#param	IN-1K	IN-ReaL	IN-V2	IN-A	IN-R	IN-Sketch
  OpenCLIP-G	1.8B	86.2	89.4	77.2	63.8	87.8	66.4
  DINOv2-g	1.1B	86.5	89.6	78.4	75.9	78.8	62.5
  EVA-01-CLIP-g	1.1B	86.5	89.3	77.4	70.5	87.7	63.1
  MAWS-ViT-6.5B	6.5B	87.8	-	-	-	-	-
  ViT-22B*	21.7B	89.5	90.9	83.2	83.8	87.4	-
  InternViT-6B (ours)	5.9B	88.2	90.4	79.9	77.5	89.8	69.1

• Semantic Segmentation [see details]

  method	decoder	#param (train/total)	crop size	mIoU
  OpenCLIP-G (frozen)	Linear	0.3M / 1.8B	512	39.3
  ViT-22B (frozen)	Linear	0.9M / 21.7B	504	34.6
  InternViT-6B (frozen)	Linear	0.5M / 5.9B	504	47.2 (+12.6)
  ViT-22B (frozen)	UperNet	0.8B / 22.5B	504	52.7
  InternViT-6B (frozen)	UperNet	0.4B / 6.3B	504	54.9 (+2.2)
  ViT-22B	UperNet	22.5B / 22.5B	504	55.3
  InternViT-6B	UperNet	6.3B / 6.3B	504	58.9 (+3.6)

• Zero-Shot Image Classification [see details]

  method	IN-1K	IN-A	IN-R	IN-V2	IN-Sketch	ObjectNet
  OpenCLIP-G	80.1	69.3	92.1	73.6	68.9	73.0
  EVA-02-CLIP-E+	82.0	82.1	94.5	75.7	71.6	79.6
  ViT-22B*	85.9	90.1	96.0	80.9	-	87.6
  InternVL-C (ours)	83.2	83.8	95.5	77.3	73.9	80.6

• Multilingual Zero-Shot Image Classification [see details]

  EN: English, ZH: Chinese, JP: Japanese, Ar: Arabic, IT: Italian

  method	IN-1K (EN)	IN-1K (ZH)	IN-1K (JP)	IN-1K (AR)	IN-1K (IT)
  Taiyi-CLIP-ViT-H	-	54.4	-	-	-
  WuKong-ViT-L-G	-	57.5	-	-	-
  CN-CLIP-ViT-H	-	59.6	-	-	-
  AltCLIP-ViT-L	74.5	59.6	-	-	-
  EVA-02-CLIP-E+	82.0	-	-	-	41.2
  OpenCLIP-XLM-R-H	77.0	55.7	53.1	37.0	56.8
  InternVL-C (ours)	83.2	64.5	61.5	44.9	65.7

• Zero-Shot Video Classification

  method	#frame	K400	K600	K700
  OpenCLIP-G	1	65.9	66.1	59.2
  EVA-02-CLIP-E+	1	69.8	69.3	63.4
  InternVL-C (ours)	1	71.0	71.3	65.7
  ViCLIP	8	75.7	73.5	66.4
  InternVL-C (ours)	8	79.4	78.8	71.5

• English Zero-Shot Image-Text Retrieval [see details]

• Chinese Zero-Shot Image-Text Retrieval [see details]

• Multilingual Zero-Shot Image-Text Retrieval on XTD [see details]

  method	EN	ES	FR	ZH	IT	KO	RU	JP	average
  AltCLIP	95.4	94.1	92.9	95.1	94.2	94.4	91.8	91.7	93.7
  OpenCLIP-XLM-R-H	97.3	96.1	94.5	94.7	96.0	90.2	93.9	94.0	94.6
  InternVL-C (ours)	97.3	95.7	95.1	95.6	96.0	92.2	93.3	95.5	95.1
  InternVL-G (ours)	98.6	97.7	96.5	96.7	96.9	95.1	94.8	96.1	96.6

Quick Start with HuggingFace

import torch
from PIL import Image
from transformers import AutoModel, CLIPImageProcessor

model = AutoModel.from_pretrained(
    'OpenGVLab/InternViT-6B-448px-V2_5',
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True).cuda().eval()

image = Image.open('./examples/image1.jpg').convert('RGB')

image_processor = CLIPImageProcessor.from_pretrained('OpenGVLab/InternViT-6B-448px-V1-5')

pixel_values = image_processor(images=image, return_tensors='pt').pixel_values
pixel_values = pixel_values.to(torch.bfloat16).cuda()

outputs = model(pixel_values)

import torch
from PIL import Image
from transformers import AutoModel, CLIPImageProcessor
from transformers import AutoTokenizer


model = AutoModel.from_pretrained(
    'OpenGVLab/InternVL-14B-224px',
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True).cuda().eval()

image_processor = CLIPImageProcessor.from_pretrained('OpenGVLab/InternVL-14B-224px')

tokenizer = AutoTokenizer.from_pretrained(
    'OpenGVLab/InternVL-14B-224px', use_fast=False, add_eos_token=True)
tokenizer.pad_token_id = 0  # set pad_token_id to 0

images = [
    Image.open('./examples/image1.jpg').convert('RGB'),
    Image.open('./examples/image2.jpg').convert('RGB'),
    Image.open('./examples/image3.jpg').convert('RGB')
]
prefix = 'summarize:'
texts = [
    prefix + 'a photo of a red panda',  # English
    prefix + '一张熊猫的照片',  # Chinese
    prefix + '二匹の猫の写真'  # Japanese
]

pixel_values = image_processor(images=images, return_tensors='pt').pixel_values
pixel_values = pixel_values.to(torch.bfloat16).cuda()
input_ids = tokenizer(texts, return_tensors='pt', max_length=80,
                      truncation=True, padding='max_length').input_ids.cuda()

# InternVL-C
logits_per_image, logits_per_text = model(
    image=pixel_values, text=input_ids, mode='InternVL-C')
probs = logits_per_image.softmax(dim=-1)
# tensor([[9.9609e-01, 5.2185e-03, 6.0070e-08],
#         [2.2949e-02, 9.7656e-01, 5.9903e-06],
#         [3.2932e-06, 7.4863e-05, 1.0000e+00]], device='cuda:0',
#        dtype=torch.bfloat16, grad_fn=<SoftmaxBackward0>)

# InternVL-G
logits_per_image, logits_per_text = model(
    image=pixel_values, text=input_ids, mode='InternVL-G')
probs = logits_per_image.softmax(dim=-1)
# tensor([[9.9609e-01, 3.1738e-03, 3.6322e-08],
#         [8.6060e-03, 9.9219e-01, 2.8759e-06],
#         [1.7583e-06, 3.1233e-05, 1.0000e+00]], device='cuda:0',
#        dtype=torch.bfloat16, grad_fn=<SoftmaxBackward0>)

# please set add_eos_token to False for generation
tokenizer.add_eos_token = False
image = Image.open('./examples/image1.jpg').convert('RGB')
pixel_values = image_processor(images=image, return_tensors='pt').pixel_values
pixel_values = pixel_values.to(torch.bfloat16).cuda()

tokenized = tokenizer("English caption:", return_tensors='pt')
pred = model.generate(
    pixel_values=pixel_values,
    input_ids=tokenized.input_ids.cuda(),
    attention_mask=tokenized.attention_mask.cuda(),
    num_beams=5,
    min_new_tokens=8,
)
caption = tokenizer.decode(pred[0].cpu(), skip_special_tokens=True).strip()
# English caption: a red panda sitting on top of a wooden platform

Here, we take the smaller OpenGVLab/InternVL2_5-8B as an example:

import numpy as np
import torch
import torchvision.transforms as T
from decord import VideoReader, cpu
from PIL import Image
from torchvision.transforms.functional import InterpolationMode
from transformers import AutoModel, AutoTokenizer

IMAGENET_MEAN = (0.485, 0.456, 0.406)
IMAGENET_STD = (0.229, 0.224, 0.225)

def build_transform(input_size):
    MEAN, STD = IMAGENET_MEAN, IMAGENET_STD
    transform = T.Compose([
        T.Lambda(lambda img: img.convert('RGB') if img.mode != 'RGB' else img),
        T.Resize((input_size, input_size), interpolation=InterpolationMode.BICUBIC),
        T.ToTensor(),
        T.Normalize(mean=MEAN, std=STD)
    ])
    return transform

def find_closest_aspect_ratio(aspect_ratio, target_ratios, width, height, image_size):
    best_ratio_diff = float('inf')
    best_ratio = (1, 1)
    area = width * height
    for ratio in target_ratios:
        target_aspect_ratio = ratio[0] / ratio[1]
        ratio_diff = abs(aspect_ratio - target_aspect_ratio)
        if ratio_diff < best_ratio_diff:
            best_ratio_diff = ratio_diff
            best_ratio = ratio
        elif ratio_diff == best_ratio_diff:
            if area > 0.5 * image_size * image_size * ratio[0] * ratio[1]:
                best_ratio = ratio
    return best_ratio

def dynamic_preprocess(image, min_num=1, max_num=12, image_size=448, use_thumbnail=False):
    orig_width, orig_height = image.size
    aspect_ratio = orig_width / orig_height

    # calculate the existing image aspect ratio
    target_ratios = set(
        (i, j) for n in range(min_num, max_num + 1) for i in range(1, n + 1) for j in range(1, n + 1) if
        i * j <= max_num and i * j >= min_num)
    target_ratios = sorted(target_ratios, key=lambda x: x[0] * x[1])

    # find the closest aspect ratio to the target
    target_aspect_ratio = find_closest_aspect_ratio(
        aspect_ratio, target_ratios, orig_width, orig_height, image_size)

    # calculate the target width and height
    target_width = image_size * target_aspect_ratio[0]
    target_height = image_size * target_aspect_ratio[1]
    blocks = target_aspect_ratio[0] * target_aspect_ratio[1]

    # resize the image
    resized_img = image.resize((target_width, target_height))
    processed_images = []
    for i in range(blocks):
        box = (
            (i % (target_width // image_size)) * image_size,
            (i // (target_width // image_size)) * image_size,
            ((i % (target_width // image_size)) + 1) * image_size,
            ((i // (target_width // image_size)) + 1) * image_size
        )
        # split the image
        split_img = resized_img.crop(box)
        processed_images.append(split_img)
    assert len(processed_images) == blocks
    if use_thumbnail and len(processed_images) != 1:
        thumbnail_img = image.resize((image_size, image_size))
        processed_images.append(thumbnail_img)
    return processed_images

def load_image(image_file, input_size=448, max_num=12):
    image = Image.open(image_file).convert('RGB')
    transform = build_transform(input_size=input_size)
    images = dynamic_preprocess(image, image_size=input_size, use_thumbnail=True, max_num=max_num)
    pixel_values = [transform(image) for image in images]
    pixel_values = torch.stack(pixel_values)
    return pixel_values

# If you have an 80G A100 GPU, you can put the entire model on a single GPU.
# Otherwise, you need to load a model using multiple GPUs, please refer to the `Multiple GPUs` section.
path = 'OpenGVLab/InternVL2_5-8B'
model = AutoModel.from_pretrained(
    path,
    torch_dtype=torch.bfloat16,
    low_cpu_mem_usage=True,
    trust_remote_code=True).eval().cuda()
tokenizer = AutoTokenizer.from_pretrained(path, trust_remote_code=True, use_fast=False)

# set the max number of tiles in `max_num`
pixel_values = load_image('./examples/image1.jpg', max_num=12).to(torch.bfloat16).cuda()
generation_config = dict(max_new_tokens=1024, do_sample=False)

# pure-text conversation (纯文本对话)
question = 'Hello, who are you?'
response, history = model.chat(tokenizer, None, question, generation_config, history=None, return_history=True)
print(f'User: {question}\nAssistant: {response}')

question =