标签：多模态

日本語LLMまとめ - Overview of Japanese LLMs

Curated visual catalog of 200+ vision-language model (VLM/MLLM) architectures: papers, diagrams, training recipes, datasets, and a release timeline for multimodal AI agents.

Offline inference engine for art, real-time voice conversations, LLM powered chatbots and automated workflows

为纯文本模型"看图“设计更好的视觉工具箱和技能，支持多图理解，图片问答，前端UI还原、GUI 自动化等，并可选无缝接入多个主流agent，直接识别粘贴图片｜ A vision toolkit and skill designed for text-only llms — image Q&A, long-screenshot OCR, frontend UI restoration, and GUI automation, with optional seamless integration for Codex, Claude Code, Pi, Oh My Pi, and OpenCode

Distributed training & inference for multimodal perception: shared embeddings across text, images, video, audio, code & graphs from 64 to 1024 dimensions 🖼️

Multi-AI MCP Server - Connect ChatGPT, Claude, Gemini & Perplexity to your coding tools without any API

A foundation model that generates synchronized video and audio in a single model

Resource, examples & tutorials for multimodal AI, RAG and agents using vector search and LLMs

NEO Series: Native Vision-Language Models from First Principles

RAG Time: A 5-week Learning Journey to Mastering RAG

About This repository is a curated collection of the most exciting and influential CVPR 2025 papers. 🔥 [Paper + Code + Demo]

Halo is an open-source framework built by White Circle for training large language and multimodal models

An open-weight 11B model series for long-form and real-time video understanding

[TMLR 2025🔥] A survey for the autoregressive models in vision.

Paddle Multimodal Integration and eXploration, supporting mainstream multi-modal tasks, including end-to-end large-scale multi-modal pretrain models and diffusion model toolbox. Equipped with high performance and flexibility.

Turn documents into AI-ready Markdown with visual understanding

This repo contains the code for "VLM2Vec / MMEB" [ICLR 2025], "VLM2Vec-V2 / MMEB-V2" [TMLR 2026], and "MMEB-V3" [COLM 2026]

Project Page For "Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement"

RAI is a vendor agnostic agentic framework for Physical AI robotics, utilizing ROS 2 tools to perform complex actions, defined scenarios, free interface execution, log summaries, voice interaction and more.

An MCP Multimodal AI Agent with eyes and ears!

LLaVA-Mini is a unified large multimodal model (LMM) that can support the understanding of images, high-resolution images, and videos in an efficient manner.

Flame is an open-source multimodal AI system designed to translate UI design mockups into high-quality React code. It leverages vision-language modeling, automated data synthesis, and structured training workflows to bridge the gap between design and front-end development.

A hub for various industry-specific schemas to be used with VLMs.

Awesome Multimodal Modeling [Covers MLLM, UMM, and NMM]

Scene-aware storyboard for DeepSeek Harness: 9-16 numbered cells with timestamps + transcript + an audio envelope, packed into ONE image (384 tokens). Two tools: video_storyboard and video_zoom.

A CLI tool/python module for generating images from text using guided diffusion and CLIP from OpenAI.

[CVPR 2025 Highlight] Official code for "Olympus: A Universal Task Router for Computer Vision Tasks"

A curated archive of breakthroughs in Agents, Architecture, Training, RAG, and On-Device AI.

[ICLR 2026] RecA: visual understanding help generation through self-supervised learning

OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks

The Official Repo for "Quick Start Guide to Large Language Models"

🏭 Mega Scale Multimodal DataPipeline for SOTA Foundation Models

A minimal codebase for finetuning large multimodal models, supporting llava-1.5/1.6, llava-interleave, llava-next-video, llava-onevision, llama-3.2-vision, qwen-vl, qwen2-vl, phi3-v etc.

[ICLR 2026] VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

Reverse-engineered Doubao (豆包) API → OpenAI-compatible REST service. Free multimodal chat, image/video/music generation, and file hosting for AI agents.

Seamlessly integrate state-of-the-art transformer models into robotics stacks

Aesthetic code generation with decoupled cross-modal rewards: turning prompts into editable, verifiable HTML/CSS infographics, with image generators as aesthetic guidance.

Phi-3, -3.5, and -4 for Mac: Locally-run Vision and Language Models for Apple Silicon

Archived snapshot of Thinking-with-Visual-Primitives

[EMNLP2024 Demo], [ICASSP 2025], [ICASSP 2026] A user-friendly library for reproducible video moment retrieval and highlight detection. It also supports audio moment retrieval.

100+ curated Seedance 2.5 prompts with real video previews, plus an installable Agent Skill that optimizes prompts, creates storyboards, and generates videos via Seedance models.

🚀🚀 A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.

Production-ready ComfyUI custom nodes for 1,400+ fal.ai models, auto-updated image, video, audio, LLM and VLM APIs with native media, caching and cost controls.

[ICML 2026] GRACE: deployable INT4 VLM via quantization-aware distillation.