标签：多模态

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

《深入理解 AI Agent：设计原理与工程实践》（李博杰 著）开源主仓库：全书正文、编译版 PDF 与按章配套代码

The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra

[NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

YC (S26) | Open Computer History | Continuously record your company computer work, map your workflows, help you find work worth automating, and power your agents' context

Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.8, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM5.3, Gemma4, Llava, Phi4, ...) (AAAI 2025).

🔍大模型应用开发实战一：RAG 技术全栈指南，在线阅读地址：https://datawhalechina.github.io/all-in-rag/

https://arxiv.org/abs/2606.28344. The end of web parsing. The beginning of scalable pixel-native search. link: https://pixelrag.ai/

Production ready toolkit to run AI locally

SeaTunnel is a multimodal, high-performance, distributed, massive data integration tool.

Mobile-Agent: The Powerful GUI Agent Family

Deeplake is AI Data Runtime for Agents. It provides serverless postgres with a multimodal datalake, enabling scalable retrieval and training.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

A text-to-speech (TTS), speech-to-text (STT) and speech-to-speech (STS) library built on Apple's MLX framework, providing efficient speech analysis on Apple Silicon.

A framework for efficient model inference with omni-modality models

Open-source framework for building agentic apps in JavaScript, Go, Dart, and Python, built and used in production by Google

Search your computer by what's in your files: photos, PDFs, documents, recordings, videos and code. On-device on Mac, Windows and Linux, built on Google DeepMind's EmbeddingGemma 2.

Solve Visual Understanding with Reinforced VLMs

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

A Low-Code MCP Framework for Building Complex and Innovative RAG Pipelines

The first vision plugin for DeepSeek Harness, and the vision bridge for every text-only coding agent. Paste an image, get structured JSON evidence (OCR, layout, semantics). | 全网最强 DeepSeek Harness 外挂视觉插件，为 DeepSeek、GLM 等纯文本模型外挂视觉能力，粘贴图片即得结构化 JSON 证据（OCR、版面、语义）。

Align Anything: Training All-modality Model with Feedback

The Ruby AI framework.

One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks

An open-source model family for long-form speech, dialogue synthesis, voice design, sound effects, and real-time streaming TTS

Fengshenbang-LM(封神榜大模型)是IDEA研究院认知计算与自然语言研究中心主导的大模型开源体系，成为中文AIGC和认知智能的基础设施。

[ICML'26] SimpleMem: Efficient Lifelong Memory for LLM Agents — Text & Multimodal

MTEB: State-of-the-art evaluation of embeddings across languages and modalities

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

The Context Layer for unstructured data: typed, versioned datasets over S3, GCS, Azure

Images to inference with no labeling (use foundation models to train supervised models).

streamline the fine-tuning process for multimodal models: PaliGemma 2, Florence-2, and Qwen2.5-VL

[EMNLP-2024] Build multimodal language agents for fast prototype and production

Comprehensive resources on Generative AI, including a detailed roadmap, projects, use cases, interview preparation, and coding preparation.

A curated collection of papers, benchmarks, surveys, and tools for model quantization, covering low-bit networks, LLMs, multimodal and generative models, vector and lattice quantization, and efficient deployment.

(ෆ`꒳´ෆ) A Survey on Text-to-Image Generation/Synthesis.

Let Claude (or any LLM) actually watch a video — scene-aware, deduplicated frames + transcript, from a URL or local file. Runs locally, MIT.

On-device, real-time multimodal AI with features similar to GPT-Live

[ICLR & NeurIPS 2025] Repository for Show-o series, One Single Transformer to Unify Multimodal Understanding and Generation.

An open-source implementaion for fine-tuning Qwen-VL series by Alibaba Cloud.

A collection of original, innovative ideas and algorithms towards Advanced Literate Machinery. This project is maintained by the OCR Team in the Language Technology Lab, Tongyi Lab, Alibaba Group.

Synthesizing Graphics Programs for Scientific Figures and Sketches with TikZ.

High Performance Serving Runtime For Interactive Super Intelligence System

Fast Multimodal LLM on Mobile Devices

A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

Get clean data from tricky documents, powered by vision-language models ⚡

Multimodal RL training framework for diffusion & omni models

Give Claude the ability to watch and understand videos — Claude Code plugin with frame extraction and multimodal audio analysis