xberg

Xberg

The fast, precise document-intelligence engine — for every language.

Point Xberg at anything — a PDF, a scanned image, a spreadsheet, an audio file, a URL, a whole archive, or a source tree — and get back clean text, tables, metadata, and structured data. One engine handles format detection, reading, OCR, and extraction, so you never stitch a pipeline together from a dozen libraries.

107 formats · 141 file extensions · 371 code languages · 15 language bindings · 6 output formats · OCR · transcription · embeddings

The fastest, most precise open-source document and PDF-to-Markdown engine — see the benchmarks.

Install · What you get · Capabilities · CLI · Docs

Xberg is the next iteration of Kreuzberg. Same document-intelligence engine, rebuilt and rebranded under a fresh v1 line.

---

What you get

Point Xberg at anything — a PDF, a spreadsheet, a scanned image, an audio file, a URL, an archive, a source tree — and get back clean, structured content you can use right away. One core does the format detection, reading, and extraction, so you don't assemble a pipeline yourself. Call it from Rust, Python, Node.js, Go, Java, C#, Ruby, PHP, Elixir, Dart, Swift, Zig, WASM, Kotlin, or C FFI, and run it as a library, CLI tool, REST API, or MCP server.

Capability	What you get
107 document formats	PDFs, Office, images, HTML, email, e-books, scientific publications, and structured data across 141 file extensions, with intelligent MIME detection and bounded extraction controls.
URLs & the web	Point Xberg at an http(s) URL — it fetches and extracts a single document, or crawls and follows links (Auto / Document / Crawl modes via the crawlberg engine). Requires the url-ingestion feature.
Audio & video transcription	Speech-to-text from MP3, M4A, WAV, WebM, and MP4 tracks via Whisper ONNX (tiny → large-v3). Requires the transcription feature.
Archives, traversed	List and recursively extract nested .zip, .tar, .gz, .7z — documents inside documents — guarded by zip-bomb, compression-ratio, and nesting-depth limits.
OCR on demand	Tesseract, PaddleOCR, Candle, or VLM backends — fallback chains, confidence scores, language auto-detection, extensible via plugins.
Layout & tables	ML layout models (PP-DocLayout-V3, RT-DETR) and table structure (TATR, SLANet) reconstruct reading order and cell grids for clean Markdown.
Code intelligence	Functions, classes, imports, symbols, docstrings from 371 programming languages. Syntax-aware chunking for RAG pipelines.
Embeddings & search	Local (ONNX) or provider-hosted embeddings (165 providers via liter-llm), sparse and late-interaction, cross-encoder reranking.
Enrichment	NER, keyword extraction (YAKE/RAKE), summarization, translation, redaction, page classification, QR detection, language detection, token reduction (TOON).
Structured extraction	Schema-driven JSON straight from any document via local (Ollama, LM Studio, vLLM) or hosted LLMs — no prompt engineering.
6 output formats	Plain text, Markdown, Djot, HTML, JSON tree, or Docling DocTags, plus registered custom renderers.
Runs anywhere	Library, CLI (14 commands), REST API (xberg serve), MCP server, Docker, Helm — CPU by default, no GPU required. Content-hash caching, parallel batch, per-file timeouts.

Capabilities marked requires a feature are Cargo feature flags on the core crate (url-ingestion, transcription, reranker, layout/ORT). Prebuilt language packages and the Docker image bundle the common set; a from-source build enables only what you select.

---

Installation

Language Packages

pip install xberg

For archived versions and the PyPI storage fallback:

pip install xberg --extra-index-url https://xberg-io.github.io/python-wheels/simple/

PyPI keeps all releases in the current minor and the latest stable patch of each older minor. See the permanent archive and retention policy.

See Python README for full documentation.

npm install @xberg-io/xberg

See Node.js README for full documentation.

cargo add xberg

See Rust README for full documentation.

go get github.com/xberg-io/xberg/packages/go@latest

⚠️ The repository root is not a Go module — go get github.com/xberg-io/xberg will fail. Always target the /packages/go subdirectory as shown above.

See Go README for full documentation.

Available on Maven Central as io.xberg:xberg. See Java README for the dependency snippet.

dotnet add package XbergIo.Xberg

See C# README for full documentation.

gem install xberg

See Ruby README for full documentation.

composer require xberg-io/xberg

See PHP README for full documentation.

Add {:xberg, "~> 1.0"} to your mix.exs dependencies. See Elixir README for full documentation.

npm install @xberg-io/xberg-wasm

See WebAssembly README for full documentation.

Available on Maven Central as io.xberg:xberg-android. See Kotlin README for the dependency snippet.

Add via Swift Package Manager. See Swift README for full documentation.

dart pub add xberg

See Dart README for full documentation.

Add via zig fetch. See Zig README for full documentation.

Build from source as part of this workspace. See C (FFI) README for full documentation.

CLI & Deployment

brew install xberg-io/tap/xberg

Windows users can install the same binary through Scoop:

scoop bucket add xberg https://github.com/xberg-io/scoop-bucket
scoop install xberg

14 commands: extract, batch, detect, formats, version, cache, tree-sitter, doctor, serve, mcp, api, embed, chunk, and completions.

See CLI usage guide for detailed documentation.

docker pull ghcr.io/xberg-io/xberg:latest

Run in API, CLI, or MCP modes. See Docker guide for examples.

xberg serve --host 0.0.0.0 --port 8000

One POST endpoint handles all formats. Returns JSON or Markdown. Stream large files. See API server guide.

xberg mcp --transport stdio

9 tools (extract, extract_batch, detect_mime_type, cache_stats, list_formats, cache_clear, get_version, cache_manifest, cache_warm). 3 prompts (extract_document, extract_with_ocr, semantic_search). 4 resources (formats, models, OCR languages, embedding presets).

Add to Claude Desktop or Cursor:

{
  "mcpServers": {
    "xberg": { "command": "xberg", "args": ["mcp"] }
  }
}

See MCP integration guide.

AI Coding Assistants

Install the Xberg plugin from xberg-io/xberg. Ships extraction APIs, OCR backends, configuration, and language conventions.

The bundle targets Claude Code, Codex, Cursor, Gemini CLI, Kimi, OpenCode, Factory Droid, Hermes, Pi, Agent Plugins and GitHub Copilot CLI. Read the plugin guide for runtime resources, CLI prerequisites and configuration. MCP-capable adapters expose the product's tools; Hermes uses a local stdio bridge and explicitly selected skills. Pi loads skills and prompt templates and requires the product CLI for executable operations.

Install only the public Agent Skills, without a plugin or MCP registration, using the skills CLI:

npx skills add xberg-io/xberg --list
npx skills add xberg-io/xberg

Choose the skills and target agents interactively. Installing instructions does not install the product CLI or configure provider credentials.

/plugin marketplace add xberg-io/xberg
/plugin install xberg@xberg

/plugins add https://github.com/xberg-io/xberg

Search for xberg and select Install Plugin.

Settings → Plugins → Add from URL → https://github.com/xberg-io/xberg, then select xberg.

From a checkout of this repository:

gemini extensions install ./plugin

The manifest is inside plugin/. See the Gemini extension reference for local installation and updates.

droid plugin marketplace add https://github.com/xberg-io/xberg
droid plugin install xberg@xberg

copilot plugin marketplace add https://github.com/xberg-io/xberg
copilot plugin install xberg@xberg

Install from a local checkout now:

pi install ./plugin

After the next plugin release, install the same package used by OpenCode:

pi install npm:@xberg-io/opencode-xberg

Pi loads the bundled skills and prompt templates. Use /xberg-extract and /xberg-ocr to start a task. Install the CLI separately as shown in the plugin README. This package does not register MCP servers, hooks, or executable extensions in Pi.

Add to opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugin": ["@xberg-io/opencode-xberg"]
}

---

Quick Start

Extract text from a document:

use xberg::{extract, ExtractInput, ExtractionConfig};

#[tokio::main]
async fn main() -> xberg::Result<()> {
    let config = ExtractionConfig::default();
    let output = extract(
        ExtractInput::from_uri("document.pdf"),
        &config
    ).await?;

    println!("{}", output.results[0].content);
    Ok(())
}

Common use cases — see Quick start guide for language-specific examples, OCR, batch processing, and API configuration.

---

Capabilities

Supported File Formats (107 formats · 141 file extensions · 56 MIME aliases)

107 formats across 140 unique file extensions, with 56 compatibility MIME aliases, intelligent format detection, and comprehensive metadata extraction.

Office Documents

Category	Formats	Capabilities
Word Processing	.docx, .docm, .doc, .dotx, .dotm, .dot, .odt, .pages, .wpd, .wp, .wp5, .wp6	Full text, tables, images, metadata, styles
Spreadsheets	.xlsx, .xlsm, .xlsb, .xls, .xla, .xlam, .xltm, .xltx, .xlt, .ods, .numbers	Sheet data, formulas, cell metadata, charts
Presentations	.pptx, .pptm, .ppt, .pps, .ppsx, .potx, .potm, .pot, .odp, .key	Slides, speaker notes, images, metadata
PDF	.pdf	Text, tables, images, metadata, OCR support
eBooks	.epub, .fb2	Chapters, metadata, embedded resources
Database	.dbf, .sqlite, .sqlite3, .db, .gpkg, .gpkx	Bounded table extraction, schema metadata, GeoPackage detection
Hangul	.hwp, .hwpx	Korean document format, text extraction

Images (OCR-Enabled)

Category	Formats	Features
Raster	.png, .jpg, .jpeg, .gif, .webp, .bmp, .tiff, .tif	OCR, table detection, EXIF metadata, dimensions, color space
Advanced	.jp2, .jpg2, .j2c, .j2k, .jpc, .jbig2, .jb2, .pnm, .pbm, .pgm, .ppm	OCR via pure-Rust JPEG2000 decoder, JBIG2 support, table detection
HEIC family	.heic, .heics, .heif, .heifs, .hif, .avif, .avcs	EXIF metadata, optional pixel decoding
Vector	.svg	DOM parsing, embedded text, graphics metadata

Audio & Video

Category	Formats	Features
Audio	.mp3, .mpga, .m4a, .wav, .webm	Whisper transcription
MP4 audio track	.mp4, .mpg4, .mp4v, .m4v	Audio-track transcription only
MPEG audio track	.mpeg, .mpg, .mpe, .m1v, .m2v	Audio-track transcription only
WebM audio track	.webm	Audio-track transcription only

Web & Data

Category	Formats	Features
Markup	.html, .htm, .xhtml, .xht, .xml, .kml, .svg	DOM parsing, metadata (Open Graph, Twitter Card), link extraction
Structured Data	.json, .geojson, .jsonl, .ndjson, .yaml, .yml, .toml, .csv, .tsv	Schema detection, nested structures, validation
Text & Markdown	.txt, .adoc, .asciidoc, .vtt, .md, .markdown, .commonmark, .qmd, .rmd, .djot, .dj, .mdx, .doctags, .rst, .org, .rtf	AsciiDoc, CommonMark, MyST Markdown, Quarto, R Markdown, Djot, MDX, DocTags, reStructuredText, Org Mode

Email & Archives

Category	Formats	Features
Email	.eml, .msg, .pst	Headers, body (HTML/plain), attachments, threading
Archives	.zip, .tar, .tgz, .gz, .7z	File listing, nested archives, metadata, recursive extraction

Academic & Scientific

Category	Formats	Features
Citations	.bib, .ris, .nbib, .enw	Structured parsing: RIS, PubMed/MEDLINE, EndNote XML, BibTeX/BibLaTeX
Scientific	.tex, .latex, .typ, .typst, .jats, .nxml	LaTeX, Typst, PubMed JATS
Text notebooks	.ipynb, .md, .py, .R, .jl	Jupyter, MyST-NB, Jupytext percent/light, saved outputs, cell visibility tags
Publishing	.fb2, .docbook, .dbk, .docbook4, .docbook5, .opml	FictionBook, DocBook XML, OPML outlines

Code Intelligence (371 Languages)

Extract structure from 371 programming languages via tree-sitter:

Feature	Description
Structure Extraction	Functions, classes, methods, structs, interfaces, enums
Import/Export Analysis	Module dependencies, re-exports, wildcard imports
Symbol Extraction	Variables, constants, type aliases, properties
Docstring Parsing	Google, NumPy, Sphinx, JSDoc, RustDoc, and 10+ formats
Syntax-Aware Chunking	Split code by semantic boundaries for RAG pipelines
Diagnostics	Parse errors with line/column positions

Powered by tree-sitter-language-pack.

Output Formats (6)

Format	Use case	Example
Plain	Raw text, no markup	"Chapter 1\nIntroduction"
Markdown	Readable, structured, RAG-friendly	"# Chapter 1\n## Introduction"
Djot	Modern lightweight markup	Similar to Markdown but stricter
HTML	Styled, browser-ready	<h1>Chapter 1</h1>
JSON	Machine-readable tree structure	Hierarchical sections with heading levels
DocTags	Docling-compatible tag stream for document elements and tables	<text>Chapter 1</text>

Deployment Modes

Mode	Command	Transport	Use case
Library	xberg::extract()	Async functions	Embed in your application
CLI	xberg extract document.pdf	14 commands	Scripts, batch jobs, CI/CD
REST API	xberg serve	HTTP POST	Microservice, serverless deployment
MCP Server	xberg mcp	stdio or HTTP	Claude, Cursor, IDE agents
Docker	docker run ghcr.io/xberg-io/xberg	All modes	Container deployment

OCR Backends

• Tesseract — Native C FFI (Linux/macOS/Windows) and WASM (browser)
• PaddleOCR — ONNX Runtime, mobile-optimized models
• Candle — Pure Rust, CPU-only, lightweight
• VLM — GPT-4 Vision, Claude Vision, Gemini Vision, or 165 providers via liter-llm

Fallback chains. Extensible via plugin system.

Embeddings

Local (ONNX Runtime):

• Preset models: fast, balanced (default), quality, multilingual
• Dimensions: 384, 768, 1024

Provider-hosted:

• OpenAI, Anthropic, Google, Hugging Face, Mistral, Cohere, and 165 providers total
• Via liter-llm integration

Reranking:

• Local ONNX rerankers (cross-encoder models)
• Provider-hosted: Cohere Rerank, others

Structured LLM Extraction

Local engines: Ollama, LM Studio, vLLM

Remote: OpenAI, Anthropic, Google, Mistral, Cohere, and 165 providers via liter-llm

Schema validation. Temperature, top-p, frequency penalty tuning.

Enrichment

• NER — GLiNER or LLM-based entity recognition
• Redaction — Mask PII (phone, email, SSN, credit card, addresses)
• Summarization — Document and section summaries via LLM
• Translation — Multi-language via LLM
• Page Classification — Tag document pages (cover, toc, content, etc.)
• QR Code Detection — Extract and decode QR codes from images
• Keyword Extraction — YAKE or RAKE algorithms
• Language Detection — Detect document language
• Layout Detection — RT-DETR + TATR models for document structure
• Table Extraction — Cell-level structure and content
• Token Reduction — TOON wire format (~30–50% fewer tokens than JSON)

---

CLI Reference

Command	Subcommands	Purpose
extract	—	Extract text from a single document (path, URL, or stdin)
batch	—	Extract from multiple documents in parallel
detect	—	Identify MIME type of a file
formats	—	List all supported formats and MIME types
version	—	Show Xberg version
cache	stats, clear, manifest, warm	Manage extraction cache and models
tree-sitter	download, list, cache-dir, clean	Manage code-intelligence grammars
doctor	—	Diagnose the local installation and runtime dependencies
serve	—	Start REST API server (default: http://127.0.0.1:8000)
mcp	—	Start MCP server (stdio or HTTP transport)
api	schema	Output OpenAPI 3.1 specification
embed	—	Generate embeddings for text (local or provider-hosted)
chunk	—	Split text into chunks (text, markdown, YAML, or semantic)
completions	—	Generate shell completion scripts

Run xberg --help or xberg <command> --help for detailed options.

---

Documentation

Full guides, API references for every binding, format reference, and configuration docs live at xberg.io.

• Getting Started
• Quick Start
• Guides
• API Reference (Rust core) — every binding has its own page under /reference/
• Format Reference
• Live Demo (browser, WASM)

---

Built with Xberg

Projects that declare Xberg as a dependency. Xberg was previously published as kreuzberg, and most of these projects declare the package under that name.

Project	What it is	Stars
basemind	AI context and content layer for coding agents over one MCP server: code map, document RAG, shared memory and web crawl
delulu	A suite of MCP servers and CLI tools that give your LLM better search and fewer hallucinations
docs-mcp-server	Grounded documentation MCP server, an open-source alternative to Context7, Nia and Ref.Tools
erato	The open-source AI platform
fastmail-cli	CLI and MCP server for Fastmail: email, contacts, masked email, attachments and text extraction
ghfdb-portal	Web portal for the Global Heat Flow Database
hawki-toolkit-file-converter	Prepares and converts PDF files for the HAWKI toolkit
haystack-core-integrations	Integrations that extend Haystack with extra components and document stores
kreuzakt	A search engine for humans and computers, aimed at your most boring documents
lilbee	The whole local AI stack in one executable, with conversational search and cited answers over your files, code and the web
llm-workflow-engine	Power CLI and workflow manager for LLMs
MANSPIDER	Spiders entire networks for files sitting on SMB shares, searching filenames or contents with regex
otoroshi-llm-extension	Connect, secure and manage LLM models behind one OpenAI-compatible API
sift-kg	Turns a collection of documents into a knowledge graph, extracting entities and relationships with an LLM
sirchmunk	Turns raw data into a self-evolving, real-time search and intelligence layer
support-chatbot	Level-1 support chatbot for the Netherlands Red Cross 510 team

Using Xberg in your project? Open a PR adding it to this list.

---

Contributing

Contributions are welcome! See CONTRIBUTING.md for guidelines.

Join our Discord community for questions and discussion.

---

Part of Xberg.io

• Xberg — the open-source content-intelligence engine: text, tables, and metadata from 107 formats (141 file extensions), with OCR, transcription, and code intelligence. MIT.
• Xberg Pro — a complete self-hosted content-intelligence backend in a single container. Commercial.
• Xberg Enterprise — the distributed, governed content-intelligence platform, scaled on Kubernetes with team governance and support. Commercial.
• crawlberg — web crawling and scraping with HTML→Markdown and headless-Chrome fallback.
• html-to-markdown — fast, lossless HTML→Markdown engine.
• liter-llm — universal LLM API client with native bindings for 14 languages and 165 providers.
• tree-sitter-language-pack — tree-sitter grammars and code-intelligence primitives.
• alef — the polyglot binding generator that produces every per-language binding across the 5 polyglot repos.

---

License

MIT License (MIT) — see LICENSE for details.