TurboOCR

aiptimizerMIT活跃维护

基于 TensorRT FP16 加速的 PP-OCRv6 推理服务，在 OmnidocBench 上跑出 200+ img/s，提供 HTTP 和 gRPC 接口，适合高吞吐文档解析与 RAG 场景。

TurboOCR, >200 img/s OmnidocBench. TensorRT FP16, PP-OCRv6, HTTP + gRPC

v3.6.0发布于 2026-09-27 20:40 UTC1,157star117fork同步于 2026-11-07 07:31 UTC

OCR 文档解析 TensorRT FP16 推理服务

对比 GitHub 官网 document-ai document-parsing easyocr fastapi fp16

怎么上手

由 README 提炼，执行前请确认

原文

docker run --gpus all -p 8000:8000 -p 50051:50051 -v trt-cache:/home/ocr/.cache/turbo-ocr ghcr.io/aiptimizer/turboocr:latest
curl -X POST http://localhost:8000/ocr/raw --data-binary @document.png -H "Content-Type: image/png"

硬件要求按 README 说明

Linux + NVIDIA 驱动 595+，Turing 及更新架构 GPU（RTX 20 系 / GTX 16 系起）。纯文本约 4 GB 显存，含版面+表格+公式的完整流水线约 8 GB，每个额外 PIPELINE_POOL_SIZE 副本再占一整套。目前仅发布 NVIDIA 后端，Apple Metal 与 Intel OpenVINO 在测试、AMD ROCm 在开发中。

需要 CUDA支持 ROCm / AMD有 Docker 镜像

下载：源码 zip·GitHub Releases（安装包）本站不镜像安装包

项目说明

由 README 自动翻译提炼

没人工点评？写下你的经验

单卡 GPU 上跑 PP-OCRv6 的文档解析服务，把 OCR、版面、表格、公式一次性输出成 Markdown

基于 TensorRT FP16 加速的 PP-OCRv6 推理服务，在 OmnidocBench 上跑出 200+ img/s，提供 HTTP 和 gRPC 接口，适合高吞吐文档解析与 RAG 场景。

核心能力

用 TensorRT FP16 多流引擎跑 PP-OCRv6 检测与识别
可选 SLANet+ 表格转 HTML、PP-FormulaNet-S 公式转 LaTeX
PP-DocLayoutV3 版面分析与阅读顺序，按请求开启
PDF 页面并行渲染与 OCR，支持导出页面图与自动旋转
同时提供 HTTP 与 gRPC 接口，并暴露 Prometheus 指标

容易踩的坑

首次启动要从 ONNX 构建 TensorRT 引擎，5090 上约 90 秒、老卡可能长达一小时，构建期间 nginx 会返回连接被拒；表格和公式默认不加载，需用 TABLE_BACKEND=slanext、FORMULA_BACKEND=ppformulanet_s 显式开启；源码构建需 GCC 13.3+/C++20、CUDA 与 TensorRT 10.2+、OpenCV 4.x、Drogon 1.9+ 和 gRPC，首次构建会自动下载 PP-OCRv6 模型到 ./models/。

内容由模型根据 README 生成，可能不完整或不准确，以官方文档为准。看原文 README

最近发版

5 个版本v3.6.0

全部 release

v3.6.0v3.6.0 — reading order from the layout model, word boxes on every route, requests past 120 s最新正式版2026-09-27 · 2026-09-27 20:40 UTC
v3.6.0 — reading order from the layout model, word boxes on every route, requests past 120 s

Response shapes are unchanged apart from the new, opt-in words. Detection now matches PaddleOCR's box geometry, so boxes and some text differ slightly from 3.5.x (see below).

Reading order follows the layout model (#37)

reading_order=1 and as_blocks=1 now order the text by the reading order PP-DocLayoutV3 predicts for its boxes, with a guard that keeps a column top to bottom, a tie guard, and the column-splitting XY-cut when the model gives no order. Multi-column pages no longer interleave their columns. OmniDocBench (100-page subset): reading-order edit distance 0.2344 → 0.1625. GPU and CPU order agree (Kendall τ 0.999).

Word boxes (#35)

?words=1 on every HTTP route (words = 20 on the gRPC requests) adds a words array to each result: each word's text, confidence and its own box, in the line's point order. Words split at spaces, every CJK character is a word of its own, punctuation stays with its word; boxes are cut at the line's real ink gaps and trimmed to the word's ink. On /ocr/pdf, text-layer lines take their words from the PDF itself. On 24 unseen OmniDocBench pages: 0.11% of words cut, 0.20% missed.

On the GPU server the per-pixel work runs on the GPU for every line of a page at once (TURBO_WORDS_GPU, default on), with the same boxes as the CPU placement to the pixel, and off the GPU worker on /ocr/raw, /ocr/stream and /ocr. RTX 5090, 32 connections: 197 pages/s with ?words=1 against 218 without (110 with TURBO_WORDS_GPU=0). Requests without words are unaffected.

Requests longer than 120 s

A request is silent on its connection until its response is written, and both the server's idle timeout and the bundled nginx's proxy timeouts were a fixed 120 s, so a long PDF failed with 502 after all its work. IDLE_CONNECTION_TIMEOUT_S / --idle-connection-timeout-s (default 120, 0 = never, up to 86400) sets the limit, and the image's nginx follows it. The server warns at startup when it is shorter than REQUEST_TIMEOUT_MS.

Detection matches PaddleOCR

Two differences from the reference pipeline with the same PP-OCRv6 models: DB's unclip grew each region's pixel outline where PaddleOCR grows its min-area rectangle, so boxes came out about a pixel smaller per side and lost the narrow last glyph of a line (、。,); and tall crops were turned clockwise where PaddleOCR turns them anticlockwise. OmniDocBench (34 pages): word recall 95.05 → 95.23 (tiny), 95.02 → 95.11 (medium); punctuation 91.31 → 92.84 (tiny). The CPU tiny pipeline is about 9% faster. With DISABLE_ANGLE_CLS=1, Latin text running bottom to top now comes out upside down, as in PaddleOCR — the angle classifier is what turns it.

GPU server fixes
- A 10,000-page PDF answered 503 "GPU queue full": all its pages were queued at once. A job now keeps a bounded window of pages in flight; the PDF completes (10 s) while other requests are served.
- CMYK JPEGs answered 503: nvJPEG does not decode them. The JPEG header now sends them to the host decoder.
- The medium models ran out of VRAM at startup: the automatic pool assumed 2 GiB per pipeline. It is now sized from the first pipeline's measured footprint; an explicit PIPELINE_POOL_SIZE is built exactly, or startup fails.
- Line order could change between restarts: lines were banded by y / 10 into fixed rows, so boxes a pixel apart could swap lines. Lines are now clustered by vertical overlap.
Verified

The images built from this tree were verified before tagging. GPU: 438 unit tests, integration 324 passed / 0 failed, all 16 routes, robustness and determinism checks; word boxes identical to the CPU placement (0 of 71,596 words differ); compute-sanitizer memcheck, racecheck, initcheck and synccheck clean; a 90 s mixed soak answered 8,606/8,606 requests with VRAM and memory flat. CPU amd64: 375 unit tests, integration 286 passed / 0 failed, all 14 routes, robustness and

v3.5.62026-09-08 · 2026-09-08 15:23 UTC

查看更新说明

v3.5.6 — removes the allocator settings that are the likely cause of the CPU slowdown reported under WSL2 (#34)

No API or model changes. Supersedes v3.5.5.

What was wrong

Since 3.5.3 (GPU server) and 3.5.5 (CPU server), startup forced a 1 MB mmap threshold and a 4 MB trim threshold on glibc. Every multi-megabyte tensor an inference allocates then became an mmap/munmap pair with fresh page faults. Measured on a 24-core Linux host, that is six times the page faults and twice the kernel time of 3.5.0 for the same document; wall time on that host does not move. The report in #34 is a 2-page document taking 30 s instead of 5 s under Docker Desktop on WSL2 with every core busy. A large munmap rate inside a VM, where each unmap is a TLB shootdown across all vCPUs, would fit that picture, and neither of the two 3.5.5 flags (TURBO_OCR_DISABLE_HOST_IMAGE_POOL, TURBO_OCR_DISABLE_MALLOC_REAPER) touches these settings, which matches the reporter's observation that they changed nothing. How much a VM amplifies that was tested in an 8-vCPU KVM guest on the same host: v3.5.5 there costs twice the page faults and kernel time of 3.5.0 but only 3% of wall time, so the 6× reported under Hyper-V on a Ryzen was not reproduced on KVM on an Intel host. The thresholds are removed because they are measured, pure overhead on every platform; whether they alone explain the reporter's number is for the reporter's machine to show.

What changed

The two thresholds are gone. The arena cap stays, sized to the CPUs the process may run on (its affinity mask, so a pinned container sizes it to its share; at least 8) instead of a flat 8, and it yields to an operator's own MALLOC_ARENA_MAX or GLIBC_TUNABLES setting. The cap bounds the resident floor without a single system call. Memory stays bounded by the mechanisms that actually do it: page-sized image buffers come from the host image pool and never touch malloc, and the idle-memory reaper returns freed pages of every arena to the OS every 5 s. The startup line reports the cap ("arena_max").

Measured

The three candidate configurations were compared under identical conditions: v3.5.5 (thresholds + cap 8), no tuning at all, and the cap alone (shipped).

	v3.5.0	v3.5.5	no tuning	cap only (v3.5.6)
CPU server, 2 × 300-dpi pages, medium + slanext, 8 pinned cores: minor page faults / kernel time (wall time 9.6–9.7 s in every column)	16.4–17.0k / 0.2 s	102–109k / 0.4 s		19–27k / 0.2 s
GPU server, glibc, 20 rounds of 1100 base64 requests: idle RSS first → last, trend of the last third		2666 → 2717 MB, +4.9 MB/round	2698 → 2851 MB, +1.7 MB/round	2696 → 2761 MB, +0.3 MB/round (44 MB above the v3.5.5 end point, but no longer rising)
CPU image amd64, 6 bursts of 120 base64 requests: idle RSS after the last burst (bursts 2 → 6)		6297 MB	6946 MB	6412–6416 MB (6396 → 6412; rerun 6378 → 6416)
KVM guest, 8 vCPUs, same 2 pages: wall / page faults / kernel time	9.8 s / 461k / 0.5 s	10.1 s / 882k / 1.1 s		9.8 s / 490k / 0.6 s
CPU image arm64, 6 bursts: idle memory after the last burst		5.17 GiB	5.85 GiB, climbing	5.30 GiB, flat

A fixed cap of 8 instead of the CPU count (measured, not shipped): 100 MB lower floor on the CPU server (6.33 GB) but the GPU server rises 9.6 MB per round over 20 rounds (2674 → 2856 MB), so the cap scales with the CPUs the process may use, which is at least 8.

jemalloc preloaded instead (measured, not shipped): 80–82k page faults for the same two pages and a 6.98 GB CPU floor, so no advantage over the cap on glibc. Absolute fault counts differ between runs by a few times depending on page-cache state; the ordering of the columns was the same in every run.

GPU throughput and FUNSD accuracy are identical with and without the change (A/B on the same box, run while the GPU was shared with other jobs, so both arms were equally slowed: 265–268 img/s both, F1 85.40% both; the candidate

v3.5.5v3.5.5 — builds on aarch64 and OpenCV 5, toolchain found where it is installed2026-09-06 · 2026-09-06 14:21 UTC

查看更新说明

v3.5.5 — builds on aarch64 and OpenCV 5, toolchain found where it is installed

No API or model changes. Supersedes v3.5.3, and v3.5.4 (tagged minutes earlier; its Dockerfiles missed the new cmake/ directory, so it has no images).

What changed

Native builds on aarch64 (Jetson, GB10 / DGX Spark, Grace). The repository vendors x86-64 copies of PDFium and the PDF renderer; a build on another architecture linked the first and then reported the second as "not found". CMake now reads the ELF header of the vendored PDFium and fetches the pinned bblanchon/pdfium-binaries release for the target when it does not match (SHA-256 verified), and builds the renderer, fastpdf2png 2.0.10, as a subproject with the same toolchain. The server looks for the renderer next to its own executable first, and a binary for another CPU is rejected with its architecture named. The install scripts check the ELF architecture of what is installed and never keep a mismatch.
CUDA and TensorRT are found where distributions put them. nvcc on PATH, else CUDAToolkit_ROOT, CUDA_HOME, CUDA_PATH, /usr/local/cuda, the newest /usr/local/cuda-<version>, /opt/cuda; TensorRT as a tarball root, the installer's /usr/local/tensorrt, or the distribution packages. TENSORRT_DIR became an optional hint. A plain cmake -B build works on a GB10 with the Debian TensorRT packages.
OpenCV 5 compiles. OpenCV 5 moved the contour and transform helpers into its geometry module; the build links it when present and keeps working with OpenCV 4.
A CUDA-enabled ONNX Runtime on aarch64 is the one thing you still build yourself (Microsoft publishes it for x86-64 only). The configure step prints the exact commands; docs/build/native.md has the section.
CPU server: idle-memory reaper. The CPU server now runs the same reaper as the GPU server, so a long-running CPU deployment returns freed request memory on glibc and jemalloc alike.
Logs. nvJPEG decoder messages are structured JSON through the rate-limited logger; a batch the batched API refuses is one line per interval, not one per batch.
fastpdf2png 2.0.10 (the PDF renderer): builds as a subproject, honours PDFium_DIR, continues after short writes, forked workers exit safely and a failed fork() falls back to in-process rendering. Rendered pages are byte-identical to 2.0.0 on x86-64, aarch64 and macOS.
CI builds and tests on GitHub's arm64 runners as well.

Verified


aarch64 (Ubuntu 24.04, OpenCV 4.6)	configure fetches the arm64 SDK, server and renderer build, unit tests pass, PDF results identical to x86-64 line for line
x86-64 (RTX 5090, OpenCV 4.10 and 5.0)	376 unit tests, FUNSD F1 85.40%, response shapes of every endpoint identical to v3.5.2, ten fixture PDF results byte-identical
renderer old vs new	byte-identical pages on 7 PDFs × 1 and 4 workers (x86-64 and macOS), same daemon replies, fork-blocked run still renders every page
`/ocr/pdf` latency (80-page scan / 40 × 23 MP pages)	0.78 s / 1.84 s, unchanged

Upgrade notes

-DTENSORRT_DIR is no longer needed; it still works as a hint.
Source builds produce build/fastpdf2png and build/libpdfium.so next to the server; the committed x86-64 copies in bin/ remain as the fallback on x86-64 only.
TURBO_BUILD_FASTPDF2PNG=OFF skips the renderer build (the Docker images do this and use the script route).

Images

GPU (TensorRT, x86_64): ghcr.io/aiptimizer/turboocr:v3.5.5 (also :latest)
CPU (ONNX Runtime, amd64+arm64): ghcr.io/aiptimizer/turboocr-cpu:v3.5.5 (also :latest, and ghcr.io/aiptimizer/turboocr:v3.5.5-cpu)

Models are unchanged from v3.5.0.

v3.5.3v3.5.3 — one JPEG path, bounded memory, no silent fallbacks2026-09-05 · 2026-09-05 15:32 UTC

查看更新说明

v3.5.3 — one JPEG path, bounded memory, no silent fallbacks

Supersedes v3.5.1 and v3.5.2. No API or model changes; two knobs retired with a warning.

What was wrong

Host memory ratcheted on long-running servers (#33): request buffers freed on 128 worker threads sat in ~80 allocator arenas that never returned them; the server's own trimming only worked with glibc, so a preloaded jemalloc made it a no-op.
The base64 route decoded on the CPU without saying so whenever the GPU decoder was busy or failed, with slightly different pixels than /ocr/raw; the decoder also let the GPU write straight into pageable heap memory on unified-memory systems (HMM), which stalled the driver under jemalloc.

What changed

One JPEG path. Every route decodes JPEG on the replica that runs inference, with that replica's own decoder: /ocr/raw, /ocr (base64), /ocr/stream, /ocr/markdown, /infer, /ocr/batch and gRPC. The decoder uses the GPU's NVJPG engine when it has one and falls back to nvJPEG's hybrid GPU backend for bitstreams the engine refuses; the startup line now says which one is in use instead of claiming hardware offload everywhere. The work-thread decoder pool is gone; NVJPEG_DECODERS and --nvjpeg-decoders are still accepted, ignored with a warning, for one release. Layout-only requests take the device path too.
Faults are errors, not detours. A GPU decoder fault returns 503 GPU_DECODE_FAILED (gRPC UNAVAILABLE, batch slot tag gpu_decode_failed). The host codec decodes only JPEG variants nvJPEG does not support at all (arithmetic, 12-bit, CMYK); when nvJPEG is present the host image decoder refuses JPEG outright, so a detour cannot pass unnoticed. Decoded pixels always land in device memory and are copied back through pinned buffers; the HMM shortcut and NVJPEG_DEVICE_COPY are removed.
Memory is a budget. Page-sized host images live in a fixed pool of reusable pinned buffers (slots = work threads + 2 × replicas + PDF workers, each capped by MAX_IMAGE_PIXELS_MP, printed at startup); the device memory pool has a 256 MiB release threshold; nvJPEG scratch is allocated through the same pools; /ocr reads the base64 text straight from the request body. The work pool defaults to clamp(pool×4, 16, 64) threads, and the idle-memory reaper detects the allocator and releases freed memory on glibc and jemalloc alike.
Two integration guards now fail on the class of bug above: /ocr and /ocr/batch must return the same text as /ocr/raw for the same JPEG, and a progressive JPEG must still succeed.

Measured (RTX 5090, tiny, pool 2, same corpus and engine cache)

Endpoint	v3.5.2	v3.5.3
`/ocr/raw` JPEG, 16 clients	144.9 req/s	144.0 req/s
`/ocr` base64 JPEG, 16 clients	48.8 req/s	145.0 req/s
`/ocr/batch` 8 × 100, 4 clients	35.2 img/s	92.8 img/s
`/ocr/pdf` 80-page scan / 40 × 23 MP pages	794 ms / 1844 ms	793 ms / 1839 ms
`/ocr/markdown` JPEG	14.9 ms	10.9 ms
`/ocr/stream` JPEG	14.6 ms	11.0 ms
`/ocr/raw?text=0&layout=1`	6.4 ms	3.3 ms
`/ocr/pixels` / PNG via `/ocr/raw`	11.7 / 6.2 ms	12.1 / 6.8 ms
FUNSD-50 F1 / img/s	85.40 / 431	85.40 / 426
RSS after 2202 base64 requests	2.06 GB	1.44 GB
jemalloc, six base64 bursts, idle floor	+180 MB per burst	+59 MB over all six
VRAM after base64 load	7.4 GB	7.2 GB

Response shapes of /ocr/raw, /ocr, /ocr/batch, /ocr/pdf, /ocr/markdown, /ocr/stream, /ocr/pixels, /health, /health/ready, /capabilities and /metrics are identical to v3.5.2. Unit tests 370, integration 311 pass (2 pre-existing stale expectations), accuracy suite unchanged.

Upgrade notes

Remove NVJPEG_DECODERS / --nvjpeg-decoders and any HTTP_THREADS=8 workaround; all are unnecessary now (the two decoder knobs are ignored with a warning and go away next release).
TURBO_OCR_DISABLE_HOST_IMAGE_POOL=1 restores OpenCV's allocator for

v3.5.2v3.5.2 — bounded nvJPEG decoder pool2026-09-04 · 2026-09-04 12:37 UTC

查看更新说明

v3.5.2 — bounded nvJPEG decoder pool

Patch release, no API or model changes. Supersedes v3.5.1, which was tagged and published as images but never announced: its pooled decoders failed silently on threads that had not bound the CUDA context and the base64 routes fell back to CPU decoding. v3.5.2 fixes that and is the release to use.

The bug

The GPU server decoded JPEGs for the JSON image routes (/ocr with base64, /ocr/batch) on whichever work-pool thread received the request, using one nvJPEG decoder per thread. Each decoder holds ~190 MB of device memory and ~50 MB of host memory until the process exits, and the server starts max(pool*32, 128) work threads. A server fed JPEGs through those routes therefore grew by one decoder per thread until every thread had one: about 24 GB of VRAM and 6 GB of RSS at the defaults, reached slowly at low concurrency (reported as unbounded growth in #33) and within a minute under 16 concurrent clients on a 32 GB card, which exhausted the device and wedged the driver on teardown. The raw-bytes route (/ocr/raw) and PDF processing were never affected: they decode on the replica that runs inference.

The fix

Decoders now come from one bounded pool owned by the server, leased per decode and returned by RAII. The pool holds NVJPEG_DECODERS instances (default: one per pipeline replica, since decode throughput is bounded by inference), all created at startup so the footprint is visible in nvidia-smi from the first second. When every decoder is busy for 2 s the request decodes on the CPU instead of waiting. New knob: NVJPEG_DECODERS / --nvjpeg-decoders (0 = auto), shown in the effective-config dump and the startup line. A decoder handed across threads binds the calling thread to its CUDA device before decoding, and a failed GPU decode is now logged instead of silently falling back.

Measured (RTX 5090, tiny, pool 2, 1101 mixed 4–20 MP JPEGs, 16 concurrent clients, same corpus and engine cache for both binaries)

path	v3.5.0	v3.5.2
base64 `/ocr`, 32 work threads: decoders / VRAM after / RSS after	35 / 12.6 GB / 2.9 GB	4 / 7.5 GB / 2.0 GB
base64 `/ocr`, 32 work threads: req/s (p50 / p99)	47.2 (261 / 1799 ms)	48.9 (257 / 742 ms)
base64 `/ocr`, 128 work threads (default)	VRAM exhausted, driver wedged	4 decoders, 2202 requests at 48.9 req/s, flat
`/ocr/raw`, 128 work threads: req/s (p50)	145.7 (91 ms)	145.7 (90 ms)
speed matrix, text only / with layout, images/s	431 / 201	447 / 194
PDF, 80 pages / 40 pages at 23 MP	795 / 1849 ms	782 / 1854 ms
FUNSD-50 F1 / img/s (`/ocr/raw`, PNG)	85.40% / 431.8	85.40% / 431.3

Unit tests: 361 cases pass (7 new for the lease pool, 1 for the config knob). Integration suite: 308 pass, 40 skip, 2 fail; the two failures (test_batch_rejects_text0, test_invalid_mode_fallback) fail identically on v3.5.0 because their expectations predate the v3.5.0 fail-loud validation, and are not behaviour changes. Accuracy suite: identical results to v3.5.0 (same fixtures above and below their floors).

Upgrade notes

No configuration change is required. Operators who set HTTP_THREADS=8 as a workaround for #33 can remove it.
Each nvJPEG decoder costs ~190 MB of VRAM; raising NVJPEG_DECODERS above the replica count does not raise throughput.
Unified-memory hosts (GB10 and similar) count the decoder footprint against the same pool as host RSS; the fix removes the slow climb there as well.

Images

GPU (TensorRT, x86_64): ghcr.io/aiptimizer/turboocr:v3.5.2 (also :latest)
CPU (ONNX Runtime, amd64+arm64): ghcr.io/aiptimizer/turboocr-cpu:v3.5.2 (also :latest, and ghcr.io/aiptimizer/turboocr:v3.5.2-cpu)

Models are unchanged from v3.5.0.

用户点评

登录用户的真实使用记录，可按版本和硬件筛选

共 0 条

还没有人点评。第一个说出真实体验的人最有价值。

README 原文

未翻译

在 GitHub 打开

An extremely fast GPU document parser — not just OCR. PP-OCRv6 detection + recognition, plus layout, tables (→ HTML), formulas (→ LaTeX) and reading-order Markdown, the whole pipeline on a single multi-stream CUDA/TensorRT engine, locally (no VLM), behind HTTP and gRPC. Whole-page OCR runs at up to 559 images/s on receipts (one RTX 5090), and full structured parsing (layout + tables + formulas) at ~20 pages/s — where VLM document parsers like PaddleOCR-VL run ~1 page/s. On forms and receipts it is accurate and 15–90× faster than classic OCR engines.

🚀 **559 img/s (receipts) · 520 (forms) · 200+ (dense docs) on one RTX 5090 — fastest by default
🎯 Accurate on forms & receipts — competitive with PaddleOCR-VL, PaddleOCR-Python, RapidOCR, EasyOCR and Tesseract (benchmarks)
🧠 PP-OCRv6 — one model covers Latin + Chinese + Japanese; pick tiny (default) / small / medium
🌐 More scripts — Arabic, Cyrillic, Korean, Thai, Greek via retained PP-OCRv5 recognizers
📄 PDF native — pages rendered and OCR'd in parallel, optional page-image export & auto-rotation
🧩 Layout + reading order — PP-DocLayoutV3 (25 classes) and the order it predicts itself (column-aware XY-cut as fallback), opt-in per request
🔢 Tables & formulas — opt-in SLANet+ table → HTML and PP-FormulaNet-S formula → LaTeX, emitted alongside the text (how to enable)
🔤 Word boxes — ?words=1 adds every word of a line with its own box, cut at the line's real ink gaps
🐳 One-line Docker deploy with TensorRT engines auto-built on first start, Prometheus metrics on /metrics

Full documentation: docs/

Quick Start

Requirements: Linux, NVIDIA driver 595+, Turing or newer GPU (RTX 20-series / GTX 16-series+). Plan for ~4 GB VRAM text-only and ~8 GB for the full pipeline (layout + tables + formulas); each extra PIPELINE_POOL_SIZE replica adds roughly another full set, so lower it on smaller cards.

docker run --gpus all -p 8000:8000 -p 50051:50051 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr \
  ghcr.io/aiptimizer/turboocr:latest

First startup builds TensorRT engines from ONNX. This takes about 90 seconds on a 5090 GPU and up to an hour on older ones. Set TRT_OPT_LEVEL=3 to cut build time 3 to 5x with a small speed regression. The named volume caches the engines, so subsequent starts are instant. During the build, requests return a connection refused error from nginx until the backend is ready. nginx (port 8000) reverse-proxies to Drogon (port 8080), and both start automatically.

curl -X POST http://localhost:8000/ocr/raw \
  --data-binary @document.png -H "Content-Type: image/png"

{"results": [{"text": "Invoice Total", "confidence": 0.97, "bounding_box": [[42,10],[210,10],[210,38],[42,38]]}]}

Start it with the stages you want

All weights are baked into the image — just set the backend env var to load a stage (no paths needed). Layout is on by default; each extra stage still only runs when the request asks for it.

# text + layout (default)
docker run --gpus all -p 8000:8000 -p 50051:50051 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr ghcr.io/aiptimizer/turboocr:latest

# + tables (→ HTML)       add  -e TABLE_BACKEND=slanext
# + formulas (→ LaTeX)    add  -e FORMULA_BACKEND=ppformulanet_s
# + both                  add  -e TABLE_BACKEND=slanext -e FORMULA_BACKEND=ppformulanet_s
# bigger / other language add  -e OCR_MODEL=medium   (tiny | small | medium | arabic | eslav | korean | thai | greek)

Then opt in per request (combine freely; tables/formulas auto-enable layout):

curl -X POST "http://localhost:8000/ocr/raw?layout=1&tables=1&formulas=1" \
  --data-binary @paper.png -H "Content-Type: image/png"
# PDF:  POST /ocr/pdf   ·   PDF → Markdown: POST /ocr/pdf?markdown=1   ·   page → Markdown: POST /ocr/markdown   ·   gRPC: port 50051

GET /capabilities reports which stages a running server has loaded.

→ Docker & deployment · Build from source

Getting higher accuracy

The defaults maximize throughput. Three levers trade speed for accuracy:

1. Bigger model tier — -e OCR_MODEL=small or medium. Same languages, same API; small fixes most of tiny's misreads (lost spaces, stylized fonts) at ~half the speed, medium is the most accurate.

2. Higher detection resolution — images whose long side exceeds 1280 px are downscaled before detection (recognition always reads full-res crops), so small text on phone screenshots or dense scans can go undetected. Raise the cap:

-e DET_MAX_SIDE_LIMIT=2560   # one-time engine rebuild, then cached

3. Orientation on every line — by default the 0°/180° classifier only checks vertical-looking lines, so an upside-down horizontal line is never corrected. For scans with mixed or rotated lines:

-e CLS_ALL_BOXES=1                                # check every line (small throughput cost)
-e CLS_ONNX=x1_0                                  # optional: full-width classifier variant

What each of these costs, measured (every combination, plus full parse with tables + formulas):

Benchmarks

On a single RTX 5090, vs every common OCR engine:

Forms & receipts (English): accurate (FUNSD 92% / CORD 93% word-F1 on the medium tier) and 15–90× faster than every other engine — up to 559 img/s (receipts) on the default tiny tier. FUNSD/CORD are English/Latin-script datasets; the EN+ZH full-document numbers below include Chinese.
Full document parsing (EN+ZH): 0.90 Overall on a 125-doc OmniDocBench subset at 20 pages/s, within ~5 points of PaddleOCR-VL (0.95, same subset) which runs at ~1 pg/s — fully local, no API. (Subset including Chinese pages, not the full 1651-page set.)

→ Full benchmarks & methodology

Models

The pipeline is a small stack of specialised models, not one monolith. Text detection + recognition + line orientation always run; everything else is opt-in and only loads when configured. Each stage links to its own model page.

Stage	Model / arch	Size	Selected by	Docs
Text detection	PP-OCRv6 det (DB, three tiers)	1.7 / 9.4 / 59 MB	`OCR_MODEL` tier (`tiny`/`small`/`medium`)	detection
Text recognition	PP-OCRv6 rec (CRNN + CTC, Latin + Chinese + Japanese)	4.3 / 20 / 73 MB	`OCR_MODEL` tier — default `tiny`	recognition · selection
Line orientation	PP-LCNet textline angle classifier (0°/180° per line)	~1 MB	always on (vertical lines only; every line with `CLS_ALL_BOXES=1`)	classification
Page orientation	PP-LCNet_x1_0_doc_ori (0/90/180/270 whole page)	~7 MB	per request via `/ocr/pdf?autorotate=1`	http api
Layout	PP-DocLayoutV3 (RT-DETR-L, 25 classes)	~124 MB	per request via `?layout=1`; disable with `DISABLE_LAYOUT=1`	layout
Table → HTML	SLANet-Plus (TRT FP16 CNN encoder + hand-written C++ GRU decoder)	~5 MB	`TABLE_BACKEND=slanext` (encoder auto-resolved from the bundle)	table
Formula → LaTeX	PP-FormulaNet-S, in-process pure-C++ (ORT-CUDA-13, no Python)	~294 MB	`FORMULA_BACKEND=ppformulanet_s`	formula

The three OCR tiers (tiny/small/medium) trade speed for accuracy, not language coverage — all cover Latin + Chinese + Japanese (Getting higher accuracy). Other scripts use retained PP-OCRv5 recognizers, also via OCR_MODEL: arabic, eslav (Cyrillic), korean, thai, greek. The legacy v5 Latin det/rec can also be swapped in for A/B comparison — see Running legacy PP-OCRv5.

→ Model selection guide

Tables & formulas

Table and formula recognition are strictly opt-in, per request. The router only loads a backend when one is configured at startup, and a stage runs only when the request explicitly asks for it with ?tables=1 and/or ?formulas=1 (gRPC: the tables / formulas request fields). layout alone never triggers them, so the default path pays nothing. When requested (and a backend is loaded), the response gains tables (HTML + cell quads) and/or formulas (LaTeX) arrays. tables=1/formulas=1 auto-enable layout. Asking for a stage the server wasn't started with is a hard error (400 TABLE_BACKEND_DISABLED / FORMULA_BACKEND_DISABLED), never a silent empty result — check GET /capabilities for what a server supports. (/ocr/markdown always includes both, best-effort, since a faithful Markdown export needs them.)

Capability	Enable at startup	Recognizer
Formula → LaTeX	`FORMULA_BACKEND=ppformulanet_s`	PP-FormulaNet-S
Table → HTML	`TABLE_BACKEND=slanext` (encoder auto-resolved from the bundle)	SLANet-Plus

Startup + request examples are in Quick Start above.

→ Tables · Formulas

Upgrading to v3

v3 renames the server binaries, moves the default engine to PP-OCRv6, and changes several defaults (timeouts, detection resize, input caps). If you are coming from v2.x, read the full migration guide: Upgrading to v3 — breaking changes.

API

One binary serves HTTP and gRPC from a shared GPU pipeline pool.

Endpoint	Purpose
`POST /ocr/raw`	OCR raw image bytes (fastest)
`POST /ocr`	OCR base64 image in JSON
`POST /ocr/pixels`	Zero-decode raw pixel buffer
`POST /ocr/batch`	Batch of images
`POST /ocr/pdf`	PDF → text (optional page images & auto-rotate); `?markdown=1` → whole PDF as Markdown
`POST /ocr/markdown`	Page → faithful Markdown (GPU build; requires layout)
`POST /infer`	OCR + layout / reading-order / blocks in one structured response
`GET /capabilities`	Runtime feature & route discovery
`GET /metrics`	Prometheus metrics
`GET /health` · `/health/live` · `/health/ready`	Liveness / readiness probes

All OCR endpoints accept ?layout=1 (region detection + reading order), ?words=1 (word-level boxes inside every line), and ?tables=1 / ?formulas=1 to additionally run table → HTML / formula → LaTeX on detected regions (strict opt-in — see Tables & formulas).

→ HTTP API · gRPC API · Monitoring

Configuration

Everything is configured by environment variable (and an equivalent CLI flag). Common ones:

Variable	Default	Description
`OCR_MODEL`	`tiny`	`tiny` / `small` / `medium`, or a PP-OCRv5 script model
`DISABLE_LAYOUT`	`0`	`1` skips the layout model (~300–500 MB VRAM)
`LAYOUT_MERGE_MODE`	`all`	Nested-box policy: `all` (keep every box) / `outer` (outer regions only) / `inner` (innermost only). Old `large`/`small`/`union` accepted as aliases.
`LAYOUT_KEEP_NESTED_CHILDREN`	`0`	Only affects `outer`/`inner` modes: `1` still keeps the model's nested child regions (`figure_title`, `footnote`, `formula_number`, `paragraph_title`) instead of dropping them inside a parent. Formulas are always kept; no effect under default `all`.
`CLS_ALL_BOXES`	`0`	`1` runs the 0°/180° orientation classifier on every text line instead of only vertical-looking ones — for scans with mixed or upside-down lines.
`REQUEST_TIMEOUT_MS`	`60000`	Per-request inference deadline; on overrun returns `504` and frees the slot. `0` = unbounded (pre-v3 behaviour).
`IDLE_CONNECTION_TIMEOUT_S`	`120`	How long one HTTP request may run (the server and the bundled nginx close a connection silent that long); raise it for long PDFs, `0` = no limit
`PIPELINE_POOL_SIZE`	auto	Concurrent GPU pipelines

→ Full configuration reference (35+ variables)

Building from Source

# Docker (recommended)
docker build -f docker/Dockerfile.gpu -t turboocr .
docker run --gpus all -p 8000:8000 -p 50051:50051 \
  -v trt-cache:/home/ocr/.cache/turbo-ocr turboocr

# Native (PP-OCRv6 models auto-fetched into ./models/ on first build)
cmake -B build            # finds CUDA and TensorRT itself; -DTENSORRT_DIR=<dir> to point at an unpacked tarball
cmake --build build -j$(nproc)
LD_LIBRARY_PATH=/usr/local/tensorrt/lib ./build/turboocr-server

Needs GCC 13.3+/C++20, CUDA + TensorRT 10.2+, OpenCV 4.x, Drogon 1.9+, gRPC. Wuffs and PDFium are vendored in third_party/.

→ Build guide & GPU-architecture notes

Acknowledgements

Built on open-source work:

PaddleOCR (Baidu) — PP-OCRv6 / PP-OCRv5 detection, recognition, and classification models, plus PP-DocLayoutV3 layout detection. This project would not exist without their research and pre-trained weights.
Drogon — high-performance async C++ HTTP framework.
Wuffs — fast PNG decoder by Google (vendored).
PDFium — PDF rendering and text extraction (vendored).

License

MIT. See LICENSE.

怎么上手

项目说明

核心能力

容易踩的坑

最近发版

v3.6.0 — reading order from the layout model, word boxes on every route, requests past 120 s

Reading order follows the layout model (#37)

Word boxes (#35)

Requests longer than 120 s

Detection matches PaddleOCR

GPU server fixes

Verified

v3.5.6 — removes the allocator settings that are the likely cause of the CPU slowdown reported under WSL2 (#34)

What was wrong

What changed

Measured

v3.5.5 — builds on aarch64 and OpenCV 5, toolchain found where it is installed

What changed

Verified

Upgrade notes

Images

v3.5.3 — one JPEG path, bounded memory, no silent fallbacks

What was wrong

What changed

Measured (RTX 5090, tiny, pool 2, same corpus and engine cache)

Upgrade notes

v3.5.2 — bounded nvJPEG decoder pool

The bug

The fix

Measured (RTX 5090, tiny, pool 2, 1101 mixed 4–20 MP JPEGs, 16 concurrent clients, same corpus and engine cache for both binaries)

Upgrade notes

Images

用户点评

README 原文

Quick Start

Start it with the stages you want

Getting higher accuracy

Benchmarks

Models

Tables & formulas

Upgrading to v3

API

Configuration

Building from Source

Acknowledgements

License

相关项目