synthadoc

axoviq-aiAGPL-3.0活跃维护

把原始文档编译成结构化、本地优先的 wiki 知识库,用可读可维护的编译式方案替代传统 RAG,支持自管理自改进。

Synthadoc: An open-source LLM knowledge compilation engine that turns raw documents into structured, local-first wikis. A transparent, human-readable alternative to traditional RAG, which can be self-managed and self-improved without the use of any tools.

v1.5.1发布于 2026-11-03 17:22 UTC2,002star +66 / 7 天188fork同步于 2026-11-10 23:46 UTC

怎么上手

由 README 提炼,执行前请确认
pip install "synthadoc[vectors]"
synthadoc --version
硬件要求按 README 说明

README 未说明显存或 GPU 要求;pip 安装仅需 Python 3.11+,Docker 方式只需容器运行时,无需在宿主机安装 Python。

下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

把原始文档用 LLM 编译成本地 Markdown 结构化 wiki,替代传统 RAG 的可读知识库引擎

把原始文档编译成结构化、本地优先的 wiki 知识库,用可读可维护的编译式方案替代传统 RAG,支持自管理自改进。

核心能力

  • 读取 PDF、表格、PPT、网页、图片、视频、Word、TXT 和 AI 会话记录
  • 自动建立交叉引用、检测矛盾并标记孤立页面
  • 每个答案都标注来源,输出为本地 Markdown 文件
  • 支持 Obsidian 插件、CLI、Web UI 和 MCP 四种接口
  • 提供 lint 等维护工作流,可持续自优化知识库

容易踩的坑

需要自行配置 LLM API key(如 MINIMAX_API_KEY),否则引擎无法工作;Docker 方式下修改 .env 后必须删除并重建容器,restart 不会重新读取;升级已有安装需运行 synthadoc plugin upgrade 才能更新 Obsidian 插件。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本v1.5.1
  1. v1.5.1v1.5.1 - Synthadoc Community Edition 最新正式版2026-11-03 · 2026-11-03 17:22 UTC

    Synthadoc Community Edition v1.5.1 Release Notes

    Synthadoc Community Edition v1.5.1 is released.


    v1.5.1 is a fix release. As part of its headline addition, the contradiction resolver and content-health resolver no longer replace the entire page body; instead, they propose individual block-level edits (old text + new text) which you can accept or skip one by one. BM25 top-hit precision has been improved from 68 to 70% in the latest release, along with startup-error, query, and ingest fixes.


    Highlights

    1. Scoped Edit Output - Per-Block Approval in the Web UI

    Both the contradiction resolver and the content-health resolver now express their changes as a list of [old_text, new_text] block pairs rather than a full-page rewrite. The web UI displays a ScopedConfirmCard which allows you to approve or skip each block independently before any changes are made.

    Block confirm card - single-view layout:

    • Block mode (default): each proposed edit is shown as a before/after pair with an include/skip checkbox. Blocks that cannot be applied cleanly (no match, overlapping spans, heading mutation, or sentence loss) are shown with a labelled rejection badge rather than a checkbox.
    • "Edit full page manually" toggle: switches to a CodeMirror 6 editor with an Edit/Preview toggle. The editor ships an inline markdown linter (unclosed fenced code blocks, unclosed wikilinks, heading level skips) and a collapsible Reference panel showing the page before any changes.
    • An optional rationale field, populated by the resolver when the reason for a change is not obvious, appears above the card header.

    Block apply engine safeguards:

    • Exact-match first, normalised-whitespace fallback - the engine matches each old_text against the raw page body exactly (no whitespace is damaged), then falls back to normalized matching to map the position to the original raw span.
    • Overlap conflict detection - two blocks whose raw spans overlap are both rejected with "conflict" so neither silently clobbers the other.
    • Heading-mutation guard - any edit that removes or renames an existing heading is rejected with "heading_mutation", unless the resolver explicitly opts in (content-health allows heading mutations for orphaned_heading and empty_section fixes).
    • Sentence-loss guard - any edit that drops more than 20 % of the sentences in old_text is rejected with "sentence_loss".
    • Fix-episode detection - a rollback always restores the snapshot taken at the start of the current fix episode, not a stale snapshot from a previous one.

    Why it matters:

    • Previously, approving a resolver meant accepting the entire rewritten page at once. With scoped edits, you can skip a single awkward change while accepting the rest.
    • The CodeMirror fallback preserves the ability to hand-edit when the proposed blocks miss the mark, without losing the accumulated context that the resolver built.
    • Rejection badges give feedback on why a block was not applied - no more silent no-ops.

    Token savings vs. full-page rewrite:

    Block-level edits reduce output tokens for each resolver call. Input tokens are unchanged (the full page must be in context either way); the savings are entirely on the generation side, which is billed at 3–5× the input rate on most providers.

    Page sizeBlock edit outputFull-page outputSavings
    Short (~400 words)~350 tokens~550 tokens~35%
    Medium (~800 words)~400 tokens~1,100 tokens~65%
    Long (~1,500 words)~450 tokens~2,000 tokens~78%

    *Assumes 1–2 blocks per contradiction (the disputed claim + 1 nearby sentence), which is typical for a localised factual conflict. Savings translate to roughly 40–50 % total call

  2. v1.5.0v1.5.0 - Synthadoc Community Edition 2026-10-28 · 2026-10-28 16:24 UTC
    查看更新说明

    Synthadoc Community Edition v1.5.0 Release Notes

    Synthadoc Community Edition v1.5.0 is released.


    v1.5.0 is a major capability release. Its headline additions are semantic search on by default with a CJK-aware fusion strategy and an independent benchmark suite, a knowledge graph cluster overview that scales to thousands of pages with named clusters and ~6 s load time at 5,000 pages, a heading-invariant check that prevents heading mutations across all apply paths, shared rollback across the contradiction, orphan, and content-health resolvers, and a Kimi provider via DashScope streaming. The release also ships a deterministic security scanner in CI (zero LLM cost), and a broad set of stability fixes.


    Highlights

    1. Semantic Search - On by Default, CJK-Aware, Benchmarked

    New wikis now ship with vector = "auto" in config.toml. When the vectors add-on is installed (or when using the Docker image, which bundles it), semantic search is active immediately - no manual configuration step.

    The search strategy fuses BM25 keyword scores with local BAAI/bge-small-en-v1.5 vector embeddings. Chinese, Japanese, and Korean pages are automatically routed to BM25 only - the model is English-only and embedding CJK text would degrade results, so those pages stay on keyword search while English pages gain semantic coverage.

    Benchmark results (independent corpora):

    • Top-5 precision with semantic search: 68% → up from 48% with BM25 alone (paraphrase questions on a 300-page wiki)
    • BM25 alone retains a 10 pp precision advantage on exact-name queries; fusion preserves that precision
    • Keyword search latency: ~20 ms at 2,000 pages; semantic add-on: +15 ms median (query embedding overlaps BM25 fetch)
    • CJK cross-lingual retrieval unchanged - validated on independent Chinese/English corpora

    Search latency benchmark: BM25 and semantic search at different wiki sizes Latency at 500, 1,000, and 2,000 pages. BM25 stays flat; semantic add-on adds ~15 ms. Both well under perceptible threshold.

    Independent corpora benchmark: top-5 precision by question type Top-5 precision on an independent test corpus not used during development. Semantic fusion leads on paraphrase queries; BM25 leads on exact-name queries; fusion is never worse by more than 2 pp.

    Design-choice comparison: fusion weight, candidate count, and reranking strategies Parameter sweep across fusion weights and candidate pool sizes. Selected configuration marked.

    Paraphrase precision by wiki size Semantic search advantage grows with wiki size: larger wikis have more near-duplicate phrasing for which keyword search undersells.

    Why it matters:

    • Users asking questions in their own words - not using the exact phrasing of a page - now get better answers automatically.
    • CJK wikis are safe: the feature never degrades non-English retrieval.
    • No operational change for existing wikis. An upgrade leaves vector = false in place; change to "auto" to opt in.

    📖 Design §38 — Semantic search 📖 User guide — Appendix N: Semantic search 📖 Benchmark index 📖 Blog - Semantic benchmark


    2. Knowledge Graph for Large W

  3. v1.4.3v1.4.3 - Synthadoc Community Edition 2026-10-20 · 2026-10-20 15:35 UTC
    查看更新说明

    Synthadoc Community Edition v1.4.3 Release Notes

    Synthadoc Community Edition v1.4.3 is released.


    v1.4.3 is a hardening and improvement release. Its headline additions are Anthropic prompt caching (lower cost and faster responses on repeated queries and 80-90% ingest cost savings), a security hardening pass that resolves all findings from an external static analysis audit, and Docker in-container management commands (plugin upgrade and templates sync now work directly inside a running container). It also delivers a broad set of content health, tool-call, provider, and Qwen improvements.


    Highlights

    1. Anthropic Prompt Caching

    System prompts are now sent with cache_control: {"type": "ephemeral"} on every Anthropic API call, enabling the 5-minute cache window. Cache token counts (cache_read_input_tokens and cache_creation_input_tokens) flow through to estimate_cost() so your cost display and session summaries remain accurate.

    Why it matters

    • Lower cost on repeated queries. Queries against the same wiki within a 5-minute window reuse the cached system prompt, which is typically the largest block of tokens in every request. This maps directly to a lower cost per query on all Anthropic models.
    • Faster first-token latency. Cache hits skip tokenisation of the system prompt on the Anthropic side, reducing time-to-first-token noticeably on longer prompts.
    • Transparent to the user. No configuration change is needed. The cache is active automatically on every Anthropic API key.

    2. Docker In-Container Management

    synthadoc plugin upgrade and synthadoc templates sync now work directly inside a running container - no host Python required. Previously these commands required a host-side synthadoc installation.

    # Apply an Obsidian plugin update to all registered wikis
    docker exec <container> synthadoc plugin upgrade
    
    # Refresh seeds.md, ROUTING.md, and agent files for all template/demo wikis
    docker exec <container> synthadoc templates sync --force
    

    Why it matters: teams running Synthadoc in a container no longer need to maintain a separate host Python environment just to push plugin or template updates. The full upgrade workflow is now self-contained in the image.

    📖 Docker Deployment Guide


    3. Security Hardening

    An external static analysis audit (modelsatlas.com L0–L4 methodology) identified several findings in the previous release. All HIGH and MEDIUM findings are resolved in v1.4.3.

    FindingCategoryResolution
    Function("return this")() in Obsidian plugineval-equivalent pattern (HIGH)Replaced withglobalThis. Fixed durably in the esbuild pipeline — every future build applies the patch automatically. Semantically identical on Electron/Chromium.
    UTF-8 BOM in_strip_bom sourceSource-level BOM (HIGH)Replaced literalU+FEFF character with chr(0xFEFF) - the source file now contains no BOM bytes; runtime behaviour is unchanged.
    UTF-8 BOM in README and docsSource-level BOM (MEDIUM)Stripped from all affected files:README.md, README-pypi.md, docs/design.md, docs/user-quick-start-guide.md, and four Python source files.
  4. v1.4.2v1.4.2 - Synthadoc Community Edition 2026-10-13 · 2026-10-13 21:33 UTC
    查看更新说明

    Synthadoc Community Edition v1.4.2 Release Notes

    Synthadoc Community Edition v1.4.2 is released.


    v1.4.2 is a feature release with two headline additions: an official Docker image for server, team, and CI/CD deployments, and the Content Health Resolver, the 8th agentic maintenance workflow, which finds and repairs structural quality problems in wiki pages with side-by-side diff approval and one-step rollback.

    It also includes fixes for long-running Anthropic calls, lifecycle lint, broken-citation reporting, OKF export, and a security patch for the Obsidian plugin build.


    Highlights

    1. Docker deployment

    Synthadoc now ships an official OCI image, chenp/synthadoc, built for linux/amd64 and linux/arm64. It runs on Docker Engine, Docker Desktop, the Apple container tool (macOS 26+, Apple Silicon), and WSL Containers (wslc, Windows 11).

    Why it matters

    • One server for the whole team. Run a wiki on a shared host. Teammates use the Web UI (http://<server>:7070/app) or connect an MCP client (/mcp); no per-user Python environment is needed.
    • Reproducible runtime. Pin chenp/synthadoc:1.4.2 to get the same engine in dev, staging, and production. The CI image installs the exact PyPI release.
    • Automation-ready. Scheduled ingest, lint, and lifecycle jobs run in GitHub Actions, GitLab CI, or any OCI runner. synthadoc lint run --wait blocks until lint finishes and exits non-zero if the server is unreachable.
    • Clean isolation. Only the mounted wiki folder is shared with the host, which helps on Windows, where Python environment conflicts are common.
    • Your workflow stays the same. Wiki files stay on the host, so Obsidian keeps reading them directly. Symmetric port mapping (-p 7070:7070) means the Obsidian plugin, Web UI, and scripts need no reconfiguration.

    Use cases

    ScenarioHow
    Team knowledge base on a shared serverOne container per wiki; teammates use the Web UI or MCP
    Several domain wikis with cross-wiki queriesdocker/compose/multi-wiki.yml; the cross-wiki registry is inlined, so no host setup is needed
    Fully local, no cloud API keydocker/compose/with-ollama.yml runs Synthadoc with an Ollama sidecar
    Nightly ingest + lint in CIdocker/examples/github-actions-ingest.yml

    What's in the box: HTTP API, Web UI, and MCP over HTTP on a single port; a non-root runtime (uid 1000); API keys passed only via --env-file; a health check that follows the wiki's configured port; docker exec <container> synthadoc <cmd> without -w; and a new synthadoc serve --host flag (also useful outside Docker).

    📖 Docker Deployment Guide

    2. Content Health Resolver: the 8th agentic maintenance workflow

    Wiki pages that are compiled and recompiled by an LLM drift over time. Repeated ingest passes leave duplicate sections, incomplete sources leave thin or empty sections, and rewrites by other workflows can break markdown structure. The Content Health Resolver finds these problems automatically and fixes them only with your approval.

    How it works to close the loop

    1. Detect (no LLM, no cost). A heuristic scanner flags six issue types: duplicate_section, empty_section, thin_section, unclosed_code_block, orphaned_heading, and broken_list_nesting. It runs after every ingest, immediately after any other workflow writes a page, and on every full lint pass.
    2. **Surfac
  5. v1.4.1v1.4.1 - Synthadoc Community Edition 2026-10-05 · 2026-10-05 21:37 UTC
    查看更新说明

    Synthadoc Community Edition v1.4.1 Release Notes

    Synthadoc Community Edition v1.4.1 is released.


    v1.4.1 is a fix release. It addresses bugs accumulated since v1.4.0, covering the DeepSeek provider, cross-wiki query routing, provider API key errors, OKF type mapping, and test-infrastructure reliability.


    Bug Fixes

    DeepSeek thinking config (#453)

    Two provider bugs corrected for users running deepseek-v4-flash:

    1. _build_extra_body() was routing DeepSeek through the Qwen branch, sending {enable_thinking: false} - a flag DeepSeek's API ignores. DeepSeek now correctly sends {thinking: {type: disabled}} like all non-Qwen providers.
    2. When a reasoning model returns content that is entirely <think>...</think> blocks, the previous fallback silently used raw reasoning_content as the wiki page body, writing chain-of-thought text into the knowledge base. For the DeepSeek provider this now raises a ValueError so the ingest job fails visibly. MiniMax's legitimate reasoning-as-answer design (prose answer in the reasoning side-channel) is preserved - the guard is scoped to provider == "deepseek" only.

    Cross-wiki peer base URL (#448)

    Peer URLs in the cross-wiki coordinator hardcoded 127.0.0.1, silently treating any non-loopback peer as unreachable. Fixed to read an optional host field from the registry entry (wikis.json). Existing entries without host behave exactly as before; a remote peer can now be registered by adding "host": "<address>" to its wikis.json entry.

    Cross-wiki no_cache parameter (#444)

    The no_cache cache-bypass flag was accepted by the CLI but not forwarded through the stream endpoint (GET /cross-wiki/query/stream) or the WebUI React hook (useQueryStream). Cross-wiki cache bypass now works end-to-end.

    Real wiki names in wikis_querying SSE event (#443)

    The wikis_querying server-sent event was emitting internal placeholder identifiers instead of the registered display names of the wikis being queried. It now emits actual registered wiki names.

    MissingApiKeyError for missing provider API keys (#442)

    A missing API key previously called cli_error() which raises typer.Exit(1). When raised inside a background job worker, str(typer.Exit(1)) == "1" produced the unhelpful "Lint dead: 1" message in the Obsidian UI. Replaced with MissingApiKeyError(RuntimeError) carrying the env-var name, provider name, and signup URL. The orchestrator treats MissingApiKeyError as a permanent failure (no retries); direct HTTP endpoints return HTTP 401.

    OKF v0.2 type mapping (#442)

    The hardcoded "concept" fallback for all page types replaced with a proper _OKF_TYPE_MAP:

    Synthadoc typeOKF v0.2 type
    concept, eventarticle
    person, organization, technology, location, productreference
    unknown / Nonearticle

    Cross-wiki merge sort defensive access (#438)

    _merge_results raised KeyError when a peer wiki response omitted the score field (older peer versions or test stubs). Changed to .get("score", 0) so such results sort to the end rather than crashing the entire cross-wiki query.

    Hook subprocess race condition (#453)

    Test polling loops waited for output file existence but not for non-empty content. On Windows a subprocess creates the file on open(..., "w") before the write completes, causing JSONDecodeError on an empty file read. Polling now checks stat().st_size > 0 in addition to existence.


    Commits

    | Hash | Description | Issue | | --------- | -----------------------------------------------------------------------------------------------------

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    Synthadoc

          .-+###############+-.
        .##                   ##.
       ##    .----.   .----.    ##
      ##    /######\ /######\    ##
      ##    |######| |######|    ##
      ##    | [SD] | | wiki |    ##
      ##    |######| |######|    ##
      ##    \######/ \######/    ##
       ##    '----'   '----'    ##
        '##                   ##'
          '-+###############+-'
    
           S Y N T H A D O C
        Community Edition  v1.5.1
      ────────────────────────────────
      Domain-agnostic LLM wiki engine
    

    Document version: v1.5.1

    Engineered for solo users and enterprises alike, providing a domain-specific knowledge base that scales seamlessly while maintaining accuracy through autonomous self-optimization.

    Built for individuals, small teams, and large organizations who need a knowledge base that stays accurate as documents accumulate.

    Documentation, FAQ and articles: https://axoviq.com/resources/synthadoc

    Synthadoc reads your raw source documents — PDFs, spreadsheets, PPTs, web pages, images, videos, Word files, TXTs, and AI session transcripts (.jsonl) — and uses an LLM to synthesize them into a persistent, structured wiki. Cross-references are built automatically, contradictions are detected and surfaced, orphan pages are flagged, and every answer cites its sources. Outputs are stored as local Markdown files, ensuring seamless integration and autonomous management within Obsidian or any wiki-compliant ecosystem.



    Table of Contents


    Who Is It For?

    Synthadoc scales from a single researcher to a company-wide knowledge platform:

    Team sizeTypical use case
    Solo / 1–2 peoplePersonal research wiki, freelance knowledge base, indie hacker documentation - run it free on Gemini Flash or a local Ollama model with zero ongoing cost
    Small team (3–20)Centralized internal knowledge base for startups and departments that aggregates diverse individual data sources into a unified, high-integrity wiki. The system automatically resolves contradictions and scales autonomously, ensuring organizational intelligence grows in tandem with your team
    Medium / enterpriseCompliance-sensitive knowledge bases that must stay local; per-department wikis on separate ports; audit trail for every ingest and cost event; hook system for CI/CD integration; OpenTelemetry for ops dashboards

    Built for large wikis. Measured on wikis of thousands of pages: keyword search finds the pages for an answer in about 20 ms at 2,000 pages; semantic search, on by default for new wikis when installed, puts the right page in the top 5 for 70% of questions asked in your own words, up from 48%; and the knowledge graph opens a 5,000-page wiki as a clickable overview of named topic clusters in about 6 seconds.

    Knowledge graph of a 5,000-page wiki: one bubble per topic cluster, named after its main pages, with a clickable list of clusters below The web UI's Graph tab on a 5,000-page test wiki: 100,801 links summarised as 24 named clusters. Click a cluster to see its pages. How it works

    No cloud account. No vendor lock-in. The wiki is plain Markdown — open it in any editor, back it up with git, sync it with any cloud drive.


    Inspiration and Vision

    "The LLM should be able to maintain a wiki for you." — Andrej Karpathy, LLM Wiki gist

    Most knowledge-management tools retrieve and summarize at query time. Synthadoc inverts this: it compiles knowledge at ingest time. Every new source enriches and cross-links the entire corpus, not just appends a new chunk. The wiki is the artifact — readable, editable, and browsable without any tool running.

    Long-term alignment:

    DirectionHow Synthadoc moves there
    Agent orchestrationOrchestrator dispatches parallel ingest, query, and lint sub-agents with cost guards and retry backoff
    Sub-agent skills/pluginsFeaturing a 3-tier lazy-load capability system, the platform allows for the injection of custom skills and hooks via a plug-and-play interface, ensuring core stability is never compromised during extension
    LLM wiki vs. RAGPre-compiled structured knowledge beats query-time synthesis for contradiction detection, graph traversal, and offline access
    CLI / HTTPA unified interface via CLI and RESTful endpoints, the system streamlines full-spectrum integration: from data ingestion and querying to automated linting, security auditing, and job orchestration
    Local-firstAll data stays on your machine; localhost-only network binding; no cloud dependency except the LLM API itself
    Provider choiceLLM backends including free-tier Gemini and Groq, paid Anthropic/OpenAI/DeepSeek/MiniMax/Qwen (DashScope), local Ollama and Qwen, and coding-tool CLI providers (Claude Code, Opencode) — no API key required if you already have a subscription

    Problems Addressed

    RAG retrieves document chunks at query time. Synthadoc compiles knowledge at ingest — synthesising sources into a linked, audited wiki graph so contradictions are caught, claims are traced to sources, and the artifact survives outside the tool.

    ProblemSynthadoc approach
    Contradictions blended silentlyIngest-time conflict detection; page flaggedstatus: contradicted; auto-resolve or queue for human review
    No links between related content[[wikilinks]] auto-built on every ingest pass; weighted graph (wikilink + co-source signals) with Louvain clustering in web UI
    Orphan pages never surfacedLint reports unreferenced pages with ready-to-paste index entries
    LLM output can be overconfidentAdversarial second-LLM pass flags overstated claims, unsupported superlatives, and contestable facts per page
    Claims lack source traceability^[file:L-L] citation on every claim; Source Viewer in Obsidian; PDF page resolution; broken-citation lint
    Knowledge lifecycle invisible5-state machine (draft → active → contradicted / stale → archived); auto-transitions via lint; immutable event log
    Repeat ingest is expensive3-layer cache (embedding, LLM, provider prompt) — repeat lint on unchanged pages costs near-zero tokens
    Knowledge locked in proprietary toolsPlain Markdown + YAML frontmatter; OKF v0.2 compatible; fully offline-readable in any editor
    Wiki structure drifts with growthscaffold regenerates index, AGENTS.md, and purpose.md from current wiki state without touching linked pages
    Migration requires full re-ingestionSingle-zip backup + restore with port/domain rewriting; no re-ingestion needed
    Cost and compliance exposureLocalhost-only; per-job token+cost log; configurable soft-warn and hard-gate thresholds

    Citation quality: Generated pages include inline citations linking every claim to its source lines. Pages without citations trigger a model-compatibility warning — use Gemini 2.5 Flash or higher for reliable citation annotation.

    Active page protection: Pages promoted to active status are protected — sources that contradict them are flagged for review rather than overwriting the human-reviewed content.


    Why Synthadoc?

    Every Yes below is a built-in feature — no add-ons or upgrades required.

    Knowledge Quality

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Ingest-time synthesis — sources compiled into the wiki at ingest; not re-summarised at query timeYesNoPartialNo
    Contradiction detection & resolution — conflicting claims flagged status: contradicted; auto-resolve available; full conflict historyYesNoNoNo
    Adversarial claim review + gate — concurrent second-LLM pass flags overstated claims, unsupported superlatives, and contestable facts per page; configurable gate auto-demotes pages that exceed the warning threshold to contradictedYesNoNoNo
    Claim-level provenance — ^[file:L-L] citation on every claim; Source Viewer in Obsidian; PDF page resolution; broken-citation lintYesNoPartialNo
    5-state lifecycle machine — draft → active → contradicted / stale → archived; auto-transitions via lint; immutable event log; cascade link cleanup on archive; only active and stale pages enter the BM25 search indexYesNoNoNo
    Pre-LLM source sanitizer — strips zero-width chars, bidi overrides, hidden HTML, and instruction-override phrases before any LLM callYesNoNoNo

    Knowledge Structure

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Weighted knowledge graph + D3 visualisation — [[wikilinks]] auto-built at ingest; co-source edges connect pages compiled from the same document; edge thickness reflects combined weight; dashed edges for co-source-only relationships; Louvain cluster colouring; click node to query; scales to thousands of pages — wikis over 300 pages open on a cluster overview (named clusters, click to drill in), 5,000 pages in ~6 s at 60 fps; Obsidian graph panel: same graph inside Obsidian via Canvas, type filter, click node to open pageYesNoPartialNo
    Orphan page detection — unreferenced pages surfaced by lint with ready-to-paste index entriesYesNoNoNo
    Query-scoped routing — ROUTING.md maps wiki branches to page slugs; queries auto-select relevant branches; new pages auto-slottedYesNoNoNo
    Candidates staging — ingest pages to a staging area first; review, promote, or discard before they enter the live wikiYesNoNoNo
    Scaffold automation — regenerates index categories, AGENTS.md/CLAUDE.md/GEMINI.md, and purpose.md from current wiki state; protected pages never overwrittenYesNoNoNo
    Domain templates — 30 fully-authored templates (finance, technology, healthcare, legal, and more) so you start with the right agent guidelines, routing, and structure from day oneYesNoNoNo

    Search & Query

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Query decomposition + gap detection — compound questions split into parallel BM25 sub-queries; thin results trigger a knowledge-gap callout with suggested web searchesYesPartialNoNo
    BM25 TF fallback + compound identifier search — reliable results on small corpora (IDF collapse → TF fallback); underscore identifiers expanded at index and query time so capex growth matches capex_growthYesNoNoNo
    Web search → wiki pages — Tavily search fans out into parallel URL ingest jobs; gap callout in web UI suggests searches inlineYesNoNoNo
    Semantic search — on by default for new wikis when installed (pip install "synthadoc[vectors]", built into the Docker image): BM25 fused with local embeddings (BAAI/bge-small-en-v1.5); Chinese/Japanese/Korean text stays on keyword search; finds pages asked about in your own words, keeps exact-name precision (benchmark)Yes (optional)VariesNoNo
    Streaming output + query cache — token-by-token streaming; cache key = question + wiki version; auto-invalidates on ingest or lifecycle changeYesPartialPartialPartial
    Proportional context budget — sources allocated proportionally to model context window (60 % wiki / 20 % history / 15 % system / 5 % index); replaces fixed top-N capYesNoNoNo
    Cross-wiki federation — synthadoc query --cross-wiki fans out a single query across all registered running wikis; LLM auto-routes to relevant wikis; [[wiki-name::PageTitle]] citation pills; amber offline banner when a target wiki is unreachable; graceful degradation when wikis are offlineYesNoNoNo

    Interfaces & Integration

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Obsidian integration — native plugin: ingest modal, streaming query, lint report, lifecycle controls, context pack builder, provenance viewer, export modal, knowledge graph panel (Canvas force graph, type filter, hover tooltip, click-to-open page); background vault monitoring (auto-snapshot on every file save, 2 s debounce, dedup so unchanged saves are free); Reading View set as default on install so citation chips are visible immediatelyYesNoNoNo
    Web chat UI — synthadoc web: streaming answers, session sidebar, multi-turn history, knowledge-gap callouts, knowledge graph tab, light/dark/system theme toggleYesNoYesYes
    MCP server — 12 tools; Claude Desktop (stdio), Claude Code (SSE), n8n/LangGraph (HTTP/SSE); brain+memory architecture; no double-LLM cost for readsYesNoNoNo
    Context packs — goal → sub-questions → token-budget evidence pack; REST + MCP callable; paste into any LLM chat as grounded contextYesNoNoNo
    Export formats — llms.txt, llms-full.txt, GraphML, JSON (provenance + lifecycle), OKF v0.2 bundle; lifecycle-filtered; zero extra LLM callsYesNoPartialNo
    Multi-platform agent skill files — AGENTS.md (Codex/OpenCode), CLAUDE.md (Claude Code), GEMINI.md (Gemini CLI); all include full CLI quick-reference, domain guidelines, MCP tool table; regenerated by scaffoldYesNoNoNo

    Content Sources

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Multi-format ingest — PDF, DOCX, PPTX, XLSX/CSV, Markdown, TXT, images (vision), web URLs, YouTube transcripts, AI session transcripts (.jsonl)YesVariesPartialPartial
    Web search decomposition — broad topics decomposed into focused Tavily keyword searches; results merged and deduplicatedYesNoNoPartial
    YouTube transcript ingest — timestamped transcript + executive summary; no API key; auto-generated captions supportedYesNoYesNo
    Multilingual / CJK queries — Chinese, Japanese, Korean — no false knowledge gapsYesLimitedNoNo
    Multiple LLM providers + coding tools — Gemini, Groq, Qwen, MiniMax, DeepSeek, Anthropic, OpenAI, Ollama, Kimi; Claude Code and Opencode (no API key needed)YesNoNoNo

    Operations & Trust

    CapabilitySynthadocTypical RAGNotebookLMNotion AI
    Local-first + offline artifact — source documents never leave your machine; compiled wiki is plain Markdown, fully readable offline in any editor without the serverYesVariesNoNo
    Portable backup / restore — single zip: wiki pages + audit/lifecycle DB + config; port and domain rewriting on restore; migrate machines without re-ingestingYesNo — re-ingest requiredNo — AI metadata lostNo
    Lifecycle content snapshots + rollback — page body captured at every lifecycle transition (manual CLI/Obsidian, lint-driven auto-transition, or Obsidian vault save); deduplicated so unchanged saves cost nothing; browse per-page version history with lifecycle history (newest-first, with reason); restore any prior version with lifecycle rollback; rollback saves the current body first so it is always undoableYesNoNoPartial — Notion has time-based edit history (plan-gated, no lifecycle tie, no auditable rollback)
    Cost guard + full audit trail — per-job token + cost log; soft-warn and hard-gate thresholds; audit citations validates every claim citation; immutable event logYesNoNoNo
    Citation faithfulness audit — on-demand LLM verification that each claim's text is actually supported by its cited source lines; per-page result cache with staleness detection; findings surfaced in the Obsidian source viewerYesNoNoNo
    Resumable job queue + retry — every ingest/lint job persisted with status and error; batch a hundred documents and resume after a crashYesNoNoNo
    Custom skills + CI hooks — subclass BaseSkill for new file formats; 2 hook events (on_ingest_complete + on_lint_complete); example git auto-commit hook included; blocking hooks can gate operationsYesLimitedNoNo
    Per-source truncation flag — --max-source-chars caps any source (PDF, DOCX, web page, plain text) before the LLM call; truncated sources flagged with truncated: true in frontmatter and warned in lint outputYesNoNoNo
    Docker deployment — official chenp/synthadoc image (amd64 + arm64); one container per wiki with HTTP API, Web UI, and MCP on one port; Compose recipes for multi-wiki cross-wiki queries and an Ollama sidecar; non-root, secrets via --env-file onlyYesVaries — usually DIYNoNo
    Multi-wiki isolation — each wiki on its own port with independent config, audit trail, and job queue; switch with synthadoc useYesNoPartialNo
    Agentic maintenance workflows — nine conversational workflows (stale-page re-ingest, broken wikilinks fix, lint report, scaffold regeneration, contradiction resolver, orphan resolver, broken citation resolver, content health resolver) via web UI or synthadoc workflow CLI; confirm-before-act gate on all writes; graph sidebar chips trigger workflows with one clickYesNoNoNo
    Sensitive Data Retract — scan all wiki pages for API keys, emails, SSNs, credit cards, and custom patterns; apply [REDACTED] in place; background scan is incremental (only pages modified since the last cycle are checked); server auto-scans ingested pages immediately after each ingest job completes; --changed-only CLI flag for incremental manual runs; synthadoc retract status captures the full retraction audit trail (slug, pattern names, counts — never the sensitive values themselves)YesNoNoNo

    Business value

    ValueHow
    Faster onboardingNew team members query the wiki instead of digging through documents
    Audit trailEvery ingest recorded inaudit.db with source hash, token count, and timestamp
    Cost controlConfigurable thresholds; 3-layer cache reduces repeat spend
    ComplianceLocal-first — source documents and compiled knowledge never leave your machine
    ExtensibilityHooks fire on every event; custom skills load without a server restart

    Architecture

    Synthadoc Architecture

    For full architecture details, data models, API reference, and plugin development guide see docs/design.md.


    What's Included

    See docs/design.md — Appendix A: Release Feature Index for a full feature list by version.


    Installation

    Choose one way to run Synthadoc:

    Install with pipRun with Docker
    Host requirementsPython 3.11+A container runtime only, with no Python or pip on the host
    Best forPersonal use on one machineTeam or shared servers, CI/CD, isolated or reproducible runtimes
    StepsProduction, then Set your API keys and Install a wiki and start the engineDocker below, then the Docker Deployment Guide

    Production

    Prerequisites: Python 3.11+. No Node.js, no Git, no build steps.

    pip install "synthadoc[vectors]"   # recommended — includes semantic search
    pip install synthadoc              # lightweight install, no semantic search
    synthadoc --version                # confirm it works
    

    The Obsidian plugin is bundled inside the package. New wikis created with synthadoc install have it installed automatically. If you are upgrading an existing Synthadoc installation, run synthadoc plugin upgrade to push the updated plugin binary to all registered wikis.

    Docker

    Prerequisites: a container runtime only (Docker Engine or Desktop, Apple container, or wslc). No Python, pip, or synthadoc install on the host: every command, including creating the wiki, runs in a container.

    For server deployments, team wikis, CI/CD pipelines, and isolated personal use, Synthadoc ships an OCI image, chenp/synthadoc, for linux/amd64 and linux/arm64. It works with Docker Engine and Docker Desktop, the Apple container tool (macOS 26+ Apple Silicon), and WSL Containers (wslc, Windows 11). One container serves one wiki. The HTTP API, Web UI (/app), and MCP over HTTP (/mcp) all share the wiki's port, and the wiki files stay on the host so Obsidian keeps working unchanged.

    Commands are shown for Linux / macOS. On Windows, expand the Windows (Command Prompt) block for single-line cmd.exe versions: \ line continuation and ~ in -v paths don't work there. (In PowerShell, replace %USERPROFILE% with $env:USERPROFILE.)

    1. Create the wiki in a one-off container. It lands in ~/wikis/history-of-computing; note the port it prints, e.g. 7070. docker run downloads the image on first use. Run docker pull chenp/synthadoc:latest later to update a cached image.

    mkdir -p ~/wikis
    docker run --rm -v ~/wikis:/wikis chenp/synthadoc:latest \
      synthadoc install history-of-computing --target /wikis --demo
    
    if not exist "%USERPROFILE%\wikis" mkdir "%USERPROFILE%\wikis"
    docker run --rm -v "%USERPROFILE%\wikis:/wikis" chenp/synthadoc:latest synthadoc install history-of-computing --target /wikis --demo
    

    2. Put your LLM API key in ~/wikis/.env, never on the command line. Create it with any text editor (Windows: notepad "%USERPROFILE%\wikis\.env"). For example, to use MiniMax:

    # ~/wikis/.env
    MINIMAX_API_KEY=sk-...
    # optional — enables web search ingest
    TAVILY_API_KEY=tvly-...
    

    Keep comments on their own line; --env-file does not strip a trailing # … from a value.

    Then select the same provider in ~/wikis/history-of-computing/.synthadoc/config.toml. Under [agents], comment out the active default = line and uncomment the matching one:

    [agents]
    default = { provider = "minimax", model = "MiniMax-M3", thinking = "disabled" }
    

    3. Run the server in a container, using the same port on both sides of -p:

    docker run -d --name history-of-computing \
      -v ~/wikis/history-of-computing:/wiki \
      -p 7070:7070 \
      --env-file ~/wikis/.env \
      chenp/synthadoc:latest
    
    docker run -d --name history-of-computing -v "%USERPROFILE%\wikis\history-of-computing:/wiki" -p 7070:7070 --env-file "%USERPROFILE%\wikis\.env" chenp/synthadoc:latest
    

    4. Use it. Open the Web UI at http://localhost:7070/app, or run the CLI inside the container (same on every platform):

    docker exec history-of-computing synthadoc query "Who was Alan Turing?"
    docker exec history-of-computing synthadoc lint run --wait
    

    After editing config.toml, run docker restart history-of-computing. After changing .env, remove and re-create the container (docker rm -f history-of-computing, then step 3 again), because restart does not re-read .env.

    See docs/docker-deploy.md for the full deployment guide: architecture, API key setup, platform-native runtimes, Docker Compose (single wiki, multi-wiki with cross-wiki queries, Ollama sidecar), CI/CD, file permissions, security, and maintenance.

    Docker users: the rest of this Installation section (Development, Set your API keys with shell exports, and Install a wiki and start the engine) is for the pip install. With Docker, API keys go in the .env file, and any synthadoc … command elsewhere in this README runs as docker exec <container> synthadoc … (no -w needed).


    Development

    For developers modifying the Python engine, running the test suite, or developing the Obsidian plugin TypeScript.

    Additional prerequisites: Git (any). Node.js 18+ only if modifying the plugin TypeScript source.

    Step 1 — Clone and install
    git clone https://github.com/axoviq-ai/synthadoc.git
    cd synthadoc
    pip install -e ".[dev]"
    

    [dev] adds pytest, respx, and the other test dependencies. Tests require a source checkout — they are not included in the pip wheel.

    Step 2 — Run the test suite
    pytest --ignore=tests/performance/ -q
    

    Expected: all tests pass, 0 failures. Performance benchmarks (optional):

    pytest tests/performance/ -v --benchmark-disable
    
    Step 3 — Obsidian plugin development (optional)

    The compiled plugin is bundled with the package and kept up to date by CI — no build step is needed to work on the Python side or run tests.

    If you modify the TypeScript source under obsidian-plugin/src/, recompile and sync it into the Python package manually:

    cd obsidian-plugin
    npm install           # first time only, or after package.json changes
    # edit src/main.ts or other source files
    npm run build         # compile TypeScript → main.js (also syncs to synthadoc/data/obsidian-plugin/)
    npm test              # run Vitest unit tests
    cd ..
    

    Set your API keys

    At least one LLM API key is required — unless you use Claude Code or Opencode as your provider (no separate API key needed — see Coding tool CLI providers).

    Synthadoc defaults to Gemini Flash — free tier, no credit card, 1 million tokens per day. Get a key at aistudio.google.com/app/apikey (click "Create API key").

    ProviderFree tierVisionGet key
    Gemini FlashYes — 15 RPM / 1M tokens/day, no credit cardYesaistudio.google.com
    GroqYes — rate-limitedNoconsole.groq.com
    OllamaYes — runs locally, no key (GPU required)Model-dependentollama.com
    QwenYes — 1M free tokens (90-day trial), then paid DashScopeModel-dependentbailian.console.aliyun.com
    MiniMaxNo — pay-per-tokenYesplatform.minimax.io
    DeepSeekNo — pay-per-token (very cheap text rates)Noplatform.deepseek.com
    KimiNo — pay-per-token; kimi-k3 ($3/$15/1M, 1M ctx), kimi-k2.6 ($0.95/$4/1M)Nobailian.console.aliyun.com
    AnthropicNoYesconsole.anthropic.com
    OpenAINoYesplatform.openai.com
    Claude CodeIncluded with subscription — no API keyNoSetprovider = "claude-code" in config.toml
    OpencodeFree via Opencode Zen — no API keyNoSetprovider = "opencode", model = "opencode/big-pickle" in config.toml; connect first: run opencode → /connect → select Zen
    # macOS / Linux — add to ~/.bashrc or ~/.zshrc to persist
    export GEMINI_API_KEY=AIza…          # default — free tier, 1M tokens/day
    export GROQ_API_KEY=gsk_…            # alternative free tier — 100K tokens/day
    export ANTHROPIC_API_KEY=sk-ant-…    # paid — highest quality
    export OPENAI_API_KEY=sk-…           # paid
    export MINIMAX_API_KEY=…             # paid — text rates (image support)
    export DEEPSEEK_API_KEY=…            # paid — text rates (no image support)
    export QWEN_API_KEY=…                # DashScope cloud — 1M free tokens trial (DASHSCOPE_API_KEY also accepted)
    export KIMI_API_KEY=…                # Kimi (DashScope) — kimi-k3 flagship, kimi-k2.6 general (QWEN_API_KEY also accepted)
    export TAVILY_API_KEY=tvly-…         # web search (optional)
    
    # Windows cmd — current session only
    set GEMINI_API_KEY=AIza…
    set GROQ_API_KEY=gsk_…
    set ANTHROPIC_API_KEY=sk-ant-…
    set OPENAI_API_KEY=sk-…
    set MINIMAX_API_KEY=…
    set DEEPSEEK_API_KEY=…
    set QWEN_API_KEY=…                   # DASHSCOPE_API_KEY also accepted
    set KIMI_API_KEY=…                   # QWEN_API_KEY also accepted
    set TAVILY_API_KEY=tvly-…
    
    

    Web search uses Tavily (TAVILY_API_KEY) — optional, only needed for synthadoc ingest "search for: …" jobs. Get a free key at tavily.com.


    Install a wiki and start the engine

    A wiki is a self-contained knowledge base — a folder of Markdown pages maintained and cross-referenced automatically by Synthadoc. The fastest way to get started is the History of Computing demo (13 pre-built pages, no LLM API key required to browse).

    # Linux / macOS
    synthadoc install history-of-computing --target ~/wikis --demo
    
    # Windows (cmd.exe)
    synthadoc install history-of-computing --target %USERPROFILE%\wikis --demo
    

    Then start the engine:

    # Foreground — logs stream to the console
    synthadoc serve -w history-of-computing
    
    # Background — releases the terminal
    synthadoc serve -w history-of-computing --background
    

    The server binds to http://localhost:7070 (localhost-only). Leave it running while you work — the Obsidian plugin, CLI ingest commands, and query commands all talk to it.

    To switch LLM provider, edit [agents] in <wiki-root>/.synthadoc/config.toml and restart synthadoc serve. See Appendix — Switching LLM providers for step-by-step instructions.

    To stop a background server:

    synthadoc stop -w history-of-computing
    

    Upgrading: after updating synthadoc (via pip install --upgrade "synthadoc[vectors]" or git pull), restart the server to pick up the new code, then run these to keep registered wikis in sync:

    synthadoc plugin upgrade         # push updated Obsidian plugin binary to all registered wikis
    synthadoc templates sync --force # demo and template wikis — refresh seeds, routing, agent files, and stubs
    

    Neither command requires the server to be running.

    Upgrading a Docker deployment: docker pull chenp/synthadoc:latest, then remove and re-create the container with the same docker run command; wiki data on the host is untouched. Then run inside the container to sync the plugin and templates:

    docker exec <container> synthadoc plugin upgrade      # push updated plugin binary to the wiki vault
    docker exec <container> synthadoc templates sync      # refresh seeds, routing, agent files, and stubs
    

    See Docker Deployment Guide — Updating the image.


    Quick-Start Guide

    The History of Computing demo includes 13 pre-built pages, raw source files covering clean-merge, contradiction, and orphan scenarios, and a full walkthrough of key Synthadoc feature.

    Full step-by-step walkthrough: docs/user-quick-start-guide.md

    The guide covers:

    1. Verify the demo server started (banner, health check)
    2. Install the Synthadoc plugin (auto-installs Dataview) and open the vault
    3. Review wiki structure and key files (index, purpose, AGENTS.md, dashboard)
    4. Query the pre-built wiki — including knowledge gap detection
    5. Batch ingest all demo source files
    6. Run lint — auto-promote clean pages to active
    7. Manage page lifecycle — 5-state machine (draft → active → stale/contradicted/archived), manual transitions, immutable audit trail
    8. Resolve a contradiction
    9. Fix an orphan page
    10. Run the adversarial lint pass — flag overstated claims across all pages
    11. Web search ingestion with automatic decomposition
    12. Ingest a YouTube video
    13. Enrich the wiki with scaffold (regenerate/update index, purpose, AGENTS.md)
    14. Audit features (token cost, history, events)
    15. Schedule recurring operations
    16. Set up query-scoped routing with ROUTING.md
    17. Stage and review candidate pages before promoting them
    18. Build a context pack for grounded LLM prompts
    19. Verify claim provenance — source-line citations, broken citation audit, global provenance table
    20. Export your wiki — llms.txt, llms-full.txt, GraphML wikilink graph, agent-ready JSON with provenance and lifecycle history, OKF v0.2 bundle for zero-code agent consumption
    21. Use the web chat UI — streaming answers, session-aware hint chips, citations in-browser
    22. Query caching — understand how answers are cached and how to bypass with --no-cache
    23. Backup and restore — create a portable wiki zip, restore on a different machine
    24. Knowledge graph — weighted edges (wikilink + co-source signals), explore clusters in the web UI Graph tab, click a node to query it; large wikis open on a cluster overview
    25. Ingest an AI session transcript — turn Claude Code, Codex, or chat conversations into structured wiki pages
    26. Agentic maintenance workflows — nine conversational workflows via web UI or CLI: re-ingest stale pages (bulk or by slug), scan and fix broken wikilinks, run lint and view the full report, run scaffold and regenerate wiki files, resolve contradicted pages interactively (diff shown before every write, human approval required), resolve orphaned pages by inserting n