Skill_Seekers

yusufkaraaslanMIT活跃维护 有专家点评

把文档网站、GitHub 仓库和 PDF 自动转成 Claude AI 技能包,并检测内容冲突,省去手工整理知识库的麻烦。

Convert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection

v3.10.0发布于 2026-09-30 18:53 UTC15,296star +3 / 7 天1,554fork同步于 2026-11-10 23:27 UTC

专家点评

上手难度:进阶更新于 2026-11-11

把文档站、仓库和 PDF 预先处理成 Claude 技能包,省去手工整理上下文

适用场景

  • 给内部框架或第三方 SDK 建长期可复用的技能包:一次抓取后导出到 Claude、LangChain、Pinecone 等多个目标,比每次对话临时贴文档更稳定
  • 离线批量构建知识包:CI 里跑 skill-seekers create 再 package,产物进版本库,避免在线抓取带来的不确定性
  • 需要从代码库而非文档提取结构信息时:AST 解析能给出类/函数与设计模式层面的摘要,纯文本抓取做不到这一点
  • 已有 MCP 客户端、想让 agent 按需查询本地文档集:内置 MCP 服务省掉自己写一层检索封装

踩坑

  • 非默认来源要装 extra:视频、Notion、Confluence、Jupyter、PPT 各自对应独立 extra,只装主包会在跑到那一步才报错;视频视觉还要额外跑 create --setup 装匹配的 PyTorch 和 easyocr,这一步在无 GPU 或驱动不匹配的机器上最容易卡住
  • issue 里反复出现抓取 GitHub 仓库时报 name 'logger' is not defined 这类运行时错误,说明非文档站路径的代码分支覆盖不如主流程;把 GitHub 仓库作为来源时先小规模试跑
  • 有 issue 指出抓取会带入已关闭 issue 和多余元数据导致 token 膨胀,产出物体积和上下文占用需要在打包后自己检查,不能默认导出即最优
  • AI 增强默认走 claude,需要可用 API 或本地 agent;纯离线环境要显式绕开这条路径

同类对比

同类里 llms.txt 直取 + 自写 chunk 脚本适合来源单一、只要纯文本的场景;需要成熟检索管线而非技能包产物时,走 LlamaIndex / LangChain 的 document loader 更直接。

怎么上手

由 README 提炼,执行前请确认
pip install skill-seekers
skill-seekers create https://docs.djangoproject.com/
skill-seekers package output/django --target claude
硬件要求按 README 说明

README 未说明显存或 GPU 型号要求;视频视觉依赖安装后可用 skill-seekers create --setup 自动检测 GPU 并装对应 PyTorch 变体。前置要求为 Python 3.10+ 和 Git。

下载:源码 zip·GitHub Releases(安装包)本站不镜像安装包

项目说明

由 README 自动翻译提炼

把文档站、GitHub 仓库和 PDF 转成 Claude 等 AI 技能包的数据预处理工具

把文档网站、GitHub 仓库和 PDF 自动转成 Claude AI 技能包,并检测内容冲突,省去手工整理知识库的麻烦。

核心能力

  • 支持 18 种来源类型,含文档站、GitHub、PDF、视频、Notion
  • 三层发现机制抓取 SPA 站点,自动识别 llms.txt 加速
  • AST 解析代码库并检测 10 种 GoF 设计模式
  • 一次抓取可导出到 22 个目标,含 Claude、LangChain、Pinecone
  • 内置 MCP 服务与 68 个 workflow 预设

容易踩的坑

视频、Jupyter、PowerPoint、Confluence、Notion 等来源需另装对应 extra,如 skill-seekers[video-full];视频视觉依赖装完后还需运行 skill-seekers create --setup 安装匹配的 PyTorch 与 easyocr;AI 增强默认走 claude,需相应 API 或本地 agent 可用。

内容由模型根据 README 生成,可能不完整或不准确,以官方文档为准。看原文 README

最近发版

5 个版本v3.10.0
  1. v3.10.0最新正式版2026-09-30 · 2026-09-30 18:53 UTC

    [3.10.0] - 2026-09-30

    Theme: Seeker HUD (beta) — a local web UI for the whole toolchain — plus a path-traversal security fix, three machine-readable CLI commands, an opt-in SQLite skill search index, MiniMax video input, and PDF vector-figure extraction.

    The Seeker HUD web UI is in beta. It ships in this release for early feedback: screens, /api/* routes, and the UI state stored under ~/.skill-seekers/ui/ may change between minor releases without a deprecation period. The CLI, the MCP server, and every existing platform adaptor are unaffected. Please report HUD issues on GitHub.

    Security

    • Path traversal in the fetch_config MCP tool (CWE-22) (#462 reported by @Harmenszoon, fixed in PR #464 by @emecii) — the git-URL mode passed the raw config_name argument into cache_dir / f"temp_{name}", and clone_or_pull both deletes that path (on refresh or a failed pull) and clones attacker-chosen content into it, so a crafted name could delete or overwrite directories outside the cache. The shared clone boundary now validates the cache name, and fetch_config validates config_name once for all three modes in both MCP servers (the named-source and API modes wrote <destination>/<name>.json from the raw name too). All name checks now go through one allowlist (services/path_safety.validate_path_segment: leading letter or digit, then letters, digits, ., _, -), which also rejects NUL bytes, whitespace and dot-files such as .git; the workflow, config-publisher and marketplace validators delegate to it. A failed pull on a cached clone now falls through to a fresh clone instead of deleting the cache and reporting an error.

    Added

    • Seeker HUD (beta) — a local web app for the whole toolchain. pip install "skill-seekers[ui]" then skill-seekers ui opens a React + FastAPI interface on http://127.0.0.1:8770 (loopback only) with Overview, Create, Skills, Configs, Workflows, Analyze, Environment, and job-history screens; every long-running action runs as a streamed job. The built frontend is bundled into the wheel, so no Node toolchain is needed to use it. The beta status is shown in the sidebar and the launcher banner and documented in docs/guides/WEB_UI.md; the HUD-specific entries below describe the screens in detail.
    • MiniMax-M3 video input and thinking modes (#468 by @octo-patch) — AgentClient.call_with_video() sends a local MP4/AVI/MOV/MKV clip as an OpenAI-compatible video_url part (MiniMax-M3 only, 50 MB inline cap enforced before the request), and MINIMAX_THINKING=adaptive|disabled (or a thinking= call argument) is carried in the request body. Both are registry-driven — supports_video, video_models, video_max_bytes, thinking_modes, thinking_env on the provider entry — so _call_api still never branches on a provider name. The thinking mode is validated once at client construction; under the Anthropic protocol it is ignored with a warning instead of silently dropped. No built-in pipeline calls the video path yet.
    • Opt-in SQLite search index for generated skills (#393, PR #463 by @emecii) — skill-seekers create <source> --index (or "index": true in a config file) writes scripts/index.db (FTS5 with BM25, LIKE-style fallback where FTS5 is unavailable) and a stdlib-only scripts/search.py that returns ranked file#anchor pointers, and appends a search-first block to SKILL.md. Off by default, Markdown untouched, zero new dependencies; the Claude packager already ships scripts/. Sections are split at headings outside fenced code blocks, index.md table-of-contents pages are skipped, anchors follow GitHub's rules, query tokens are quoted so NOT/OR are plain words, and nothing is installed when a skill has no indexable references. Enhancement and indexing now resolve the skill directory the same way the scrapers do, including a config-file output_dir.
    • skill-seekers doctor --json (#459 by @Whxuan0701) — machine-readable diagnostics for
  2. v3.9.12026-08-03 · 2026-08-03 00:52 UTC
    查看更新说明

    [3.9.1] - 2026-08-02

    Theme: Documentation and project-infrastructure release. No runtime code changed — the package is functionally identical to 3.9.0.

    Added

    • Sponsorship program (#442) — SPONSORSHIP.md (five tiers plus the rules), SPONSORS.md, and a sponsors.json single source of truth rendered into all 12 READMEs by scripts/render_sponsors.py. Adding a sponsor is one JSON edit instead of thirteen hand edits, and CI (--check) fails on drift. .github/FUNDING.yml gains GitHub Sponsors alongside Buy Me a Coffee.
    • Sponsor URLs are policy-checked in code — the renderer refuses any sponsor link carrying utm_/ref/fbclid/gclid-style parameters, enforcing rule 4 of the published policy. This immediately caught 24 UTM-tagged links already present across the READMEs; all sponsor links are now clean.

    Changed

    • README restructured into a lean landing page — 1,371 → 424 lines, with the deep-dive material moved behind links to the docs/ pages that already covered it. Mirrored across all 11 translations (~16,500 → ~5,100 lines total), each keeping its machine-translation notice and language switcher.
    • Repo root decluttered — BULLETPROOF_QUICKSTART.md → docs/getting-started/, the dated DOCUMENTATION_AUDIT_REPORT_2026-05-30.md → docs/archive/historical/, with every reference repointed.
    • QWEN.md reduced from 719 to 64 lines — it duplicated AGENTS.md while advertising three different version numbers at once (v3.6.0, 3.3.0, and "17+ source types" beside "18 source types"). It is now a pointer to AGENTS.md that keeps the essential commands and conventions inline and carries no hardcoded version.

    Fixed

    • Stale and incorrect README claims corrected against the codebase — version badge 3.7.0 → 3.9.0; the documented scan --quick|--comprehensive|--enhance flags do not exist and are now create --preset quick|standard|comprehensive; export targets "16 formats"/"21 platforms" → 22; CLI reference "all 20 commands" → 19; the install-agent table 15 → 19 entries; workflow presets "24+" → 68; and the dead # skill-seekers list-configs line removed.
    • Three overlapping troubleshooting guides unified — root TROUBLESHOOTING.md (485 lines), docs/TROUBLESHOOTING.md (1,102) and docs/user-guide/06-troubleshooting.md (108) shared only 2 of 22 headings, so none was a stale copy. docs/TROUBLESHOOTING.md is now the single comprehensive reference (1,398 lines) with the five sections that existed only at root merged in; the user-guide chapter stays as the short entry in the numbered series.
    • 92 broken relative links repaired across the docs tree (109 → 17) — targets relinked to their real locations, case/separator mismatches fixed (integrations/cursor.md → CURSOR.md, advanced/api-reference.md → reference/API_REFERENCE.md), and links to never-written docs unlinked rather than left as 404s. The remaining 17 are intentional (template placeholders, illustrative generated-output samples, archived snapshots).
    • Version stamps refreshed in 28 docs files (14 English + 14 zh-CN) from v3.6.0 to v3.9.0. Historical references in MIGRATION_GUIDE.md and "added in vX.Y.Z" notes are deliberately untouched.
    • test_cli_paths.py no longer pins a version literal — it asserted "3.9.0" and broke the moment the dev version moved, the same fragility already fixed in test_package_structure.py. It now tracks skill_seekers.__version__.
    • The published MCP Docker image was broken and is now fixed — Dockerfile.mcp installed the MCP SDK with a bare pip install mcp, bypassing the mcp>=1.25,<2 pin in pyproject.toml. Once mcp 2.0.0 shipped, that resolved to 2.x, where FastMCP moved out of mcp.server, so the container died on startup with cannot import name 'FastMCP'. It now installs via the [mcp] extra so the pin applies. Verified by building the image and confirming the server answers /health in ~4s on mcp 1.29.0.
    • **Docker MCP image smoke test po
  3. v3.9.02026-07-29 · 2026-07-29 11:33 UTC
    查看更新说明

    [3.9.0] - 2026-07-29

    Added

    • HTML parser fallback for broken markup (#96, task F1.4) — new parse_html() helper (cli/html_parsing.py) keeps html.parser as the primary parser (well-formed pages stay byte-identical) but retries with lxml (if installed) then html5lib when parsing raises or yields a tag-free tree for tag-bearing markup, so severely malformed pages no longer scrape as empty. Wired into all web-scraper parse sites (doc_scraper.py) and local HTML file parsing (html_scraper.py); html5lib added as a core dependency so the fallback always exists.
    • Doc scraper retries transient network failures (#97) — scrape_page/scrape_page_async now wrap fetches in retry_with_backoff, so a connection blip or 5xx no longer silently drops the page and ships an incomplete skill. Only transient faults retry (connect/timeout errors, 5xx); 4xx is a definitive answer and returns un-retried. Attempts configurable via the new max_retries config option (default 3; 1 disables retrying).
    • MCP fetch_config retries transient network failures (#92) — the registry list/detail/download GETs go through _get_with_retry (3 attempts, exponential backoff); 4xx like "config not found" still returns immediately.
    • Whisper transcription fallback (Tier 2) implemented (#420) — transcribe_with_whisper() was a placeholder that always raised, so local videos without subtitle files produced no transcript despite the documented 3-tier fallback. Now implemented with faster-whisper (device auto, --whisper-model size, language hint, per-segment confidence), and get_transcript() logs the actual transcription error instead of swallowing it.
    • MiniMax image OCR + registry-driven multimodal provider support (#423) — MiniMax joins the AI provider registry for text enhancement and vision OCR (MINIMAX_API_REGION global_en/cn_zh, MINIMAX_API_PROTOCOL openai/anthropic — China-issued keys no longer 401 against the global endpoint). API_PROVIDERS entries now declare wire protocol and supports_images, _call_api branches on protocol (not provider name), and AgentClient.call_with_image() serves all image-capable providers with SKILL_SEEKER_VISION_PROVIDER auto-detection for video_visual frame OCR.
    • word (docx) source in the scrape_generic MCP tool (#41) — the converter existed in CONVERTER_REGISTRY but the MCP surface never listed it, so MCP clients had no way to scrape .docx sources.

    Changed

    • Token-lean GitHub issue defaults (#169) — GitHub-sourced skills no longer bundle a repo's full closed-issue history by default: issue_state defaults to "open" (was "all"), max_issues to 20 (was 100), and labels/milestones are emitted only with the new include_issue_labels/include_issue_milestones flags (default off). issues.md omits empty sections and no longer prints "No labels" stubs. All defaults overridable.

    Fixed

    • Unified multi-source skip_scrape now rebuilds from cache — UnifiedScraper reloads each configured source from .skillseeker-cache/cached extraction files before conflict detection and build, so cached unified configs no longer re-scrape the network (#405).
    • Quality completeness checks no longer match inside code blocks (#229) — SkillQualityChecker._check_skill_completeness ran its workflow-step/prerequisites/error-handling regexes against raw SKILL.md, so a # Step 1: comment in a fenced example counted as workflow guidance. Fenced blocks and inline code are stripped before matching; prose detection unchanged.
    • Pattern-detector precision: getters are not factories, Java Singletons are detectable (#425) — FactoryDetector matched creation keywords as substrings, so a plain Java POJO scored Factory from its getters and a canonical getInstance() Singleton was misclassified as Factory. Creation verbs now match as word-boundary prefixes with accessor prefixes (get/set/is/has) excluded, and SingletonDetector recognizes c
  4. v3.8.02026-06-15 · 2026-06-15 21:23 UTC
    查看更新说明

    [3.8.0] - 2026-06-15

    Theme: The Grand Unification refactor (one build pipeline, one AI transport, one parser definition, in-process MCP tools — see docs/UNIFICATION_PLAN.md) with a 13-bug audit fix-up and a real-world end-to-end CLI testing pass — plus MiniMax-M3, model selection, registry-driven platform targets, a Windows subprocess fix, and codebase de-duplication.

    Fixed

    • Codebase skills built from a config no longer drop the API reference + dependency graph — the local source carried api_reference and dependency_graph, but the unified builder never wrote them out, so a skill built from a scan-emitted *-codebase.json shipped without its API reference (it was stranded in the scrape cache). Both are now promoted into references/codebase_analysis/<source>/ and linked from the index.
    • create ./path defaults to deep analysis — local codebase create defaulted to surface depth, producing an empty code_analysis.json and a misleading "analyzed 0 files" log; it now defaults to deep (matching the scraper default and the scan-config path), with an explicit --depth still taking precedence.
    • package works non-interactively — the quality-gate prompt raised EOFError when stdin wasn't a TTY (CI/pipes); it now auto-proceeds on a non-TTY or with the new --yes/-y flag, while an interactive terminal still prompts.
    • quality prints the score — the standalone command saved quality_report.json but printed nothing; it now shows the score/grade summary, and the report serializes metric levels as "info"/"warning" instead of the Python enum repr "MetricLevel.INFO".
    • doctor no longer miscounts GITHUB_TOKEN as an AI provider key — the API-keys check now names which keys are set, so a bare GitHub token isn't misread as a provider key being configured.
    • estimate <url> gives an actionable error — passing a URL (which create accepts) printed a bare "Config file not found"; it now explains that estimate takes a config file and points at create --dry-run, and exits non-zero.
    • Web sitemap discovery fails fast on unreachable hosts — the pre-crawl sitemap probes used a single scalar timeout; they now use a (connect, read) timeout so an unreachable host doesn't block the full window before the crawl starts.
    • Fork-bomb guard covers the primary LOCAL enhance path — _run_agent_command now marks SKILL_SEEKER_ENHANCE_ACTIVE in the spawned agent's environment (including terminal-mode scripts) and run() refuses nested spawns, so a local agent enhancing a skill can no longer recursively launch enhancement.
    • --dry-run and --output honored for unified configs — create skipped injecting both into UnifiedScraper (every other source type got them); dry-run now previews and returns without creating directories, for legacy and unified configs alike.
    • Trailing-slash --output no longer leaks intermediates into the packaged skill — SkillConverter resolves skill_dir once and strips trailing separators, so --output out/x/ can't place _extracted.json inside the skill directory.
    • snake_case config classification — config_extractor._path_has_word used \b, which never matches inside snake_case names (app_db.yaml); explicit lookarounds fix detection while keeping dbeaver/blog false positives excluded.
    • How-to guides no longer come out empty for control-flow-wrapped tests — step extraction recursively descends with/for/if/try blocks in source order, and empty AST results fall back to the heuristic instead of emitting empty guides.
    • Slack 429 retry exhaustion logs a truncation warning (matching the Discord path) instead of silently breaking pagination.
    • Kimi CLI output parsing no longer swallows unknown records — record boundary is any CamelCase constructor, so new record types (ToolCallPart, …) can't leak internals into extracted text.
    • scan re-runs no longer churn .archived/ — canonical-named files fetched i
  5. v3.7.02026-05-30 · 2026-05-30 20:46 UTC
    查看更新说明

    [3.7.0] - 2026-05-30

    Theme: AI-driven project knowledge base (skill-seekers scan) — bootstrap a complete skill set for a project in one command, with safety/observability/coverage hardening throughout.

    Added

    • skill-seekers scan <dir> command (#327) — point at any project; an AI agent inspects manifests, README, Dockerfile/CI, sampled source files (first 2 KB each), and the git remote, then emits one Skill Seekers config per detected framework plus a <project>-codebase.json for the project's own code. Each config stamped with metadata.detected_version so re-scans report added / version-bumped / removed dependencies. Internationalized canonical-name resolver (CJK + EU language suffixes) so detections like "Godot 引擎" resolve godot. Out-dir cache means re-scans reuse prior emissions and respect manual edits. Doctor-style report with pluralized counts and resolved / AI-generated / unresolved / archived breakdown.
    • Coverage: scan recognizes ~50 manifest types (Pipfile, environment.yml, deno.json, flake.nix, Chart.yaml, stack.yaml, deps.edn, dune-project, BUILD.bazel, …) and walks src/lib/app/cmd/crates/packages/apps/services/backend/frontend plus root-level files (catches Django, flat-layout Python, Go, Rust workspaces, JS monorepos).
    • Cost + safety flags: --max-ai-generations N (default 10) caps unbounded AI generation for monorepos; --dry-run previews what would be emitted without writing or invoking AI; --probe-urls HEAD-probes AI-generated URLs with retry-on-404; --no-fetch / --no-generate / --no-publish-prompt for offline / CI use.
    • Community submission (opt-in): freshly AI-generated configs can be submitted to the community registry via a native-async flow. Pre-checks GITHUB_TOKEN, idempotency-guards against duplicate issues, retries transient failures with backoff.
    • Archival: configs that disappear from detections are moved (not deleted) to out_dir/.archived/<UTC-timestamp>/ so the user never loses hand-edited work and out_dir stays clean.
    • Docs: new docs/getting-started/05-scan-a-project.md; entries in README, FAQ, CLI Reference, Feature Matrix, Config Format, Environment Variables, and the Quick Start cross-link.

    Changed

    • CLI dispatch unified (#327) — scan and doctor now consume the parsed-args namespace directly via Command(args).execute() instead of building a second argparse.ArgumentParser. Eliminates the _reconstruct_argv hack for these commands; remaining ~14 commands flagged for migration.
    • Config schema: detected_version lives under metadata.detected_version (alongside metadata.version for the config-schema version) rather than at top level. Backwards-compatible reader; old top-level placements migrate on next stamp.
    • SourceDetector.CODE_PROJECT_MARKERS is now public (was _CODE_PROJECT_MARKERS); cross-module callers no longer reach into a private attribute.

    Fixed

    • Correctness (#327) — diff layer keyed by stable filename slug instead of internal config name (eliminates phantom add/remove churn); resolve_config_path lookups now append .json so local-disk + user-dir paths actually find files; out-dir cache prevents redundant API/AI calls on re-scan; lowercase filename slugs prevent duplicate-file accumulation across runs.
    • Safety (#327) — atomic JSON writes via os.replace so SIGINT mid-write can't corrupt a config and silently flip it to "removed" on the next scan; _safe_size guards stat() so a broken symlink in src/ no longer crashes the scan; AgentClient.call exceptions caught and logged; AI-generated config names rejected if they fail the registry regex; URL probe catches AI hallucinations of base_url before writing.
    • Observability (#327) — logging.basicConfig in scan so logger.warning/error reaches the user (was silently dropped); non-zero exit code when no configs and no codebase config were emitted, so

用户点评

登录用户的真实使用记录,可按版本和硬件筛选
共 0 条

    还没有人点评。第一个说出真实体验的人最有价值。

    README 原文

    未翻译

    Skill Seekers

    English | 简体中文 | 日本語 | 한국어 | Español | Français | Deutsch | Português | Türkçe | العربية | हिन्दी | Русский

    Version License: MIT Python 3.10+ MCP Integration Tested PyPI version PyPI - Downloads Website GitHub Repo stars PyPI Downloads

    🧠 The data layer for AI systems. Skill Seekers turns documentation sites, GitHub repos, PDFs, videos, notebooks, wikis, and more — 18 source types — into structured knowledge assets, ready to power AI Skills (Claude, Gemini, OpenAI), RAG pipelines (LangChain, LlamaIndex, Pinecone), and AI coding assistants (Cursor, Windsurf, Cline). Prepare once, export to 22 targets.

    💛 Sponsors

    Bronze Sponsors

    Become a sponsor · GitHub Sponsors


    🚀 Quick Start

    # 1. Install
    pip install skill-seekers
    
    # 2. Create a skill from any source
    skill-seekers create https://docs.djangoproject.com/
    
    # Optional: preview how a source will be detected without creating anything
    skill-seekers detect https://docs.djangoproject.com/ --json
    
    # 3. Package it for your AI platform
    skill-seekers package output/django --target claude
    

    You now have output/django-claude.zip, ready to use.

    # Pick a different AI agent for enhancement (default: claude)
    skill-seekers create https://docs.djangoproject.com/ --agent kimi
    skill-seekers create https://docs.djangoproject.com/ --agent-cmd "my-custom-agent run"
    

    🛰️ AI-driven project scan

    Point scan at a project and an AI agent reads its manifests, README, Dockerfile/CI and sampled source imports — then emits one config per detected framework, plus a <project>-codebase.json for your own code:

    skill-seekers scan ./my-react-app --out ./configs/scanned/
    # → react.json, vite.json, tailwind.json, jest.json, my-react-app-codebase.json
    
    skill-seekers create ./configs/scanned/react.json
    

    If a detection has no existing preset, the AI generates a fresh config; on exit you can optionally publish it back to the community registry.

    All 18 source types

    skill-seekers create facebook/react            # GitHub repository
    skill-seekers create ./my-project              # Local codebase
    skill-seekers create manual.pdf                # PDF
    skill-seekers create report.docx               # Word
    skill-seekers create book.epub                 # EPUB
    skill-seekers create notebook.ipynb            # Jupyter
    skill-seekers create openapi.yaml              # OpenAPI/Swagger
    skill-seekers create presentation.pptx         # PowerPoint
    skill-seekers create guide.adoc                # AsciiDoc
    skill-seekers create page.html                 # Local HTML (or a whole dir)
    skill-seekers create feed.rss                  # RSS/Atom
    skill-seekers create curl.1                    # Man page
    
    # Video (YouTube, Vimeo, or local — needs skill-seekers[video])
    skill-seekers create --video-url https://www.youtube.com/watch?v=... --name mytutorial
    skill-seekers create --setup                   # auto-install GPU-aware visual deps
    
    skill-seekers create --space-key TEAM --name wiki               # Confluence
    skill-seekers create --database-id ... --name docs              # Notion
    skill-seekers create --chat-export-path ./slack-export --name team-chat  # Slack/Discord
    

    See the Scraping Guide for every source type and its options.


    📦 Installation

    pip install skill-seekers              # Core: scraping, GitHub, PDF, packaging
    pip install skill-seekers[all-llms]    # + every LLM platform
    pip install skill-seekers[mcp]         # + MCP server
    pip install skill-seekers[all]         # Everything
    

    Not sure what you need? Run the wizard: skill-seekers-setup

    InstallAdds
    skill-seekers[gemini]Google Gemini support
    skill-seekers[openai]OpenAI ChatGPT support
    skill-seekers[all-llms]All LLM platforms
    skill-seekers[mcp]MCP server for Claude Code, Cursor, etc.
    skill-seekers[video]YouTube/Vimeo transcript & metadata extraction
    skill-seekers[video-full]+ Whisper transcription & visual frame extraction
    skill-seekers[jupyter]Jupyter Notebook support
    skill-seekers[pptx]PowerPoint support
    skill-seekers[confluence]Confluence wiki support
    skill-seekers[notion]Notion pages support
    skill-seekers[rss]RSS/Atom feed support
    skill-seekers[chat]Slack/Discord chat export support
    skill-seekers[asciidoc]AsciiDoc support
    skill-seekers[all]Everything

    Video visual deps (GPU-aware): after installing skill-seekers[video-full], run skill-seekers create --setup to auto-detect your GPU and install the matching PyTorch variant + easyocr.

    Prerequisites: Python 3.10+, Git. New here? → Bulletproof Quick Start 🎯


    📚 Documentation

    I want to...Read this
    Get started quicklyQuick Start — 3 commands to your first skill
    Understand the conceptsCore Concepts
    Scrape sourcesScraping Guide — all 18 source types
    Enhance skills with AIEnhancement Guide · Enhancement Modes
    Export skillsPackaging Guide
    Build workflowsWorkflows
    Look up a commandCLI Reference — all 19 commands
    ConfigureConfig Format · Environment Variables
    Set up MCPMCP Setup · MCP Reference
    Integrate with RAG / IDEsLangChain · RAG Pipelines · Cursor · Windsurf · Cline
    Handle huge doc setsLarge Documentation — 10K–40K+ pages
    Understand the architectureUML Architecture — 14 diagrams
    Fix a problemTroubleshooting

    Complete documentation index: docs/README.md


    🎯 What you get

    Use caseOutputPowers
    AI SkillsComprehensive SKILL.md + reference filesClaude Code, Gemini, GPT
    RAG pipelinesChunked documents with rich metadataLangChain, LlamaIndex, Haystack
    Vector databasesPre-formatted data ready for upsertPinecone, Chroma, Weaviate, FAISS, Qdrant
    AI coding assistantsContext files your IDE AI reads automaticallyCursor, Windsurf, Cline, Continue.dev

    Export targets (22)

    skill-seekers package output/react --target claude      # → Claude Skill (ZIP + YAML)
    skill-seekers package output/react --target langchain   # → LangChain Documents
    skill-seekers package output/react --target llama-index # → LlamaIndex TextNodes
    skill-seekers package output/react --target ibm-bob     # → IBM Bob skill directory
    

    LLM platforms (12): claude · gemini · openai · minimax · opencode · kimi · deepseek · qwen · openrouter · together · fireworks · markdown RAG & vector (8): langchain · llama-index · haystack · chroma · faiss · weaviate · qdrant · pinecone Other (2): atlas · ibm-bob

    See the Feature Matrix for per-platform support details.

    Why it matters

    • ⚡ 99% faster — days of manual data prep → 15–45 minutes
    • 🎯 Real skill quality — 500+ line SKILL.md files with examples, patterns, and guides
    • 📊 RAG-ready chunks — smart chunking preserves code blocks and context
    • 🔄 Multi-source — combine docs + GitHub + PDFs + videos into one knowledge asset
    • 🌐 One prep, every target — export to 22 targets without re-scraping
    • ✅ Battle-tested — 3,900+ tests, 68 workflow presets, production-ready

    ✨ Key capabilities

    Three-layer discovery for JavaScript SPA sites (sitemap.xml → llms.txt → headless browser rendering), automatic llms.txt detection (10× faster when present), smart topic categorization, and a lenient HTML parser fallback so broken markup still scrapes.

    → Scraping Guide · llms.txt Support

    Three-stream architecture: code analysis (AST, design patterns, tests), documentation (README, docs/, wiki), and community (issues, PRs, metadata). The C3.x pipeline adds 10 GoF pattern detectors across 9 languages, usage examples extracted from tests, AI-written how-to guides, config extraction, and architecture overviews.

    skill-seekers create ./my-project --preset quick          # 1–2 min, surface level
    skill-seekers create ./my-project --preset standard       # balanced (default)
    skill-seekers create ./my-project --preset comprehensive  # deep, exhaustive
    

    → Pattern Detection · How-To Guides · Test Example Extraction

    Every AI call runs through one transport, in API mode (Anthropic, Google Gemini, OpenAI, Moonshot/Kimi, MiniMax) or LOCAL mode (Claude Code, Kimi Code, Codex, Copilot, OpenCode, custom agents — no API costs). Control depth with --enhance-level 0-3 and pick an agent with --agent.

    → Enhancement Guide · Enhancement Modes · Multi-Agent Setup

    One config can pull documentation, GitHub, PDFs, videos, and more into a single knowledge asset, with conflict detection and pairwise synthesis across sources.

    → Unified Scraping

    skill-seekers create <source> --index adds a stdlib-only scripts/search.py and a SQLite FTS5 index over the generated references, so an agent can find the right file#anchor before reading a large Markdown file wholesale. Off by default; the Markdown is untouched.

    → Skill Search Index

    YouTube, Vimeo, and local files. Three-tier transcript fallback (subtitles → YouTube transcript API → local Whisper), plus optional visual extraction that OCRs on-screen code from sampled frames.

    → Video Guide

    Quality scoring with a gate (skill-seekers quality output/react/ --threshold 7) or JSON stdout (skill-seekers quality output/react/ --json), provisional English readability metrics (informational — they never affect the score), doc-change detection with scheduled re-scrapes and notifications, streaming ingestion for very large doc sets, and incremental updates.

    → Large Documentation · Code Quality


    🔌 MCP Integration (40 tools)

    Skill Seekers ships an MCP server for Claude Code, Cursor, Windsurf, VS Code + Cline, and IntelliJ IDEA.

    # stdio mode (Claude Code, VS Code + Cline)
    python -m skill_seekers.mcp.server_fastmcp
    
    # HTTP mode (Cursor, Windsurf, IntelliJ)
    python -m skill_seekers.mcp.server_fastmcp --transport http --port 8765
    

    Then just ask your assistant: "Package and upload the React skill."

    → MCP Setup · MCP Reference · HTTP Transport


    🤖 Installing to AI agents

    Skills install automatically into 19 AI coding agents:

    skill-seekers install-agent output/react/ --agent cursor
    skill-seekers install-agent output/react/ --agent all      # every detected agent
    skill-seekers install-agent output/react/ --agent cursor --dry-run
    
    AgentPathScope
    Claude Code~/.claude/skills/Global
    Cursor.cursor/skills/Project
    VS Code / Copilot.github/skills/Project
    Amp~/.amp/skills/Global
    Goose~/.config/goose/skills/Global
    OpenCode~/.opencode/skills/Global
    Letta~/.letta/skills/Global
    Aide~/.aide/skills/Global
    Windsurf~/.windsurf/skills/Global
    Neovate~/.neovate/skills/Global
    Roo Code.roo/skills/Project
    Cline.cline/skills/Project
    Aider~/.aider/skills/Global
    Bolt.bolt/skills/Project
    Kilo Code.kilo/skills/Project
    Continue~/.continue/skills/Global
    Kimi Code~/.kimi/skills/Global
    IBM Bob.bob/skills/Project

    Uploading to Claude

    export ANTHROPIC_API_KEY=sk-ant-...
    skill-seekers package output/react/ --upload   # package + upload
    skill-seekers upload output/react.zip          # upload an existing zip
    

    No API key? Package it and upload output/react.zip manually at claude.ai/skills.

    → Upload Guide


    ⚙️ How it works

    graph LR
        A[Documentation Website] --> B[Skill Seekers]
        B --> C[Scraper]
        B --> D[AI Enhancement]
        B --> E[Packager]
        C --> F[Organized References]
        D --> F
        F --> E
        E --> G[AI Skill .zip]
        G --> H[Upload to AI Platform]
    
    1. Scrape — extract every page (checking llms.txt first)
    2. Categorize — organize content into topics (API, guides, tutorials, …)
    3. Enhance — AI writes a comprehensive SKILL.md with examples
    4. Package — bundle into a platform-ready artifact
    5. Upload — ship it to your AI platform (optional)

    Architecture

    8 core modules + 5 utility modules (~200 classes):

    ModulePurpose
    CLICoreGit-style command dispatcher, source auto-detection
    Scrapers18 source-type extractors on a shared build layer
    Adaptors22 output platform formats behind one SkillAdaptor ABC
    AnalysisC3.x codebase pipeline, 10 GoF pattern detectors
    EnhancementAI improvement via a single AgentClient transport
    PackagingPackage, upload, and install skills
    MCPFastMCP server (40 tools, 10 tool modules)
    SyncDoc change detection and notification

    → UML Architecture · API Reference · Skill Architecture


    🆕 New in v3.9.0

    • HTML parser fallback for broken markup (#96) — severely malformed pages no longer scrape as empty; well-formed pages are byte-identical.
    • Transient-failure retries — the doc scraper (#97) and MCP fetch_config (#92) now retry connection blips and 5xx with backoff; 4xx still fails fast.
    • Whisper transcription fallback (#420) — local videos without subtitles finally get a real transcript.
    • MiniMax image OCR + registry-driven multimodal providers (#423) — providers declare their wire protocol and image capability; China-issued keys work against the right endpoint.
    • Token-lean GitHub issue defaults (#169) — GitHub skills no longer bundle full closed-issue history by default.
    • Env-driven CORS across all three servers (#422, #424) — no more wildcard origins with credentials.

    Full history: CHANGELOG.md


    📈 Performance

    Documentation sizeTimeOutput
    Small (< 100 pages)5–10 min~2 MB
    Medium (100–500 pages)15–30 min~10 MB
    Large (500–2,000 pages)30–60 min~40 MB
    Huge (10K–40K+ pages)Use streamSee Large Documentation

    🐛 Troubleshooting

    skill-seekers doctor          # diagnose installation & environment
    skill-seekers doctor --json   # machine-readable diagnostics for CI and agents
    skill-seekers sync-config     # detect config drift
    

    Common issues and fixes: Troubleshooting Guide · TROUBLESHOOTING.md


    🤝 Contributing

    Contributions are welcome — see CONTRIBUTING.md.


    📝 License

    MIT — see LICENSE.

    🔒 Security

    MseeP.ai Security Assessment Badge


    🌐 Ecosystem

    Skill Seekers is a multi-repo project:

    RepositoryDescriptionLinks
    Skill_SeekersCore CLI & MCP server (this repo)PyPI
    skillseekerswebWebsite & documentationLive
    skill-seekers-configsCommunity config repository
    skill-seekers-actionGitHub Action for CI/CD
    skill-seekers-pluginClaude Code plugin
    homebrew-skill-seekersHomebrew tap for macOS

    Want to contribute? The website and configs repos are great starting points for new contributors!