pdf-craft

PDF Craft v2.6.6

PDF Craft 2.6.6 fixes concurrent PDF extraction, footnote OCR masks, XML fill whitespace recovery, and chapter consistency after refinement. Downstream content now comes from one finalized extraction result.

Migration from 2.6.5

• Extraction PAGE progress now uses stages 0–7 (max_stage=7). Update numeric labels: the previous OCR-persisted/JEV/LLM/final stages 2/3/4/5 become 4/5/6/7. First OCR, second-pass image preparation and second OCR have separate entry events. Async PDF pre-rendering remains in stage 0. See the stage table. #552, #553.
• Extraction on_body / on_footnote callbacks now deliver finalized, validated content once, after optional refinement and chapter assembly. The first content callback arrives later; use on_extraction_event for live progress. Callback delivery is not a task-success signal. Metadata may still arrive earlier.
• Derived TOC, chapter and furniture files are rebuilt from adopted content; existing toc.xml is no longer trusted as input. Valid raw OCR caches remain reusable. Re-extraction may assign different chapter IDs; existing PCEX files retain their format and identities. See streaming and cache migration.

Fixes

• Fix PCEX translation end-of-input races: title-dependent furniture waits for its producer, and body/footnote callbacks drain the final content exactly once in channel order. Failed or cancelled input scans are not successful EOF. Queue emptiness is checked after observing successful production completion.

• Serialize live page consumption across concurrent preparation/completion events, fixing duplicate pages and KeyError. Isolate blocking analysis and rendering workers from the short-IO pool, coordinate cancellation and callback draining, and settle concurrent work-unit completion only once. #560.

• Finalize traditional/refined content before deriving chapter IDs, TOC and furniture. Removing or merging a heading no longer leaves stale directory anchors; furniture uses final ownership. Formal content callbacks match adopted chapters. Private disk chunks bound intermediate aggregation without restoring pre-refinement directory constraints. #561.

• Preserve the complete union of first-pass OCR redaction masks. A lower, narrower text box no longer leaves the right side of a wider box exposed to the second footnote pass, preventing duplicate recognition from that leak. #554.

• Share document-whitespace semantics across friendly XML, segmentation and fill recovery. Preserve inline word boundaries and special spaces; reconstruct unambiguous canonical whitespace instead of spending another LLM request. Protected contexts retain their semantics; ambiguous allocation and changed words or punctuation still require correction. See LLM acceptance. #556, #558.

• Allocate merged canonical formulas through immutable math references, retain required empty source slots, and keep boundary spaces outside formula content that will be replaced. Accepted separators survive source-formula restoration. #557, #559.

• Preserve mixed-content text when serializing ordinary and streamed PCEX chapters; XML indentation no longer inserts separators between adjacent formulas. #555.

PCEX v4 and the progress event object structure are unchanged. The PAGE stage numbers and extraction content-callb

PDF Craft v2.6.5

PDF Craft 2.6.5 adds translation-size previews and operation-scoped resource reports, so applications can inspect source words before execution and observed remote consumption after completion, failure or cancellation.

• New pcex_words_count and epub_words_count read-only queries share the actual translation pre-scan and source-word rules. Matching options produce the same total as translation's STARTED.total_words. Both Craft facades expose methods; the async facade keeps parsing outside the event loop.
• New UsageCollector, UsageReport, Usage and RequestUsage group observed requests and input/output/cache-hit input tokens by original provider object. Pass usage= to model-calling operations and inspect usage.report() without changing the operation's existing return type.
• Accepted and discarded consumption are separated as effective and wasted, including transport retries and business validation loops. Missing usage is not estimated. Provider prompt cache is distinguished from ordinary input; local cache hits do not create external consumption.
• Support LLM, JEV and OCR vendors. OCR counts recognition submissions rather than pages, authentication calls or polling. Existing OCRTokensMetering is retained.
• Classify translation and fill consumption for withdrawn PCEX metadata collections as wasted, including otherwise valid members of an incomplete subjects list. Independent accepted fields retain their effective consumption. Collections are translated separately from scalar fields so their usage can be settled accurately.
• Adopt doc-page-extractor 1.2.2 and clip second-pass vendor OCR boxes against the actual redaction masks. This prevents later-stage detections from extending into previously recognized regions, including oversized footer boxes that blocked PDF translation writeback. See #545.
• Read PCEX queue readiness and results from one SQLite snapshot, avoiding a race where online rendering or callbacks could read an unfinished result during scanning.
• See usage and sizing or 中文说明 for examples, the accounting boundaries, and limitations. PCEX v4 and progress events are unchanged.

Resource reporting and task sizing: #550.

Full Changelog: https://github.com/oomol-lab/pdf-craft/compare/v2.6.4...v2.6.5

PDF Craft v2.6.4

PDF Craft 2.6.4 unifies extraction and translation progress, counts source words consistently, and starts independent metadata and PCEX translation work as soon as its inputs are ready. This release includes breaking progress API changes; callback consumers should follow the migration notes below. PCEX v4 is unchanged.

Unified work-unit progress

• Extraction and translation now report ProgressEvent with only STARTED, PROGRESS, and COMPLETED. An operation starts with required per-kind unit_counts; translation also supplies pre-scanned source total_words.
• Work units receive operation-local integer IDs only when execution begins. Each reports integer stage / max_stage values and a final UnitOutcome. Stages describe an individual unit, not a serial phase of the whole document.
• Standard XML translation reports after each accepted or final-fallback round, including heading, metadata, furniture and chapter work. Retries do not count work twice. Allowed ignore, fallback, skip and cache paths settle their units and words; operation completion follows successful output publication.
• EPUB pre-scans the same source tasks used for translation without buffering the book. Empty XML tasks settle as SKIPPED; assets are counted independently of request batches. EPUB navigation and document titles share NAVIGATION, while OPF book fields belong to METADATA.
• See #546, #547, and the progress protocol and stage numbers. Fixes the progress-reporting gaps discussed in #544.

Shared source-word counting

• All translation accounting uses one streaming implementation, exported as WordCounter, count_text_words, and MAX_WORD_LENGTH.
• Each Han character counts as one word. Other consecutive Unicode letters and numbers count as ceil(length / 20) words; combining marks extend a word without increasing its length. Whitespace and punctuation separate words.
• XML inline tags and streaming chunks do not split words; paragraph and field boundaries do. Original context prompts and reused headings are not counted as additional translation work. Counts use source text, not output or tokens.

Dependency-aware concurrency

• OCR-based book metadata starts from committed front-page evidence while later OCR continues. Additional reads wait only for requested pages, up to twelve. Native-only metadata is reported before OCR and saved consistently regardless of whether content or progress callbacks are registered.
• PCEX metadata, body, footnotes, image/table text and furniture share one bounded window and the existing translator executor. Internal group windows do not multiply that capacity; window=1 remains serial.
• Furniture waiting for a heading occupies no execution slot. A final heading result releases its dependants immediately, including fallback. Assets wait only for necessary original-text context, never for translated body text.
• Pending work and out-of-order results live on temporary disk. Body and footnote callbacks each preserve source order independently of the other channel and ordered output writing. Closing a content stream early cancels remaining work without reporting false operation completion.
• Close progress SQLite connections explicitly and avoid waiting for replacement events on failed caches belonging to pages not selected for this operation.
• See #548 and PCEX scheduling and delivery.

Breaking API migration

• Replace root OCREvent / OCREventKind and TranslationEvent / TranslationEventKind / TranslationItemKind imports with the unified ProgressEvent, ProgressEventKind, OperationKind, `UnitKin

PDF Craft v2.6.3

PDF Craft 2.6.3 preserves the original cause of concurrent service failures, coordinates temporary rate limits across request pools, and replaces automatic PDF input gating with optional input assertions. It also fixes XML fill text handling and PDF-to-PCEX directory/chapter consistency during refinement and recovery. Operation-scoped request headers now propagate across model-service calls.

Operation-scoped request headers

• Model-calling operations accept keyword-only request_headers, including synchronous/asynchronous Craft facades and advanced components. Headers reach LLM, DeepSeek OCR/OCR2, Unlimited OCR token/submission/polling, and JEV calls, including retries and nested translation/fill recovery.
• Each operation snapshots its headers without mutating shared clients or model configurations. Concurrent operations remain isolated; nested operations inherit headers by default, an explicit mapping replaces them, and {} clears them.
• Headers are not attached to resource downloads or caller-owned image resolvers. Authentication, routing and body-framing headers remain client-managed.
• Pass headers to the operation, not the LLM or OCR configuration constructor. Applications migrating from experimental instance-level headers support must move their mapping to request_headers on each operation.
• See #542 and the request-header API.

Concurrent failures

• The first recorded fatal error remains the cause of the operation failing, even when sibling cancellation is collected first or cleanup also fails.
• OCR HTTP 402 and HTTP 429 quota failures retain their original HTTP status and exception chain. Ordinary user cancellation still propagates as cancellation.
• The same protection applies across OCR, JEV, translation/fill and their surrounding pipelines. See #535.

Shared rate-limit recovery

• New optional RateLimitScope() instances coordinate executors using the same provider quota. Executors without an explicit scope receive their own scope.
• Temporary limits trigger shared cooldown, one probe request, and gradual, spaced recovery. Waiting releases request capacity and supports cancellation.
• Defaults are cooldown_seconds=1.0 and max_wait_seconds=600.0. The latter is a separate cumulative waiting budget for each logical request. Exhaustion raises RateLimitTimeoutError; existing total request timeouts still apply.
• LLM, JEV and OCR vendor temporary limits no longer consume ordinary retry counts, including when those counts are zero. Existing retry parameters remain available for non-rate failures and provider-specific recovery.
• Numeric and HTTP-date Retry-After values are supported, along with structured LLM rate-limit errors received after a streaming HTTP 200 response starts. Payment/quota failures remain fatal, rather than triggering cooldown.
• See #537 and the shared rate-limit API.

Optional input checks and migration

• Conversion no longer runs mandatory whole-file PDF preflight. Failures surface at the actual operation, retaining supported page-level recovery.
• Applications can explicitly call assert_pdf_readable, assert_pdf_renderable, assert_pdf_editable or assert_epub_readable. These blocking checks return None or raise the original backend/parser error; they do not repair inputs or guarantee a complete conversion.
• Breaking change: PDFPreflightError is removed from pdf_craft and pdf_craft.error, together with the automatic gate. Remove its imports and catch clauses; optional assertions propagate original backend/parser errors.
• Existing conversion entry points and LLM/OCR configuration parameters remain available. See [#536](