All notable changes to PDFOxide are documented here.
Stable Recursion and Refined Table Heuristics
- Refined Table Detection — The spatial table detector now requires at least 2 columns to identify a region as a table. This significantly reduces false positives where single-column lists or bullet points were incorrectly wrapped in ASCII boxes.
- Optimized Text Extraction — Refactored the internal extraction pipeline to eliminate redundant work when processing Tagged PDFs. The structure tree and page spans are now extracted once and shared across the detection and rendering phases.
- Resolved
RefCellalready borrowed panic (#237) — Fixed a critical reentrancy issue where recursive Form XObject processing (e.g., extracting images from nested forms) could trigger a runtime panic. Replaced long-lived borrows with scoped, tiered cache access using Rust best practices. (Reported by @marph91)
🥇 @marph91 — Thank you for identifying the complex RefCell borrow conflict in nested image extraction (#237). This report led to a comprehensive safety audit of our interior mutability patterns and a more robust, recursion-safe caching architecture! 🚀
Advanced Visual Table Detection and Automated Python Stubs
- Smart Hybrid Table Extraction (#206) — Introduced a robust, zero-config visual detection engine that handles both bordered and borderless tables.
- Localized Grid Detection: Uses Union-Find clustering to group vector paths into discrete table regions, enabling multiple tables per page.
- Visual Line Analysis: Detects cell boundaries from actual drawing primitives (lines and rectangles), significantly improving accuracy for untagged PDFs.
- Visual Spans: Identifies colspans and rowspans by analyzing the absence of internal grid lines and text-overflow signals.
- Visual Headers: Heuristically identifies hierarchical (multi-row) header rows.
- Professional ASCII Tables: Added high-quality ASCII table formatting for plain text output, featuring automatic multiline text wrapping and balanced column alignment.
- Auto-generated Python type stubs (#220) — Integrated automated
.pyistub generation using mypy's stubgen in the CI pipeline, ensuring Python IDEs always have up-to-date type information for the Rust bindings. - Python
PdfDocumentpath-like and context manager (#223) —PdfDocumentnow acceptspathlib.Path(or any path-like object) and supports the context manager protocol (with PdfDocument(path) as doc:), ensuring scoped usage and automatic resource cleanup. - Enabled by Default: Table extraction is now active by default in all Markdown, HTML, and Plain Text conversions.
- Robust Geometry: Updated
Rectprimitive to handle negative dimensions and coordinate normalization natively.
- Fixed segfault in nested Form XObject text extraction (#228) — Resolved aliased
&mutreferences during recursive XObject processing using interior mutability (RefCell/Cell). - Fixed Python Coordinate Scaling: Corrected
erase_regioncoordinate mapping in Python bindings to use the standard[x1, y1, x2, y2]format. - Improved ASCII Table Wrapping: Reworked text wrapping to be UTF-8 safe, preventing panics on multi-byte characters.
- Refined Rendering API: Restored backward compatibility for the
render_pagemethod.
🥇 @hoesler — Huge thanks for PR #228! Your fix for the nested XObject aliasing UB is a critical stability improvement that eliminates segfaults in complex PDFs. By correctly employing interior mutability, you've made the core extraction engine significantly more robust and spec-compliant. Outstanding work! 🚀
🥈 @monchin — Thank you for the fantastic initiative on automated stub generation (#220) and the ergonomic improvements for Python (#223)! We've integrated these into the v0.3.16 release, providing consistent, IDE-friendly type hints and modern path-like/context manager support. Outstanding contributions! 🚀
Header & Footer Management, Multi-Column Stability, and Font Fixes
- PDF Header/Footer Management API (#207) — Added a dedicated API for managing page artifacts across Rust, Python, and WASM.
- Add: Ability to insert custom headers and footers with styling and placeholders via
PageTemplate. - Remove: Heuristic detection engine to automatically identify and strip repeating artifacts. Includes modular methods:
remove_headers(),remove_footers(), andremove_artifacts(). Prioritizes ISO 32000 spec-compliant/Artifacttags when available. - Edit: Ability to mask or erase existing content on a per-page basis via
erase_header(),erase_footer(), anderase_artifacts().
- Add: Ability to insert custom headers and footers with styling and placeholders via
- Page Templates — Introduced
PageTemplate,Artifact, andArtifactStyleclasses for reusable page design. Supports dynamic placeholders like{page},{pages},{title}, and{author}. - Scoped Extraction Filtering — Updated all extraction methods to respect
erase_regions, enabling clean text extraction by excluding identified headers and footers. - Python
PdfDocument.from_bytes()— Open PDFs directly from in-memory bytes without requiring a file path. (Contributed by @hoesler in #216) - Future-Proofed Rust API — Implemented
Defaulttrait for key extraction structs (TextSpan,TextChar,TextContent) to protect users from future field additions.
- Fixed Multi-Column Reading Order (#211) — Refactored
extract_words()andextract_text_lines()to use XY-Cut partitioning. This prevents text from adjacent columns from being interleaved and standardizes top-to-bottom extraction. (Reported by @ankursri494) - Resolved Font Identity Collisions (#213) — Improved font identity hashing to include
ToUnicodeandDescendantFontsreferences. Fixes garbled text extraction in documents where multiple fonts share the same name but use different character mappings. (Reported by @productdevbook) - Fixed
Linestable strategy false positives (#215) —extract_tables()withhorizontal_strategy="lines"now builds the grid purely from vector path geometry and returns empty when no lines are found, preventing spurious tables on plain-text pages. (Contributed by @hoesler) - Optimized CMap Parsing — Standardized 2-byte consumption for Identity-H fonts and improved robust decoding for Turkish and other extended character sets.
🥇 @hoesler — Huge thanks for PR #216 and #215! Your contribution of from_bytes() for Python unlocks new serverless and in-memory workflows for the entire community. Additionally, your fix for the Lines table strategy significantly improves the precision of our table extraction engine. Outstanding work! 🚀
🥈 @ankursri494 (Ankur Srivastava) — Thank you for identifying the multi-column reading order issue (#211). Your detailed report and sample document were the catalyst for our new XY-Cut partitioning engine, which makes PDFOxide's reading order detection among the best in the ecosystem! 🎯
🥉 @productdevbook — Thanks for reporting the complex font identity collision issue (#213). This report led to a deep dive into PDF font internals and a significantly more robust font hashing system that fixes garbled text for thousands of professional documents! 🔍✨
Parity in API & Bug Fixing (Issue #185, #193, #202)
- High-Level Rendering API (#185, #190) — added
Pdf::render_page()to Rust, Python, and WASM. Supports rendering any page toImage(Png/Jpeg). Restored backward compatibility for Rust by maintaining the 1-argumentrender_pageand addingrender_page_with_options. - Word and Line Extraction (#185, #189) — added
extract_words()andextract_text_lines()to all bindings. Provides semantic grouping of characters with bounding boxes, font info, and styling (parity withpdfplumber). - Geometric Primitive Extraction (#185, #191) — added
extract_rects()andextract_lines()to identify vector graphics. - Hybrid Table Detection (#185, #192) — updated
SpatialTableDetectorto use vector lines as hints, significantly improving detection of "bordered" tables. - API Harmonization — implemented the fluent
.within(page, rect)pattern across Rust, Python, and WASM for scoped extraction. - Area Filtering — added optional
regionsupport to all extraction methods (extract_text,extract_chars, etc.) in Python and WASM, using backward-compatible signatures. - Deep Data Access — added
.charsproperty toTextWordandTextLineobjects in Python, enabling granular access to individual character metadata. - CLI Enhancements — added
pdf-oxide renderfor image generation andpdf-oxide pathsfor geometric JSON extraction. Integrated--areafiltering across all extraction commands.
Reported by @MarcRene71 — AttributeError: 'builtins.PdfDocument' object has no attribute 'extract_text_ocr' when using the library without the OCR feature enabled.
- Improved Feature Gating Discovery (#204) — ensured that all optional features (OCR, Office, Rendering) are always visible in the Python API. If a feature is disabled at build time, calling its methods now returns a helpful
RuntimeErrorexplaining how to enable it (e.g.,pip install pdf_oxide[ocr]), instead of throwing anAttributeError. - Always-on Type Stubs (#204) — updated
.pyifiles to include all methods regardless of build features, providing full IDE autocompletion support for all capabilities.
Reported by @cole-dda — repeated calls to extract_texts() and extract_spans() return inconsistent results (empty lists on second/third calls).
- Fixed XObject span cache poisoning (#193) — resolved an issue where
extract_chars()(low-level API) would incorrectly populate the high-levelxobject_spans_cachewith empty results. Becauseextract_chars()does not collect spans, it was "poisoning" the cache for subsequentextract_spans()calls, causing them to return empty data for any content inside Form XObjects. - Improved extraction mode isolation (#193) — ensured that the text extractor explicitly separates character and span extraction paths. The span result cache is now only accessed and updated when in span extraction mode, and internal span buffers are cleared when entering character mode.
Reported by @vincenzopalazzo — extract_text() returns empty string for encrypted PDFs with CID TrueType Identity-H fonts.
- Support for V=4 Crypt Filters (#202) — fixed a bug in
EncryptDictwhere version 4 encryption was hardcoded to AES-128. It now correctly parses the/CFdictionary and/CFMentry to select between RC4-128 (/V2) and AES-128 (/AESV2), enabling support for PDFs produced by OpenPDF. - Encrypted CIDToGIDMap decryption (#202) — fixed a missing decryption step when loading
CIDToGIDMapstreams. Previously, the stream was decompressed but remained encrypted, causing invalid glyph mapping and failed text extraction. - Enhanced font diagnostic logging (#202) — replaced silent failures with descriptive warnings when ToUnicode CMaps or FontFile2 streams fail to load or decrypt, making it easier to diagnose complex extraction issues.
- Consolidated text decoding and positioning logic (#187) — unified the high-level
extract_text_spans()and low-levelextract_chars()paths into a single shared engine to prevent logic drift and ensure consistent character handling. - Fixed render_page for in-memory PDFs — ensured that PDFs created from bytes or strings can be rendered by automatically initializing a temporary editor if needed.
- Improved Clustering Accuracy — updated character clustering to use gap-based distance instead of center-to-center distance, ensuring accurate word grouping regardless of font size.
Thank you to @MarcRene71 for identifying the critical API discoverability issue with OCR (#204). Your report led to a more robust "Pythonic" approach to feature gating, ensuring that users always see the full API and receive helpful guidance when features are disabled!
Thank you to @vincenzopalazzo for identifying and fixing the critical issues with encrypted CID fonts and V=4 crypt filters (#202). Your contribution of both the fix and the reproduction fixture was essential for ensuring PDFOxide handles professional PDFs from diverse producers!
Thank you to @ankursri494 (Ankur Srivastava) for the excellent proposal to bridge the gap between PdfPlumber's flexibility and PDFOxide's performance (#185). Your detailed breakdown of word-level and table extraction requirements was the roadmap for this release!
Thank you to @cole-dda for identifying the critical caching bug (#193). The detailed reproduction case was essential for pinpointing the interaction between the low-level character API and the document-level XObject caches.
Character Extraction Quality, Multi-byte Encoding (Issue #186)
Reported by @cole-dda — garbled output when using extract_chars() on PDFs with multi-byte encodings (CJK text, Type0 fonts).
- Multi-byte decoding in show_text — fixed
extract_chars()to correctly handle 2-byte and variable-width encodings (Identity-H/V, Shift-JIS, etc.). Previously, characters were processed byte-by-byte, causing multi-byte characters to be split and garbled. Now uses the same robust decoding logic asextract_spans(). - Improved character positioning accuracy — replaced the 0.5em fixed-width estimate in
show_textwith actual glyph widths from the font dictionary. This ensures that character bounding boxes (bbox) and origins are precisely positioned, matching the actual PDF rendering. - Accurate character advancement — character spacing (
Tc) and word spacing (Tw) are now correctly scaled by horizontal scaling (Th) during character-level extraction, ensuring correct text matrix updates.
Thank you to @cole-dda for identifying and reporting the character extraction quality issue with an excellent reproduction case (#186). Your report directly led to identifying the divergence between our high-level and low-level extraction paths, making extract_chars() significantly more robust for CJK and other multi-byte documents. We really appreciate your contribution to making PDF Oxide better!
Text Extraction Quality, Determinism, Performance, Markdown Conversion
Reported by @Goldziher — systematic evaluation across 10 PDFs covering word merging, encoding failures, and RTL text.
-
CID font width calculation — fixed text-to-user space conversion for CID fonts. Glyph widths were not correctly scaled, causing word boundary detection to merge adjacent words (
destinationmachine→destination machine,helporganizeas→help organize as). -
Font-change word boundary detection — when PDF font changes mid-line (e.g., regular→italic for product names in LaTeX), we now detect this as a word boundary even if the visual gap is small. Previously, these were merged into single words with mixed formatting.
-
Non-Standard CID mapping fallback — implemented a fallback mechanism for CID fonts with broken
/ToUnicodemaps. If mapping fails, we now attempt to use the font's internalcmaptable directly. Fixed encoding failures in 3 PDFs from the corpus. -
RTL text directionality foundation — added basic support for identifying RTL (Right-to-Left) script spans (Arabic, Hebrew) based on Unicode range. Provides correctly ordered spans for simple RTL layouts.
- Optimized Markdown engine — significantly improved the performance of
to_markdown()by implementing recursive spatial partitioning (XY-Cut). This ensures that multi-column layouts and complex document structures are converted into accurate, readable Markdown. - Heading Detection — automated identification of headers (H1-H6) based on font size variance and document-wide frequency analysis.
- List Reconstruction — detects bulleted and numbered lists by analyzing leading character patterns and indentation consistency.
- Zero-copy page tree traversal — refactored internal page navigation to avoid redundant dictionary cloning during deep page tree traversal for multi-page extraction.
- Structure tree caching — Structure tree result cached after first access, avoiding redundant parsing on every
extract_text()call (major impact on tagged PDFs like PDF32000_2008.pdf). - BT operator early-out —
extract_spans(),extract_spans_with_config(), andextract_chars()skip the full text extraction pipeline for image-only pages that contain noBT(Begin Text) operators. - Larger I/O buffer for big files —
BufReadercapacity increased from 8 KB to 256 KB for files >100 MB, reducing syscall overhead on 1.5 GB newspaper archives. - Xref reconstruction threshold removed — Eliminated the
xref.len() < 5heuristic that triggered full-file reconstruction on valid portfolio PDFs with few objects (5-13s → <100ms).
Thank you to @Goldziher for the exhaustive evaluation of PDF extraction quality (#181). Your systematic approach to testing across 10 diverse documents directly resulted in critical fixes for font scaling and encoding fallbacks. The feedback from power users like you is what drives PDF Oxide's quality forward!
Stability, Image Extraction & Error Recovery (Issue #41, #44, #45, #46)
- 100% pass rate on 3,830 PDFs across three independent test suites: veraPDF (2,907), Mozilla pdf.js (897), SafeDocs (26).
- Zero timeouts, zero panics — every PDF completes within 120 seconds.
- p50 = 0.6ms, p90 = 3.0ms, p99 = 33ms — 97.6% of PDFs complete in under 10ms.
- Added
verify_corpusexample binary for reproducible batch verification with CSV output, timeout handling, and per-corpus breakdown.
- Owner password authentication (Algorithm 7 for R≤4, Algorithm 12 for R≥5).
- R≤4: Derives RC4 key from owner password via MD5 hash chain, decrypts
/Ovalue to recover user password, then validates via user password authentication. - R≥5: SHA-256 verification with SASLprep normalization and owner validation/key salts per PDF spec §7.6.3.4.
- Both algorithms now fully wired into
EncryptionHandler::authenticate().
- R≤4: Derives RC4 key from owner password via MD5 hash chain, decrypts
- R≥5 user password verification with SASLprep — Full AES-256 password verification using SHA-256 with validation and key salts per PDF spec §7.6.4.3.3.
- Public password authentication API —
Pdf::authenticate(password)andPdfDocument::authenticate(password)exposed for user-facing password entry.
- XMP metadata validation — Parses XMP metadata stream and checks for
pdfaid:partandpdfaid:conformanceidentification entries (clause 6.7.11). - Color space validation — Scans page content streams for device-dependent color operators (
rg,RG,k,K,g,G) without output intent (clause 6.2). - AFRelationship validation — For PDF/A-3 documents with embedded files, validates each file specification dictionary contains the required
AFRelationshipkey (clause 6.8).
- XMP PDF/X identification — Parses XMP metadata for
pdfxid:GTS_PDFXVersion, validates against declared level (clause 6.7.2). - Page box relationship validation — Validates TrimBox ⊆ BleedBox ⊆ MediaBox and ArtBox ⊆ MediaBox with 0.01pt tolerance (clause 6.1.1).
- ExtGState transparency detection — Checks
SMask(not/None),CA/ca< 1.0, andBMnotNormal/Compatiblein extended graphics state dictionaries (clause 6.3). - Device-dependent color detection — Flags DeviceRGB/CMYK/Gray color spaces used without output intent (clause 6.2.3).
- ICC profile validation — Validates ICCBased color space profile streams contain required
/Nentry (clause 6.2.3).
- Spec-correct clipping (PDF §8.5.4) — Clip state scoped to
q/Qsave/restore via clip stack; new clips intersect with existing clip region;W/W*no longer consume the current path (deferred to next paint operator); clip mask applied to all painting operations including text and images. - Glyph advance width calculation — Text position advances per PDF spec §9.4.4:
tx = (w0/1000 × Tfs + Tc + Tw) × Thwith 600-unit default glyph width. - Form XObject rendering — Parses
/Matrixtransform, uses form's/Resources(or inherits from parent), and recursively executes form content stream operators.
- Missing objects resolve to Null — Per PDF spec §7.3.10, unresolvable indirect references now return
Nullinstead of errors, fixing 16 files across veraPDF/pdf.js corpora. - Lenient header version parsing — Fixed fast-path bug where valid headers with unusual version strings were rejected.
- Non-standard encryption algorithm matching — V=1,R=3 combinations now handled leniently instead of rejected.
- Non-dictionary Resources — Pages with invalid
/Resourcesentries (e.g., Null, Integer) treated as empty resources instead of erroring. - Null nodes in page tree — Null or non-dictionary child nodes in page tree gracefully skipped during traversal.
- Corrupt content streams — Malformed content streams return empty content instead of propagating parse errors.
- Enhanced page tree scanning —
/Resources+/Parentheuristic and/Kidsdirect resolution added as fallback passes for damaged page trees.
- Bogus /Count bounds checking — Page count validated against PDF spec Annex C.2 limit (8,388,607) and total object count; unreasonable values fall back to tree scanning.
- Content stream image extraction —
extract_images()now processes page content streams to findDooperator calls, extracting images referenced via XObjects that were previously missed. - Nested Form XObject images — Recursive extraction with cycle detection handles images inside Form XObjects.
- Inline images —
BI...ID...EIsequences parsed with abbreviation expansion per PDF spec. - CTM transformations — Image bounding boxes correctly transformed using full 4-corner affine transform (handles rotation, shear, and negative scaling).
- ColorSpace indirect references — Resolved indirect references (e.g.,
7 0 R) in image color space entries before extraction.
- Multi-line object headers — Parser now handles
1 0\nobjformat used by Google-generated PDFs instead of requiring1 0 objon a single line. - Extended header search — Header search window extended from 1024 to 8192 bytes to handle PDFs with large binary prefixes.
- Lenient version parsing — Malformed version strings like
%PDF-1.aor truncated headers no longer cause parse failures in lenient mode.
- Missing Contents entry — Pages without a
/Contentskey now return empty content data instead of erroring. - Cyclic page tree detection — Page tree traversal tracks visited nodes to prevent stack overflow on malformed circular references.
- Null stream references — Null or invalid stream references handled gracefully instead of panicking.
- Wider page scanning fallback — Page scanning fallback triggers on more error conditions, improving compatibility with damaged PDFs.
- Pages without /Type entry — Page scanning now finds pages missing the
/Type /Pageentry by checking for/MediaBoxor/Contentskeys.
- Short encryption key panic — AES decryption with undersized keys now returns an error instead of panicking.
- Xref stream parsing hardened — Malformed xref streams with invalid entry sizes or out-of-bounds data no longer cause panics.
- Indirect /Encrypt references —
/Encryptdictionary values that are indirect references are now resolved before parsing.
- Dictionary-as-Stream fallback — When a stream object is a bare dictionary (no stream data), it is now treated as an empty stream instead of causing a decode error.
- Filter abbreviations — Abbreviated filter names (
AHx,A85,LZW,Fl,RL,CCF,DCT) and case-insensitive matching now supported. - Operator limit — Content stream parsing enforces a configurable operator limit (default 1,000,000) to prevent pathological slowdowns on malformed streams.
- Structure tree indirect object references —
ObjectRefvariants in structure tree/Kentries are now resolved at parse time instead of being silently skipped, ensuring complete structure tree traversal. - Lexer
Rtoken disambiguation —tag(b"R")no longer matches theRprefix ofRG/ri/reoperators;1 0 RGis now correctly parsed as a color operator instead of indirect reference1 0 R+ orphanG. - Stream whitespace trimming —
trim_leading_stream_whitespacenow only strips CR/LF (0x0D/0x0A), no longer strips NUL bytes (0x00) or spaces from binary stream data (fixes grayscale image extraction and object stream parsing).
- 8 previously ignored tests un-ignored and fixed:
test_extract_raw_grayscale_image_from_xobject— Fixed stream trimming stripping binary pixel data.test_parse_object_stream_with_whitespace— Fixed stream trimming affecting object stream offsets.test_parse_object_stream_graceful_failure— Relaxed assertion for improved parser recovery.test_markdown_reading_order_top_to_bottom— Fixed test coordinates to use PDF convention (Y increases upward).test_html_layout_multiple_elements— Fixed assertions for per-character positioning.test_reading_order_graph_based_simple— Fixed test coordinates to PDF convention.test_reading_order_two_columns— Fixed test coordinates to PDF convention.test_parse_color_operators— Fixed lexer R/RG token disambiguation.
- Deleted empty
PdfImagestub (src/images.rs) and its module export — image extraction usesImageInfofromsrc/extractors/images.rs. - Deleted commented-out
DocumentType::detect()test block insrc/extractors/gap_statistics.rs. - Removed stale TODO comments in
scripts/setup-hooks.sh,src/bin/analyze_pdf_features.rs,src/document.rs.
🥇 @SeanPedersen — Huge thanks for reporting multiple issues (#41, #44, #45, #46) that drove the entire stability focus of this release. His real-world testing uncovered a parser bug with Google-generated PDFs, image extraction failures on content stream references, and performance problems — each report triggering deep investigation and significant fixes. The parser robustness, image extraction, and testing infrastructure improvements in v0.3.5 all trace back to Sean's thorough bug reports. 🙏🔍
Parsing Robustness, Character Extraction & XObject Paths
parse_header()function signature - Now includes offset tracking.- Before:
parse_header(reader) -> Result<(u8, u8)> - After:
parse_header(reader, lenient) -> Result<(u8, u8, u64)> - Migration: Replace
let (major, minor) = parse_header(&mut reader)?;withlet (major, minor, _offset) = parse_header(&mut reader, true)?; - Note: This is a public API function; consider using
doc.version()for typical use cases instead.
- Before:
- Header offset support - PDFs with binary prefixes or BOM headers now open successfully.
- Parse header function now searches first 1024 bytes for
%PDF-marker (PDF spec compliant). - Supports UTF-8 BOM, email headers, and other leading binary data.
parse_header()returns byte offset where header was found.- Lenient mode (default) handles real-world malformed PDFs; strict mode for compliance testing.
- Fixes parsing errors like "expected '%PDF-', found '1b965'".
- Parse header function now searches first 1024 bytes for
extract_chars()API - Low-level character-level extraction for layout analysis.- Returns
Vec<TextChar>with per-character positioning, font, and styling data. - Includes transformation matrix, rotation angle, advance width.
- Sorted in reading order (top-to-bottom, left-to-right).
- Overlapping characters (rendered multiple times) deduplicated.
- 30-50% faster than span extraction for character-only use cases.
- Exposed in both Rust and Python APIs.
- Python binding:
doc.extract_chars(page_index)returns list ofTextCharobjects.
- Returns
- Form XObject support in path extraction - Now extracts vectors from embedded XObjects.
extract_paths()recursively processes Form XObjects viaDooperator.- Image XObjects properly skipped (only Form XObjects extracted).
- Coordinate transformations via
/Matrixproperly applied. - Graphics state properly isolated (save/restore).
- Duplicate XObject detection prevents infinite loops.
- Nested XObjects (XObject containing XObject) supported.
- Dependencies: Upgraded nom parser library from 7.1 to 8.0.
- Updated all parser combinators to use
.parse()method. - No user-facing API changes.
- All parser functionality maintained.
- Performance stable (no regressions detected).
- Updated all parser combinators to use
parse_header()signature updated: now returns(major, minor, offset)tuple.- All parse_header test cases updated to use new signature.
Form Fields, Multimedia & Python 3.8-3.14
- Parent/Child Field Structures - Create complex form hierarchies like
address.street,address.city.add_parent_field()- Create container fields without widgets.add_child_field()- Add child fields to existing parents.add_form_field_hierarchical()- Auto-create parent hierarchy from dotted names.ParentFieldConfigfor configuring container fields.- Property inheritance between parent and child fields (FT, V, DV, Ff, DA, Q).
- Edit All Field Properties - Beyond just values.
set_form_field_readonly()/set_form_field_required()- Flag manipulation.set_form_field_rect()- Reposition/resize fields.set_form_field_tooltip()- Set hover text (TU).set_form_field_max_length()- Text field length limits.set_form_field_alignment()- Text alignment (left/center/right).set_form_field_default_value()- Default values (DV).BorderStyleandAppearanceCharacteristicssupport.
- Critical Bug Fix - Modified existing fields now persist on save (was only saving new fields).
- Forms Data Format Export - ISO 32000-1:2008 Section 12.7.7.
FdfWriter- Binary FDF export for form data exchange.XfdfWriter- XML XFDF export for web integration.export_form_data_fdf()/export_form_data_xfdf()on FormExtractor, DocumentEditor, Pdf.- Hierarchical field representation in exports.
- TextChar Transformation - Per-character positioning metadata (#27).
origin- Font baseline coordinates (x, y).rotation_degrees- Character rotation angle.matrix- Full transformation matrix.- Essential for pdfium-render migration.
- DPI Calculation - Resolution metadata for images.
horizontal_dpi/vertical_dpifields onImageContent.resolution()- Get (h_dpi, v_dpi) tuple.is_high_resolution()/is_low_resolution()/is_medium_resolution()helpers.calculate_dpi()- Compute from pixel dimensions and bbox.
- Spatial Filtering - Extract text from rectangular regions.
RectFilterMode::Intersects- Any overlap (default).RectFilterMode::FullyContained- Completely within bounds.RectFilterMode::MinOverlap(f32)- Minimum overlap fraction.TextSpanSpatialtrait -intersects_rect(),contained_in_rect(),overlap_with_rect().TextSpanFilteringtrait -filter_by_rect(),extract_text_in_rect().
- MovieAnnotation - Embedded video content.
- SoundAnnotation - Audio content with playback controls.
- ScreenAnnotation - Media renditions (video/audio players).
- RichMediaAnnotation - Flash/video rich media content.
- ThreeDAnnotation - 3D model embedding.
- U3D and PRC format support.
ThreeDView- Camera angles and lighting.ThreeDAnimation- Playback controls.
- PathExtractor - Vector graphics extraction.
- Lines, curves, rectangles, complex paths.
- Path transformation and bounding box calculation.
- XfaExtractor - Extract XFA form data.
- XfaParser - Parse XFA XML templates.
- XfaConverter - Convert XFA forms to AcroForm.
- True Python 3.8-3.14 Support - Fixed via
abi3-py38(was only working on 3.11). - Modern Tooling - uv, pdm, ruff integration.
- Code Quality - All Python code formatted with ruff.
🥇 @monchin - Massive thanks for revolutionizing our Python ecosystem! Your PR #29 fixed a critical compatibility issue where PDFOxide only worked on Python 3.11 despite claiming 3.8+ support. By switching to abi3-py38, you enabled true cross-version compatibility (Python 3.8-3.14). The introduction of modern tooling (uv, pdm, ruff) brings PDFOxide's Python development to 2026 standards. This work directly enables thousands more Python developers to use PDFOxide. 💪🐍
🥈 @bikallem - Thanks for the thoughtful feature request (#27) comparing PDFOxide to pdfium-render. Your detailed analysis of missing origin coordinates and rotation angles led directly to our TextChar transformation feature. This makes PDFOxide a viable migration path for pdfium-render users. 🎯
Unified API, PDF Creation & Editing
- One API for Extract, Create, and Edit - The new
Pdfclass unifies all PDF operations.Pdf::open("input.pdf")- Open existing PDF for reading and editing.Pdf::from_markdown(content)- Create new PDF from Markdown.Pdf::from_html(content)- Create new PDF from HTML.Pdf::from_text(content)- Create new PDF from plain text.Pdf::from_image(path)- Create PDF from image file.- DOM-like page navigation with
pdf.page(0)for querying and modifying content. - Seamless save with
pdf.save("output.pdf")orpdf.save_encrypted().
- Fluent Builder Pattern -
PdfBuilderfor advanced configuration.PdfBuilder::new() .title("My Document") .author("Author Name") .page_size(PageSize::A4) .from_markdown("# Content")?
- PDF Creation API - Fluent
DocumentBuilderfor programmatic PDF generation.Pdf::create()/DocumentBuilder::new()entry points.- Page sizing (Letter, A4, custom dimensions).
- Text rendering with Base14 fonts and styling.
- Image embedding (JPEG/PNG) with positioning.
- Table Rendering -
TableRendererfor styled tables.- Headers, borders, cell spans, alternating row colors.
- Column width control (fixed, percentage, auto).
- Cell alignment and padding.
- Graphics API - Advanced visual effects.
- Colors (RGB, CMYK, grayscale).
- Linear and radial gradients.
- Tiling patterns with presets.
- Blend modes and transparency (ExtGState).
- Page Templates - Reusable page elements.
- Headers and footers with placeholders.
- Page numbering formats.
- Watermarks (text-based).
- Barcode Generation (requires
barcodesfeature)- QR codes with configurable size and error correction.
- Code128, EAN-13, UPC-A, Code39, ITF barcodes.
- Customizable colors and dimensions.
- Editor API - DOM-like editing with round-trip preservation.
DocumentEditorfor modifying existing PDFs.- Content addition without breaking existing structure.
- Resource management for fonts and images.
- Annotation Support - Full read/write for all types.
- Text markup: highlights, underlines, strikeouts, squiggly.
- Notes: sticky notes, comments, popups.
- Shapes: rectangles, circles, lines, polygons, polylines.
- Drawing: ink/freehand annotations.
- Stamps: standard and custom stamps.
- Special: file attachments, redactions, carets.
- Form Fields - Interactive form creation.
- Text fields (single/multiline, password, comb).
- Checkboxes with custom appearance.
- Radio button groups.
- Dropdown and list boxes.
- Push buttons with actions.
- Form flattening (convert fields to static content).
- Link Annotations - Navigation support.
- External URLs.
- Internal page navigation.
- Styled link appearance.
- Outline Builder - Bookmark/TOC creation.
- Hierarchical structure.
- Page destinations.
- Styling (bold, italic, colors).
- PDF Layers - Optional Content Groups (OCG).
- Create and manage content layers.
- Layer visibility controls.
- PDF/A Validation - ISO 19005 compliance checking.
- PDF/A-1a, PDF/A-1b (ISO 19005-1).
- PDF/A-2a, PDF/A-2b, PDF/A-2u (ISO 19005-2).
- PDF/A-3a, PDF/A-3b (ISO 19005-3).
- PDF/A Conversion - Convert documents to archival format.
- Automatic font embedding.
- XMP metadata injection.
- ICC color profile conversion.
- PDF/X Validation - ISO 15930 print production compliance.
- PDF/X-1a:2001, PDF/X-1a:2003.
- PDF/X-3:2002, PDF/X-3:2003.
- PDF/X-4, PDF/X-4p.
- PDF/X-5g, PDF/X-5n, PDF/X-5pg.
- PDF/X-6, PDF/X-6n, PDF/X-6p.
- 40+ specific error codes for violations.
- PDF/UA Validation - ISO 14289 accessibility compliance.
- Tagged PDF structure validation.
- Language specification checks.
- Alt text requirements.
- Heading hierarchy validation.
- Table header validation.
- Form field accessibility.
- Reading order verification.
- Encryption on Write - Password-protect PDFs when saving.
- AES-256 (V=5, R=6) - Modern 256-bit encryption (default).
- AES-128 (V=4, R=4) - Modern 128-bit encryption.
- RC4-128 (V=2, R=3) - Legacy 128-bit encryption.
- RC4-40 (V=1, R=2) - Legacy 40-bit encryption.
Pdf::save_encrypted()for simple password protection.Pdf::save_with_encryption()for full configuration.
- Permission Controls - Granular access restrictions.
- Print, copy, modify, annotate permissions.
- Form fill and accessibility extraction controls.
- Digital Signatures (foundation, requires
signaturesfeature)- ByteRange calculation for signature placeholders.
- PKCS#7/CMS signature structure support.
- X.509 certificate parsing.
- Signature verification framework.
- Page Labels - Custom page numbering.
- Roman numerals, letters, decimal formats.
- Prefix support (e.g., "A-1", "B-2").
PageLabelsBuilderfor creation.- Extract existing labels from documents.
- XMP Metadata - Extensible metadata support.
- Dublin Core properties (title, creator, description).
- PDF properties (producer, keywords) .
- Custom namespace support.
- Full read/write capability.
- Embedded Files - File attachments.
- Attach files to PDF documents.
- MIME type and description support.
- Relationship specification (Source, Data, etc.).
- Linearization - Web-optimized PDFs.
- Fast web view support.
- Streaming delivery optimization.
- Text Search - Pattern-based document search.
- Regex pattern support.
- Case-sensitive/insensitive options.
- Position tracking with page/coordinates.
- Whole word matching.
- Page Rendering (requires
renderingfeature)- Render pages to PNG/JPEG images.
- Configurable DPI and scale.
- Pure Rust via tiny-skia (no external dependencies).
- Debug Visualization (requires
renderingfeature)- Visualize text bounding boxes.
- Element highlighting for debugging.
- Export annotated page images.
- Office to PDF (requires
officefeature)- DOCX: Word documents with paragraphs, headings, lists, formatting.
- XLSX: Excel spreadsheets via calamine (sheets, cells, tables).
- PPTX: PowerPoint presentations (slides, titles, text boxes).
OfficeConverterwith auto-detection.OfficeConfigfor page size, margins, fonts.- Python bindings:
OfficeConverter.from_docx(),from_xlsx(),from_pptx().
Pdfclass for PDF creation.Color,BlendMode,ExtGStatefor graphics.LinearGradient,RadialGradientfor gradients.LineCap,LineJoin,PatternPresetsfor styling.save_encrypted()method with permission flags.OfficeConverterclass for Office document conversion.
- Description updated to "The Complete PDF Toolkit: extract, create, and edit PDFs".
- Python module docstring updated for v0.3.0 features.
- Branding updated with Extract/Create/Edit pillars.
- Outline action handling - correctly dereference actions indirectly referenced by outline items.
🥇 @jvantuyl - Thanks for the thorough PR #16 fixing outline action dereferencing! Your investigation uncovered that some PDFs embed actions directly while others use indirect references - a subtle PDF spec detail that was breaking bookmark navigation. Your fix included comprehensive tests ensuring this won't regress. 🔍✨
🙏 @mert-kurttutan - Thanks for the honest feedback in issue #15 about README clutter. Your perspective as a new user helped us realize we were overwhelming people with information. The resulting documentation cleanup makes PDFOxide more approachable. 📚
CJK Support & Structure Tree Enhancements
- TagSuspect/MarkInfo support (ISO 32000-1 Section 14.7.1).
- Parse MarkInfo dictionary from document catalog (
marked,suspects,user_properties). PdfDocument::mark_info()method to retrieve MarkInfo.- Automatic fallback to geometric ordering when structure tree is marked as suspect.
- Parse MarkInfo dictionary from document catalog (
- Word Break /WB structure element (Section 14.8.4.4).
- Support for explicit word boundaries in CJK text.
StructType::WBvariant andis_word_break()helper.- Word break markers emitted during structure tree traversal.
- Predefined CMap support for CJK fonts (Section 9.7.5.2).
- Adobe-GB1 (Simplified Chinese) - ~500 common character mappings.
- Adobe-Japan1 (Japanese) - Hiragana, Katakana, Kanji mappings.
- Adobe-CNS1 (Traditional Chinese) - Bopomofo and CJK mappings.
- Adobe-Korea1 (Korean) - Hangul and Hanja mappings.
- Fallback identity mapping for common Unicode ranges.
- Abbreviation expansion /E support (Section 14.9.5).
- Parse
/Eentry from marked content properties. expansionfield onStructElemfor structure-level abbreviations.
- Parse
- Object reference resolution utility.
PdfDocument::resolve_references()for recursive reference handling in complex PDF structures.
- Type 0 /W array parsing for CIDFont glyph widths.
- Proper spacing for CJK text using CIDFont width specifications.
- ActualText verification tests - comprehensive test coverage for PDF Spec Section 14.9.4.
- Soft hyphen handling (U+00AD) - now correctly treated as valid continuation hyphen for word reconstruction.
- Enhanced artifact filtering with subtype support.
ArtifactType::Paginationwith subtypes: Header, Footer, Watermark, PageNumber.ArtifactType::LayoutandArtifactType::Backgroundclassification.
OrderedContent.mcidchanged toOption<u32>to support word break markers.
Image Embedding & Export
- Image embedding: Both HTML and Markdown now support embedded base64 images when
embed_images=true(default).- HTML:
<img src="data:image/png;base64,..."> - Markdown:
(works in Obsidian, Typora, VS Code, Jupyter).
- HTML:
- Image file export: Set
embed_images=false+image_output_dirto save images as files with relative path references. - New
embed_imagesoption inConversionOptionsto control embedding behavior. PdfImage::to_base64_data_uri()method for converting images to data URIs.PdfImage::to_png_bytes()method for in-memory PNG encoding.- Python bindings: new
embed_imagesparameter forto_html,to_markdown, and*_allmethods.
CTM Fix & Formula Rendering
- CTM (Current Transformation Matrix) now correctly applied to text positions per PDF Spec ISO 32000-1:2008 Section 9.4.4 (#11).
- Structure tree:
/Alt(alternate description) parsing for accessibility text on formulas and figures. - Structure tree:
/Pg(page reference) resolution - correctly maps structure elements to page numbers. FormulaRenderermodule for extracting formula regions as base64 images from rendered pages.ConversionOptions: new fieldsrender_formulas,page_images,page_dimensionsfor formula image embedding.- Regression tests for CTM transformation.
🐛➡️✅ @mert-kurttutan - Thanks for the detailed bug report (#11) with reproducible sample PDF! Your report exposed a fundamental CTM transformation bug affecting text positioning across the entire library. This fix was critical for production use. 🎉
BT/ET Matrix Reset & Text Processing
- BT/ET matrix reset per PDF spec Section 9.4.1 (PR #10 by @drahnr).
- Geometric spacing detection in markdown converter (#5).
- Verbose extractor logs changed from info to trace (#7).
- docs.rs build failure (excluded tesseract-rs).
apply_intelligent_text_processing()method for ligature expansion, hyphenation reconstruction, and OCR cleanup (#6).
- Removed unused tesseract-rs dependency.
🥇 @drahnr - Huge thanks for PR #10 fixing the BT/ET matrix reset issue! This was a subtle PDF spec compliance bug (Section 9.4.1) where text matrices weren't being reset between text blocks, causing positions to accumulate and become unusable. Your fix restored correct text positioning for all PDFs. 💪📐
🔬 @JanIvarMoldekleiv - Thanks for the detailed bug report (#5) about missing spaces and lost table structure! Your analysis even identified the root cause in the code - the markdown converter wasn't using geometric spacing analysis. This level of investigation made the fix straightforward. 🕵️♂️
🎯 @Borderliner - Thanks for two important catches! Issue #6 revealed that apply_intelligent_text_processing() was documented but not actually available (oops! 😅), and #7 caught our overly verbose INFO-level logging flooding terminals. Both fixed immediately! 🔧
Discoverability Improvements
- Optimized crate keywords for better discoverability.
Encrypted PDF Fixes
- Encrypted stream decoding improvements (#3).
- CI/CD pipeline fixes.
🥇 @threebeanbags - Huge thanks for PRs #2 and #3 fixing encrypted PDF support! 🔐 Your first PR identified that decryption needed to happen before decompression - a critical ordering issue. Your follow-up PR #3 went deeper, fixing encryption handler initialization timing and adding Form XObject encryption support. These fixes made PDFOxide actually work with password-protected PDFs in production. 💪🎉
- Encrypted stream decoding (#2).
- Documentation and doctest fixes.
- Encrypted stream decoding refinements.
- Python 3.13 support.
- GitHub sponsor configuration.
- Cross-platform binary builds (Linux, macOS, Windows).
- Initial release.
- PDF text extraction with spec-compliant Unicode mapping.
- Intelligent reading order detection.
- Python bindings via PyO3.
- Support for encrypted PDFs.
- Form field extraction.
- Image extraction.
💖 @magnus-trent - Thanks for issue #1, our first community feedback! Your message that PDFOxide "unlocked an entire pipeline" you'd been working on for a month validated that we were solving real problems. Early encouragement like this keeps open source projects going. 🚀