v1.0.0
What's Changed
- ✨ feat: a safe-string for HTML, a markupsafe drop-in by @gaborbernat in #28
- ✨ feat: linkify() — URL/email auto-linking, a bleach successor by @gaborbernat in #29
- ✨ feat: an HTML sanitizer (clean()), faster than nh3, a bleach successor by @gaborbernat in #30
- 🐛 fix: detection and URL-scheme bugs found running competitors' test suites by @gaborbernat in #31
- ⚡ perf: faster serialize and data-state scan by @gaborbernat in #73
- 🐛 fix(linkify): keep underscore in host labels by @gaborbernat in #100
- 🐛 fix(treebuilder): keep foreign inside its root by @gaborbernat in #101
- 🐛 fix(serialize): emit frame as a void element by @gaborbernat in #102
- 🐛 fix(treebuilder): ignore stray end tag before by @gaborbernat in #103
- 🐛 fix(sanitizer): drop URL attr when a duplicate scheme is unsafe by @gaborbernat in #104
- 🐛 fix(tokenizer): drop duplicate attribute names per WHATWG by @gaborbernat in #105
- 🐛 fix(treebuilder): synthesize head/body for html fragments by @gaborbernat in #108
- 🐛 fix(treebuilder): restore table mode after fostered rawtext by @gaborbernat in #110
- 🐛 fix(treebuilder): adjust MathML definitionURL attribute name by @gaborbernat in #107
- 🐛 fix(serialize): walk trees iteratively to avoid stack overflow by @gaborbernat in #106
- 🐛 fix(parse): keep head mode on redundant html/duplicate head by @gaborbernat in #111
- 🐛 fix(treebuilder): honor foreign scope in adoption agency by @gaborbernat in #112
- 🐛 fix(tokenizer): report nameless DOCTYPE name as None by @gaborbernat in #115
- 🐛 fix(treebuilder): case-adjust SVG attribute names by @gaborbernat in #113
- 🐛 fix(tree): guard descendants walk against a detached cursor by @gaborbernat in #117
- 🐛 fix(tokenizer): reclaim consumed input in streaming feed() by @gaborbernat in #114
- 🐛 fix(treebuilder): ignore stray DOCTYPE after initial mode by @gaborbernat in #118
- 🐛 fix(encoding): complete the WHATWG charset-label table by @gaborbernat in #121
- 🐛 fix(tree): pickle elements with parser-only tag names by @gaborbernat in #119
- 🐛 fix(treebuilder): process stray in colgroup via in-body by @gaborbernat in #123
- 🐛 fix(select): decode CSS escape sequences in identifiers by @gaborbernat in #122
- 🐛 fix(tokenizer): reject Tokenizer constructor arguments by @gaborbernat in #124
- 🐛 fix(linkify): end a URL at Unicode whitespace by @gaborbernat in #120
- 🐛 fix(treebuilder): keep NUL-adjacent table whitespace inside by @gaborbernat in #126
- 🐛 fix(treebuilder): foster out of a table-section fragment by @gaborbernat in #135
- 🐛 fix: keep a trailing asterisk in a linkified URL by @gaborbernat in #134
- 🐛 fix: report missing DOCTYPE identifier as None on the node by @gaborbernat in #133
- 🐛 fix: decode gbk label with gb18030 decoder by @gaborbernat in #132
- 🐛 fix: raise TypeError for non-mapping Element attrs by @gaborbernat in #131
- 🐛 fix: report exact-case WHATWG name for legacy encodings by @gaborbernat in #129
- 🐛 fix: accept slash separator after meta in encoding prescan by @gaborbernat in #128
- 🐛 fix: require str for parse_fragment context and Token.attr name by @gaborbernat in #127
- 🐛 fix: declare _reconstruct pickle hook in type stub by @gaborbernat in #130
- 🐛 fix(select): apply HTML case-insensitive attribute set by @gaborbernat in #125
- 🐛 fix: keep foreign element namespace across a pickle round-trip by @gaborbernat in #139
- 🐛 fix: match custom-element type selector case-insensitively by @gaborbernat in #137
- 🐛 fix(encoding): cap the meta charset prescan at 1024 bytes by @gaborbernat in #136
- 🐛 fix(selectors): accept namespace-qualified type selectors by @gaborbernat in #141
- 🐛 fix: require exact annotation-xml encoding for integration point by @gaborbernat in #142
- 🐛 fix(treebuilder): keep foreign
inside its root by @gaborbernat in #143 - 🐛 fix: keep raw-text serialization across construct and pickle by @gaborbernat in #145
- 🐛 fix: put plain xmlns on a foreign element in the xmlns namespace by @gaborbernat in #140
- 🐛 fix: pop open optgroup/option in a select-context fragment by @gaborbernat in #147
- 🐛 fix(sanitizer): always neutralize plaintext by @gaborbernat in #146
- 🐛 fix(encoding): add the WHATWG replacement decoder by @gaborbernat in #148
- 🐛 fix(sanitizer): keep allowlisted SVG/MathML elements by @gaborbernat in #144
- 🐛 fix: keep after-head whitespace under html, not in body by @gaborbernat in #150
- ✨ feat(select): tree-structural pseudo-classes by @gaborbernat in #151
- 🐛 fix: report a valueless attribute as the empty string by @gaborbernat in #153
- 🐛 fix: pop open select for a keygen start tag in select mode by @gaborbernat in #154
- 🐛 fix: pop open option before hr in a select-context fragment by @gaborbernat in #149
- 🐛 fix(tokenizer): lock shared Tokenizer for free-threaded safety by @gaborbernat in #152
- 🐛 fix(parse): template fragment enters in-template mode by @gaborbernat in #155
- 🐛 fix(treebuilder): pop select for a table tag in select-in-table by @gaborbernat in #156
- 🐛 fix: lock the shared node tree for free-threaded safety by @gaborbernat in #157
- 🧪 test(freethread): stress, GIL guard, and ThreadSanitizer by @gaborbernat in #159
- ✨ feat(select): :is(), :where(), and :has() pseudo-classes by @gaborbernat in #158
- ⚡ perf(select): index by subject atom and cache compiled selectors by @gaborbernat in #162
- 🧪 test(treebuilder): lock foreign /
containment by @gaborbernat in #184 - 🐛 fix(select): fold class/ID case in quirks mode by @gaborbernat in #185
- ✨ feat(select): support the :not() negation pseudo-class by @gaborbernat in #188
- 🐛 fix(select): match whitespace-only elements with :empty by @gaborbernat in #187
- 🐛 fix(parser): treat empty DOCTYPE system id as quirks by @gaborbernat in #190
- ✨ feat(xpath): native XPath 1.0 engine by @gaborbernat in #186
- ✨ feat(select): support :scope, UI/form, :lang() and :dir() pseudo-classes by @gaborbernat in #193
- 🐛 fix(select): make :is()/:where() forgiving selector lists by @gaborbernat in #195
- ✨ feat(select): support :nth-child(An+B of S) selector lists by @gaborbernat in #198
- ✨ feat(serializer): round-trip-safe HTML minify mode by @gaborbernat in #196
- ✨ feat(compat): html.parser-compatible callback adapter by @gaborbernat in #199
- ✨ feat(xpath): complete lxml/parsel call-option parity by @gaborbernat in #194
- ✨ feat(serialize): to_markdown and to_text exporters with full markdownify/html2text/inscriptis parity by @gaborbernat in #191
- 📝 docs(migration): note the encoding-detection boundary by @gaborbernat in #202
- 📝 docs: compare and migrate resiliparse, note gumbo by @gaborbernat in #205
- 📝 docs: compare and migrate lxml-html-clean & html-sanitizer by @gaborbernat in #203
- 📝 docs: compare and migrate w3lib HTML utilities by @gaborbernat in #204
- ✨ feat(query): pyquery-style fluent chaining query API by @gaborbernat in #200
- 📝 docs(migration): add a From parsel guide by @gaborbernat in #201
- 📝 docs: migration guides for html5-parser, pyquery & html.parser by @gaborbernat in #207
- 👷 bench: compare the html.parser adapter to the standard library by @gaborbernat in #208
- 📊 docs(performance): show parsel in the Querying table by @gaborbernat in #209
- 🔒 fix(sanitizer): scheme-check ping and srcset URL lists by @gaborbernat in #228
- ✨ feat(sanitizer): add Policy.set_attributes by @gaborbernat in #230
- ✨ feat(sanitizer): add Policy.remove_with_content by @gaborbernat in #229
- ✨ feat(export): annotation spans and Google-Docs Markdown mode by @gaborbernat in #232
- ✨ feat(sanitizer): scrub the style attribute against a CSS property allowlist by @gaborbernat in #231
- feat(tokenizer): char-ref tokens and token source by @gaborbernat in #240
- feat(parser): streaming IncrementalParser (push parse to a tree) by @gaborbernat in #241
- ✨ feat(tree): add form-field value helpers by @gaborbernat in #243
- feat(serialize): attribute order and meta-charset options by @gaborbernat in #238
- feat(markdown): per-tag converter hook for to_markdown by @gaborbernat in #239
- feat(linkify): add plain-text link detector by @gaborbernat in #236
- ✨ feat(parser): collect and report parse errors by @gaborbernat in #242
- feat(linkify): reprocess anchors and customize detection by @gaborbernat in #237
- ✨ feat(export): add to_markdown output modes by @gaborbernat in #235
- ✨ feat(url): add Document.base_url and meta_refresh by @gaborbernat in #233
- ✨ feat(parser): source line and column on parsed elements by @gaborbernat in #245
- ✨ feat(links): enumerate and resolve document links by @gaborbernat in #244
- docs: note migration limitations per section by @gaborbernat in #234
- ✨ feat(encoding): content-based encoding detection (chardetng-style) by @gaborbernat in #206
- ✨ feat(tree): bulk wrap_children and wrap_siblings by @gaborbernat in #254
- feat(markdown): add to_markdown strip/convert filters by @gaborbernat in #255
- feat(node): readability main-content extraction by @gaborbernat in #257
- ✨ feat(tree): add Node.prune to keep a CSS selector by @gaborbernat in #258
- feat(tree): parsel-style string extraction by @gaborbernat in #256
- ✨ feat(annotation): surface-form and inline-tag exporters by @gaborbernat in #253
- fix(types): drop duplicate Node.re stub by @gaborbernat in #259
- refactor(layout): subsystem-first source + test tree by @gaborbernat in #260
- docs(changelog): rewrite entries as concise end-user release notes by @gaborbernat in #261
- 📚 docs: reorganize the site and type the full reference by @gaborbernat in #262
- ✨ feat(tree): add classList editing methods by @gaborbernat in #284
- ✨ feat(query): extract JSON-LD, Microdata, and OpenGraph by @gaborbernat in #291
- ✨ feat(build): add E element builder for HTML generation by @gaborbernat in #289
- ✨ feat(query): find nodes by text content by @gaborbernat in #285
- ✨ feat(tree): set inner HTML, text, and adjacent HTML by @gaborbernat in #286
- ✨ feat(tree): bulk remove and strip_tags by selector by @gaborbernat in #283
- 📊 test(build): benchmark E builder and consolidate tests by @gaborbernat in #294
- ✨ feat(xpath): add precompiled expression object by @gaborbernat in #281
- ✨ feat(tree): read tables as rows and records by @gaborbernat in #290
- ✨ feat(xpath): return node-sets from extension functions by @gaborbernat in #279
- docs(migration): add html-text and MechanicalSoup by @gaborbernat in #293
- ✨ feat(xpath): add EXSLT set, str, math, and date functions by @gaborbernat in #288
- ✨ feat(xpath): bind namespace prefixes by @gaborbernat in #282
- feat(export): article metadata on main_content by @gaborbernat in #292
- ✨ feat(xpath): bind node-set variables by @gaborbernat in #278
- ✨ feat(query): css_path and xpath_path for a node by @gaborbernat in #287
- 📚 docs(migration): order guides by monthly downloads by @gaborbernat in #295
- 📊 docs(migration): benchmark every operational migration feature by @gaborbernat in #296
- 📖 docs: add design principles to the landing page by @gaborbernat in #297
- 📊 docs(performance): add speed-up multipliers to all benchmark tables by @gaborbernat in #299
- 📖 docs: tighten and reorganize the README by @gaborbernat in #298
- 📚 docs(migration): unify all guides — monthly badges, perf inside Why turbohtml, speed-up multipliers by @gaborbernat in #300
- 📝 docs(migration): report monthly downloads not totals by @gaborbernat in #301
- 🔧 build(pre-commit): ruff-format and lint RST doc snippets by @gaborbernat in #302
- 📝 docs(migration): per-library builder + metadata guides by @gaborbernat in #303
- 🔧 build(c): clear -Wall/-Wextra warnings and gate them in CI by @gaborbernat in #304
- ✨ feat(bench): isolate competitors per-venv and migrate every operation by @gaborbernat in #305
- ⚡ perf(tokenizer): speed long DATA text scans by @gaborbernat in #309
- ⚡ perf(dom): pool node wrappers on a freelist by @gaborbernat in #308
- 🐛 fix(bench): measure mutating ops on fresh trees by @gaborbernat in #310
- 📝 docs(migration): unify the migration guide layout by @gaborbernat in #311
- 🐛 fix(bench): query a page the selectors match by @gaborbernat in #312
- ✨ feat(bench): run size-pinned ops across the corpus ladder by @gaborbernat in #327
- ♻️ refactor: config objects for over-stuffed method signatures by @gaborbernat in #328
- ⚡ perf(find): match literal text= filters in C by @gaborbernat in #325
- ⚡ perf(css): fast-path type combinators by @gaborbernat in #326
- ⚡ perf(path): build node locators in linear time by @gaborbernat in #329
- ♻️ refactor(linkify): config object for linkify and Linker by @gaborbernat in #330
- ♻️ refactor(api): clean + extract topic namespaces by @gaborbernat in #331
- ✨ feat(clean): HTML minify and From minify-html guide by @gaborbernat in #341
- ✨ feat(clean): add a value-safe CSS minifier by @gaborbernat in #346
- 📊 docs: benchmark tables from committed JSON feeds, tinted and grouped by @gaborbernat in #348
- ✨ feat(serialize): native JavaScript minifier by @gaborbernat in #344
- ✨ feat(match): soupsieve-style CSS matching surface by @gaborbernat in #354
- ✨ feat(clean): expose HTML minify (minify-html/htmlmin) by @gaborbernat in #353
- 📝 docs(build): htbuilder migration coverage by @gaborbernat in #356
- ✨ feat(detect): standalone encoding detection (chardet/charset-normalizer) by @gaborbernat in #361
- 📝 docs(build): simple-html migration coverage by @gaborbernat in #357
- 📝 docs(build): hyperpython migration coverage by @gaborbernat in #358
- ✨ feat(convert): css_to_xpath translator (cssselect) by @gaborbernat in #355
- 📝 docs(build): markyp migration coverage by @gaborbernat in #359
- ✨ feat(extract): per-paragraph boilerplate classification (justext) by @gaborbernat in #362
- 📝 docs(build): fast-html migration coverage by @gaborbernat in #360
- 📝 docs(migration): boilerpy3 migration coverage by @gaborbernat in #364
- ✨ feat(extract): URL clean/normalize/extract helpers (courlan) by @gaborbernat in #363
- 📝 docs(migration): goose3 migration coverage by @gaborbernat in #365
- ✨ feat(extract): w3lib URL surface via UrlCleaning query filters by @gaborbernat in #366
- 📝 docs(migration): readabilipy migration coverage by @gaborbernat in #367
- 📝 docs(migration): news-please migration coverage by @gaborbernat in #368
- ✨ feat(extract): microdata get_items shim (microdata) by @gaborbernat in #369
- ✨ feat(extract): opengraph card shim (opengraph) by @gaborbernat in #370
- ✨ feat(extract): publication-date extraction (htmldate) by @gaborbernat in #371
- 🐛 fix(selectors): decode quoted-string escapes and fix XPath round() ties by @gaborbernat in #376
- 👷 ci: add CodSpeed continuous benchmarking by @gaborbernat in #380
- 👷 ci: pin PYTHONHASHSEED in the CodSpeed run by @gaborbernat in #382
- 👷 ci: stabilize CodSpeed noise floor via glibc hwcaps by @gaborbernat in #383
- 🐛 fix(selectors): four Selectors-4 conformance gaps by @gaborbernat in #381
- 🐛 fix(query): XPath memory-safety and conformance by @gaborbernat in #440
- 🐛 fix(serialize): JavaScript minifier miscompiles by @gaborbernat in #441
- 🐛 fix(parser): tree-construction whitespace handling by @gaborbernat in #442
- 🐛 fix(extract): content-extraction scorer correctness by @gaborbernat in #446
- 🐛 fix(clean): CSS minifier value-safety by @gaborbernat in #443
- 🐛 fix(convert): to_markdown and to_text correctness by @gaborbernat in #445
- 🐛 fix(detect): character-encoding conformance by @gaborbernat in #448
- 🐛 fix(clean): sanitizer conformance and error types by @gaborbernat in #447
- 🐛 fix(build): table-model conformance by @gaborbernat in #450
- 🐛 fix(extract): microdata extraction conformance by @gaborbernat in #449
- 🐛 fix(extract): URL normalization conformance by @gaborbernat in #452
- 🐛 fix(clean): linkify boundary conformance by @gaborbernat in #451
- 🐛 fix(clean): minify optional-end-tag omission by @gaborbernat in #453
- 🐛 fix(build): form-associated element conformance by @gaborbernat in #444
- 🐛 fix(query): CSS Selectors Level 4 conformance by @gaborbernat in #454
- 🐛 fix(build): reject invalid attribute names and void-element children by @gaborbernat in #456
- 👷 ci(bench): stabilize noise-prone CodSpeed benchmarks on real corpora by @gaborbernat in #455
- 📝 docs(migration): standardize guides on a head-to-head format by @gaborbernat in #457
- ✨ feat(clean): sanitizer attribute-prefix, value-allowlist, and host-allowlist policy by @gaborbernat in #461
- 🧹 refactor(api): actionable errors, correct types, :raises: docstrings by @gaborbernat in #458
- ✨ feat(clean): minify CSS in <style> and style attributes by @gaborbernat in #460
- ✨ feat(serialize): preserve license/bang comments in minify_js by @gaborbernat in #462
- ✨ feat(extract): resolve relative URLs in metadata against a base_url by @gaborbernat in #463
- ✨ feat(detect): report UTF-8-SIG and BOM-based UTF-16/32 labels by @gaborbernat in #464
- ✨ feat(clean): minify boolean-attribute collapse and optional end-tag omission by @gaborbernat in #465
- ✨ feat(serialize): lenient never-fail mode for minify_js by @gaborbernat in #466
- ✨ feat(query): Node.equals for structural tree comparison by @gaborbernat in #468
- ✨ feat(extract): registrable-domain boundary for link filtering by @gaborbernat in #467
- ✨ feat(clean): sanitize <style> element contents against the CSS policy by @gaborbernat in #469
- ✨ feat(build): full-document page-shell helper by @gaborbernat in #471
- ✨ feat(extract): RDFa and Dublin Core structured-data extraction by @gaborbernat in #472
- ✨ feat: python -m turbohtml command-line interface by @gaborbernat in #470
- ✨ feat(serialize): streaming/incremental serialization by @gaborbernat in #473
- ✨ feat(detect): content-based language detection by @gaborbernat in #474
- ✨ feat(extract): canonical, site_name, tags, and lead-image metadata by @gaborbernat in #476
- ⚡ perf(extract): skip the RDFa walk when a document has no typeof by @gaborbernat in #475
- ⚡ perf(extract): fuse article metadata into one head walk by @gaborbernat in #477
- ♻️ refactor(api): unify the 1.0 public surface by @gaborbernat in #479
- 📝 docs: restructure for 1.0 discoverability and taxonomy by @gaborbernat in #480
- ♻️ refactor(c): house style, decompose run_drain by @gaborbernat in #482
- ⚡ perf(build): profile-guided optimization for the release build by @gaborbernat in #481
- ✨ feat(api): OpenGraph and CSSMinify records complete the 1.0 surface by @gaborbernat in #483
- 🔒 build(data): pin the PSL and TLD table sources by @gaborbernat in #484
- ♻️ refactor(c): enable LTO, split hot/cold headers, move query out of element.c by @gaborbernat in #485
- ⚡ perf(url): port URL splitting to C by @gaborbernat in #486
- ⚡ perf(core): find() atom index for tag+attr, presize streaming chunks by @gaborbernat in #487
- ⚡ perf(url): port percent-encoding to C by @gaborbernat in #488
- ⚡ perf(build): cover every hot op in the PGO training workload by @gaborbernat in #489
- ⚡ perf(url): port relative-reference resolution to C by @gaborbernat in #490
- 🔧 build(ci): measure a deterministic LTO-only build on CodSpeed by @gaborbernat in #491
- ⚡ perf(build): train PGO on a representative real-world corpus by @gaborbernat in #492
- ⚡ perf(url): port IDNA (punycode + UTS-46) host encoding to C by @gaborbernat in #493
- ♻️ refactor(url): consolidate registrable-domain into the url module by @gaborbernat in #494
- 🔒 fix(url): guard registrable-domain against non-ASCII host OOB read by @gaborbernat in #495
- ⚡ perf(extract): port date parsing to C by @gaborbernat in #496
- 📝 docs: add a how-this-was-built transparency note by @gaborbernat in #497
- 🔧 build(bench): add --pgo flag to build the release-representative baseline by @gaborbernat in #498
- ♻️ refactor(url): route the sanitizer allowlist through the unified parser by @gaborbernat in #499
- 🔒 docs(security): real security policy and pinned-table provenance by @gaborbernat in #500
- 🔒 fix(url): harden IDNA punycode decoder against overflow and OOB by @gaborbernat in #501
- 🔒 test(clean): run DOMPurify's XSS corpus against the sanitizer by @gaborbernat in #502
- 🔒 test(fuzz): ASan/UBSan fuzz harnesses for the untrusted-input entry points by @gaborbernat in #503
- 🔒 fix(parse): cap tree nesting depth to prevent stack overflow (DoS) by @gaborbernat in #505
- 🔒 test(parse): full html5lib-tests differential conformance oracle by @gaborbernat in #504
- 🔒 fix(clean): per-node namespace-reachability check (mXSS defense-in-depth) by @gaborbernat in #507
- 📝 docs(security): document single-pass sanitization by @gaborbernat in #506
- 🔒 fix(tokenize): O(n) attribute-name deduplication to close a parse DoS by @gaborbernat in #508
- ⚡ perf(query): linear single-pass :has() evaluation by @gaborbernat in #509
- 🔒 build(data): SHA-256 provenance for the IDNA table generator by @gaborbernat in #510
- ♻️ refactor(c): overflow-safe shared vec.h buffer growth by @gaborbernat in #511
- ♻️ refactor(c): shared ascii.h and reconcile whitespace/case drift by @gaborbernat in #512
- ♻️ refactor(tests): mirror src layout + quality by @gaborbernat in #513
- 📝 docs(project): present shipped code as turbohtml's own by @gaborbernat in #514
- ♻️ refactor(project): fold match into the query namespace by @gaborbernat in #515
- 📄 chore: fold NOTICE into LICENSE by @gaborbernat in #516
- 🧹 chore: whole-tree simp + no-slop sweep by @gaborbernat in #517
- 📝 docs(changelog): collapse 0.4.0→1.0.0 into a 1.0 overview by @gaborbernat in #518
- 📦 perf(build): shrink the release wheel by @gaborbernat in #519
- 📊 docs(bench): final 1.0 benchmark numbers (PGO+LTO) by @gaborbernat in #520
- 🐛 fix(serialize): skip null memcpy on empty run by @gaborbernat in #521
- ✨ feat(bench): cover every op each library shares by @gaborbernat in #523
- 📊 feat(bench): measure minifier size, add industry tools, close coverage gaps by @gaborbernat in #524
- 📝 docs(changelog): fold the last fragment into the 1.0 overview by @gaborbernat in #525
New Contributors
- @dependabot[bot] made their first contribution in #192
Full Changelog: 0.4.0...1.0.0