Skip to content

v1.0.0

Choose a tag to compare

@gaborbernat gaborbernat released this 06 Jul 22:28
0c1279a

What's Changed

  • ✨ feat: a safe-string for HTML, a markupsafe drop-in by @gaborbernat in #28
  • ✨ feat: linkify() — URL/email auto-linking, a bleach successor by @gaborbernat in #29
  • ✨ feat: an HTML sanitizer (clean()), faster than nh3, a bleach successor by @gaborbernat in #30
  • 🐛 fix: detection and URL-scheme bugs found running competitors' test suites by @gaborbernat in #31
  • ⚡ perf: faster serialize and data-state scan by @gaborbernat in #73
  • 🐛 fix(linkify): keep underscore in host labels by @gaborbernat in #100
  • 🐛 fix(treebuilder): keep foreign

    inside its root by @gaborbernat in #101
  • 🐛 fix(serialize): emit frame as a void element by @gaborbernat in #102
  • 🐛 fix(treebuilder): ignore stray end tag before by @gaborbernat in #103
  • 🐛 fix(sanitizer): drop URL attr when a duplicate scheme is unsafe by @gaborbernat in #104
  • 🐛 fix(tokenizer): drop duplicate attribute names per WHATWG by @gaborbernat in #105
  • 🐛 fix(treebuilder): synthesize head/body for html fragments by @gaborbernat in #108
  • 🐛 fix(treebuilder): restore table mode after fostered rawtext by @gaborbernat in #110
  • 🐛 fix(treebuilder): adjust MathML definitionURL attribute name by @gaborbernat in #107
  • 🐛 fix(serialize): walk trees iteratively to avoid stack overflow by @gaborbernat in #106
  • 🐛 fix(parse): keep head mode on redundant html/duplicate head by @gaborbernat in #111
  • 🐛 fix(treebuilder): honor foreign scope in adoption agency by @gaborbernat in #112
  • 🐛 fix(tokenizer): report nameless DOCTYPE name as None by @gaborbernat in #115
  • 🐛 fix(treebuilder): case-adjust SVG attribute names by @gaborbernat in #113
  • 🐛 fix(tree): guard descendants walk against a detached cursor by @gaborbernat in #117
  • 🐛 fix(tokenizer): reclaim consumed input in streaming feed() by @gaborbernat in #114
  • 🐛 fix(treebuilder): ignore stray DOCTYPE after initial mode by @gaborbernat in #118
  • 🐛 fix(encoding): complete the WHATWG charset-label table by @gaborbernat in #121
  • 🐛 fix(tree): pickle elements with parser-only tag names by @gaborbernat in #119
  • 🐛 fix(treebuilder): process stray in colgroup via in-body by @gaborbernat in #123
  • 🐛 fix(select): decode CSS escape sequences in identifiers by @gaborbernat in #122
  • 🐛 fix(tokenizer): reject Tokenizer constructor arguments by @gaborbernat in #124
  • 🐛 fix(linkify): end a URL at Unicode whitespace by @gaborbernat in #120
  • 🐛 fix(treebuilder): keep NUL-adjacent table whitespace inside by @gaborbernat in #126
  • 🐛 fix(treebuilder): foster out of a table-section fragment by @gaborbernat in #135
  • 🐛 fix: keep a trailing asterisk in a linkified URL by @gaborbernat in #134
  • 🐛 fix: report missing DOCTYPE identifier as None on the node by @gaborbernat in #133
  • 🐛 fix: decode gbk label with gb18030 decoder by @gaborbernat in #132
  • 🐛 fix: raise TypeError for non-mapping Element attrs by @gaborbernat in #131
  • 🐛 fix: report exact-case WHATWG name for legacy encodings by @gaborbernat in #129
  • 🐛 fix: accept slash separator after meta in encoding prescan by @gaborbernat in #128
  • 🐛 fix: require str for parse_fragment context and Token.attr name by @gaborbernat in #127
  • 🐛 fix: declare _reconstruct pickle hook in type stub by @gaborbernat in #130
  • 🐛 fix(select): apply HTML case-insensitive attribute set by @gaborbernat in #125
  • 🐛 fix: keep foreign element namespace across a pickle round-trip by @gaborbernat in #139
  • 🐛 fix: match custom-element type selector case-insensitively by @gaborbernat in #137
  • 🐛 fix(encoding): cap the meta charset prescan at 1024 bytes by @gaborbernat in #136
  • 🐛 fix(selectors): accept namespace-qualified type selectors by @gaborbernat in #141
  • 🐛 fix: require exact annotation-xml encoding for integration point by @gaborbernat in #142
  • 🐛 fix(treebuilder): keep foreign
    inside its root by @gaborbernat in #143
  • 🐛 fix: keep raw-text serialization across construct and pickle by @gaborbernat in #145
  • 🐛 fix: put plain xmlns on a foreign element in the xmlns namespace by @gaborbernat in #140
  • 🐛 fix: pop open optgroup/option in a select-context fragment by @gaborbernat in #147
  • 🐛 fix(sanitizer): always neutralize plaintext by @gaborbernat in #146
  • 🐛 fix(encoding): add the WHATWG replacement decoder by @gaborbernat in #148
  • 🐛 fix(sanitizer): keep allowlisted SVG/MathML elements by @gaborbernat in #144
  • 🐛 fix: keep after-head whitespace under html, not in body by @gaborbernat in #150
  • ✨ feat(select): tree-structural pseudo-classes by @gaborbernat in #151
  • 🐛 fix: report a valueless attribute as the empty string by @gaborbernat in #153
  • 🐛 fix: pop open select for a keygen start tag in select mode by @gaborbernat in #154
  • 🐛 fix: pop open option before hr in a select-context fragment by @gaborbernat in #149
  • 🐛 fix(tokenizer): lock shared Tokenizer for free-threaded safety by @gaborbernat in #152
  • 🐛 fix(parse): template fragment enters in-template mode by @gaborbernat in #155
  • 🐛 fix(treebuilder): pop select for a table tag in select-in-table by @gaborbernat in #156
  • 🐛 fix: lock the shared node tree for free-threaded safety by @gaborbernat in #157
  • 🧪 test(freethread): stress, GIL guard, and ThreadSanitizer by @gaborbernat in #159
  • ✨ feat(select): :is(), :where(), and :has() pseudo-classes by @gaborbernat in #158
  • ⚡ perf(select): index by subject atom and cache compiled selectors by @gaborbernat in #162
  • 🧪 test(treebuilder): lock foreign

    /
    containment by @gaborbernat in #184
  • 🐛 fix(select): fold class/ID case in quirks mode by @gaborbernat in #185
  • ✨ feat(select): support the :not() negation pseudo-class by @gaborbernat in #188
  • 🐛 fix(select): match whitespace-only elements with :empty by @gaborbernat in #187
  • 🐛 fix(parser): treat empty DOCTYPE system id as quirks by @gaborbernat in #190
  • ✨ feat(xpath): native XPath 1.0 engine by @gaborbernat in #186
  • ✨ feat(select): support :scope, UI/form, :lang() and :dir() pseudo-classes by @gaborbernat in #193
  • 🐛 fix(select): make :is()/:where() forgiving selector lists by @gaborbernat in #195
  • ✨ feat(select): support :nth-child(An+B of S) selector lists by @gaborbernat in #198
  • ✨ feat(serializer): round-trip-safe HTML minify mode by @gaborbernat in #196
  • ✨ feat(compat): html.parser-compatible callback adapter by @gaborbernat in #199
  • ✨ feat(xpath): complete lxml/parsel call-option parity by @gaborbernat in #194
  • ✨ feat(serialize): to_markdown and to_text exporters with full markdownify/html2text/inscriptis parity by @gaborbernat in #191
  • 📝 docs(migration): note the encoding-detection boundary by @gaborbernat in #202
  • 📝 docs: compare and migrate resiliparse, note gumbo by @gaborbernat in #205
  • 📝 docs: compare and migrate lxml-html-clean & html-sanitizer by @gaborbernat in #203
  • 📝 docs: compare and migrate w3lib HTML utilities by @gaborbernat in #204
  • ✨ feat(query): pyquery-style fluent chaining query API by @gaborbernat in #200
  • 📝 docs(migration): add a From parsel guide by @gaborbernat in #201
  • 📝 docs: migration guides for html5-parser, pyquery & html.parser by @gaborbernat in #207
  • 👷 bench: compare the html.parser adapter to the standard library by @gaborbernat in #208
  • 📊 docs(performance): show parsel in the Querying table by @gaborbernat in #209
  • 🔒 fix(sanitizer): scheme-check ping and srcset URL lists by @gaborbernat in #228
  • ✨ feat(sanitizer): add Policy.set_attributes by @gaborbernat in #230
  • ✨ feat(sanitizer): add Policy.remove_with_content by @gaborbernat in #229
  • ✨ feat(export): annotation spans and Google-Docs Markdown mode by @gaborbernat in #232
  • ✨ feat(sanitizer): scrub the style attribute against a CSS property allowlist by @gaborbernat in #231
  • feat(tokenizer): char-ref tokens and token source by @gaborbernat in #240
  • feat(parser): streaming IncrementalParser (push parse to a tree) by @gaborbernat in #241
  • ✨ feat(tree): add form-field value helpers by @gaborbernat in #243
  • feat(serialize): attribute order and meta-charset options by @gaborbernat in #238
  • feat(markdown): per-tag converter hook for to_markdown by @gaborbernat in #239
  • feat(linkify): add plain-text link detector by @gaborbernat in #236
  • ✨ feat(parser): collect and report parse errors by @gaborbernat in #242
  • feat(linkify): reprocess anchors and customize detection by @gaborbernat in #237
  • ✨ feat(export): add to_markdown output modes by @gaborbernat in #235
  • ✨ feat(url): add Document.base_url and meta_refresh by @gaborbernat in #233
  • ✨ feat(parser): source line and column on parsed elements by @gaborbernat in #245
  • ✨ feat(links): enumerate and resolve document links by @gaborbernat in #244
  • docs: note migration limitations per section by @gaborbernat in #234
  • ✨ feat(encoding): content-based encoding detection (chardetng-style) by @gaborbernat in #206
  • ✨ feat(tree): bulk wrap_children and wrap_siblings by @gaborbernat in #254
  • feat(markdown): add to_markdown strip/convert filters by @gaborbernat in #255
  • feat(node): readability main-content extraction by @gaborbernat in #257
  • ✨ feat(tree): add Node.prune to keep a CSS selector by @gaborbernat in #258
  • feat(tree): parsel-style string extraction by @gaborbernat in #256
  • ✨ feat(annotation): surface-form and inline-tag exporters by @gaborbernat in #253
  • fix(types): drop duplicate Node.re stub by @gaborbernat in #259
  • refactor(layout): subsystem-first source + test tree by @gaborbernat in #260
  • docs(changelog): rewrite entries as concise end-user release notes by @gaborbernat in #261
  • 📚 docs: reorganize the site and type the full reference by @gaborbernat in #262
  • ✨ feat(tree): add classList editing methods by @gaborbernat in #284
  • ✨ feat(query): extract JSON-LD, Microdata, and OpenGraph by @gaborbernat in #291
  • ✨ feat(build): add E element builder for HTML generation by @gaborbernat in #289
  • ✨ feat(query): find nodes by text content by @gaborbernat in #285
  • ✨ feat(tree): set inner HTML, text, and adjacent HTML by @gaborbernat in #286
  • ✨ feat(tree): bulk remove and strip_tags by selector by @gaborbernat in #283
  • 📊 test(build): benchmark E builder and consolidate tests by @gaborbernat in #294
  • ✨ feat(xpath): add precompiled expression object by @gaborbernat in #281
  • ✨ feat(tree): read tables as rows and records by @gaborbernat in #290
  • ✨ feat(xpath): return node-sets from extension functions by @gaborbernat in #279
  • docs(migration): add html-text and MechanicalSoup by @gaborbernat in #293
  • ✨ feat(xpath): add EXSLT set, str, math, and date functions by @gaborbernat in #288
  • ✨ feat(xpath): bind namespace prefixes by @gaborbernat in #282
  • feat(export): article metadata on main_content by @gaborbernat in #292
  • ✨ feat(xpath): bind node-set variables by @gaborbernat in #278
  • ✨ feat(query): css_path and xpath_path for a node by @gaborbernat in #287
  • 📚 docs(migration): order guides by monthly downloads by @gaborbernat in #295
  • 📊 docs(migration): benchmark every operational migration feature by @gaborbernat in #296
  • 📖 docs: add design principles to the landing page by @gaborbernat in #297
  • 📊 docs(performance): add speed-up multipliers to all benchmark tables by @gaborbernat in #299
  • 📖 docs: tighten and reorganize the README by @gaborbernat in #298
  • 📚 docs(migration): unify all guides — monthly badges, perf inside Why turbohtml, speed-up multipliers by @gaborbernat in #300
  • 📝 docs(migration): report monthly downloads not totals by @gaborbernat in #301
  • 🔧 build(pre-commit): ruff-format and lint RST doc snippets by @gaborbernat in #302
  • 📝 docs(migration): per-library builder + metadata guides by @gaborbernat in #303
  • 🔧 build(c): clear -Wall/-Wextra warnings and gate them in CI by @gaborbernat in #304
  • ✨ feat(bench): isolate competitors per-venv and migrate every operation by @gaborbernat in #305
  • ⚡ perf(tokenizer): speed long DATA text scans by @gaborbernat in #309
  • ⚡ perf(dom): pool node wrappers on a freelist by @gaborbernat in #308
  • 🐛 fix(bench): measure mutating ops on fresh trees by @gaborbernat in #310
  • 📝 docs(migration): unify the migration guide layout by @gaborbernat in #311
  • 🐛 fix(bench): query a page the selectors match by @gaborbernat in #312
  • ✨ feat(bench): run size-pinned ops across the corpus ladder by @gaborbernat in #327
  • ♻️ refactor: config objects for over-stuffed method signatures by @gaborbernat in #328
  • ⚡ perf(find): match literal text= filters in C by @gaborbernat in #325
  • ⚡ perf(css): fast-path type combinators by @gaborbernat in #326
  • ⚡ perf(path): build node locators in linear time by @gaborbernat in #329
  • ♻️ refactor(linkify): config object for linkify and Linker by @gaborbernat in #330
  • ♻️ refactor(api): clean + extract topic namespaces by @gaborbernat in #331
  • ✨ feat(clean): HTML minify and From minify-html guide by @gaborbernat in #341
  • ✨ feat(clean): add a value-safe CSS minifier by @gaborbernat in #346
  • 📊 docs: benchmark tables from committed JSON feeds, tinted and grouped by @gaborbernat in #348
  • ✨ feat(serialize): native JavaScript minifier by @gaborbernat in #344
  • ✨ feat(match): soupsieve-style CSS matching surface by @gaborbernat in #354
  • ✨ feat(clean): expose HTML minify (minify-html/htmlmin) by @gaborbernat in #353
  • 📝 docs(build): htbuilder migration coverage by @gaborbernat in #356
  • ✨ feat(detect): standalone encoding detection (chardet/charset-normalizer) by @gaborbernat in #361
  • 📝 docs(build): simple-html migration coverage by @gaborbernat in #357
  • 📝 docs(build): hyperpython migration coverage by @gaborbernat in #358
  • ✨ feat(convert): css_to_xpath translator (cssselect) by @gaborbernat in #355
  • 📝 docs(build): markyp migration coverage by @gaborbernat in #359
  • ✨ feat(extract): per-paragraph boilerplate classification (justext) by @gaborbernat in #362
  • 📝 docs(build): fast-html migration coverage by @gaborbernat in #360
  • 📝 docs(migration): boilerpy3 migration coverage by @gaborbernat in #364
  • ✨ feat(extract): URL clean/normalize/extract helpers (courlan) by @gaborbernat in #363
  • 📝 docs(migration): goose3 migration coverage by @gaborbernat in #365
  • ✨ feat(extract): w3lib URL surface via UrlCleaning query filters by @gaborbernat in #366
  • 📝 docs(migration): readabilipy migration coverage by @gaborbernat in #367
  • 📝 docs(migration): news-please migration coverage by @gaborbernat in #368
  • ✨ feat(extract): microdata get_items shim (microdata) by @gaborbernat in #369
  • ✨ feat(extract): opengraph card shim (opengraph) by @gaborbernat in #370
  • ✨ feat(extract): publication-date extraction (htmldate) by @gaborbernat in #371
  • 🐛 fix(selectors): decode quoted-string escapes and fix XPath round() ties by @gaborbernat in #376
  • 👷 ci: add CodSpeed continuous benchmarking by @gaborbernat in #380
  • 👷 ci: pin PYTHONHASHSEED in the CodSpeed run by @gaborbernat in #382
  • 👷 ci: stabilize CodSpeed noise floor via glibc hwcaps by @gaborbernat in #383
  • 🐛 fix(selectors): four Selectors-4 conformance gaps by @gaborbernat in #381
  • 🐛 fix(query): XPath memory-safety and conformance by @gaborbernat in #440
  • 🐛 fix(serialize): JavaScript minifier miscompiles by @gaborbernat in #441
  • 🐛 fix(parser): tree-construction whitespace handling by @gaborbernat in #442
  • 🐛 fix(extract): content-extraction scorer correctness by @gaborbernat in #446
  • 🐛 fix(clean): CSS minifier value-safety by @gaborbernat in #443
  • 🐛 fix(convert): to_markdown and to_text correctness by @gaborbernat in #445
  • 🐛 fix(detect): character-encoding conformance by @gaborbernat in #448
  • 🐛 fix(clean): sanitizer conformance and error types by @gaborbernat in #447
  • 🐛 fix(build): table-model conformance by @gaborbernat in #450
  • 🐛 fix(extract): microdata extraction conformance by @gaborbernat in #449
  • 🐛 fix(extract): URL normalization conformance by @gaborbernat in #452
  • 🐛 fix(clean): linkify boundary conformance by @gaborbernat in #451
  • 🐛 fix(clean): minify optional-end-tag omission by @gaborbernat in #453
  • 🐛 fix(build): form-associated element conformance by @gaborbernat in #444
  • 🐛 fix(query): CSS Selectors Level 4 conformance by @gaborbernat in #454
  • 🐛 fix(build): reject invalid attribute names and void-element children by @gaborbernat in #456
  • 👷 ci(bench): stabilize noise-prone CodSpeed benchmarks on real corpora by @gaborbernat in #455
  • 📝 docs(migration): standardize guides on a head-to-head format by @gaborbernat in #457
  • ✨ feat(clean): sanitizer attribute-prefix, value-allowlist, and host-allowlist policy by @gaborbernat in #461
  • 🧹 refactor(api): actionable errors, correct types, :raises: docstrings by @gaborbernat in #458
  • ✨ feat(clean): minify CSS in <style> and style attributes by @gaborbernat in #460
  • ✨ feat(serialize): preserve license/bang comments in minify_js by @gaborbernat in #462
  • ✨ feat(extract): resolve relative URLs in metadata against a base_url by @gaborbernat in #463
  • ✨ feat(detect): report UTF-8-SIG and BOM-based UTF-16/32 labels by @gaborbernat in #464
  • ✨ feat(clean): minify boolean-attribute collapse and optional end-tag omission by @gaborbernat in #465
  • ✨ feat(serialize): lenient never-fail mode for minify_js by @gaborbernat in #466
  • ✨ feat(query): Node.equals for structural tree comparison by @gaborbernat in #468
  • ✨ feat(extract): registrable-domain boundary for link filtering by @gaborbernat in #467
  • ✨ feat(clean): sanitize <style> element contents against the CSS policy by @gaborbernat in #469
  • ✨ feat(build): full-document page-shell helper by @gaborbernat in #471
  • ✨ feat(extract): RDFa and Dublin Core structured-data extraction by @gaborbernat in #472
  • ✨ feat: python -m turbohtml command-line interface by @gaborbernat in #470
  • ✨ feat(serialize): streaming/incremental serialization by @gaborbernat in #473
  • ✨ feat(detect): content-based language detection by @gaborbernat in #474
  • ✨ feat(extract): canonical, site_name, tags, and lead-image metadata by @gaborbernat in #476
  • ⚡ perf(extract): skip the RDFa walk when a document has no typeof by @gaborbernat in #475
  • ⚡ perf(extract): fuse article metadata into one head walk by @gaborbernat in #477
  • ♻️ refactor(api): unify the 1.0 public surface by @gaborbernat in #479
  • 📝 docs: restructure for 1.0 discoverability and taxonomy by @gaborbernat in #480
  • ♻️ refactor(c): house style, decompose run_drain by @gaborbernat in #482
  • ⚡ perf(build): profile-guided optimization for the release build by @gaborbernat in #481
  • ✨ feat(api): OpenGraph and CSSMinify records complete the 1.0 surface by @gaborbernat in #483
  • 🔒 build(data): pin the PSL and TLD table sources by @gaborbernat in #484
  • ♻️ refactor(c): enable LTO, split hot/cold headers, move query out of element.c by @gaborbernat in #485
  • ⚡ perf(url): port URL splitting to C by @gaborbernat in #486
  • ⚡ perf(core): find() atom index for tag+attr, presize streaming chunks by @gaborbernat in #487
  • ⚡ perf(url): port percent-encoding to C by @gaborbernat in #488
  • ⚡ perf(build): cover every hot op in the PGO training workload by @gaborbernat in #489
  • ⚡ perf(url): port relative-reference resolution to C by @gaborbernat in #490
  • 🔧 build(ci): measure a deterministic LTO-only build on CodSpeed by @gaborbernat in #491
  • ⚡ perf(build): train PGO on a representative real-world corpus by @gaborbernat in #492
  • ⚡ perf(url): port IDNA (punycode + UTS-46) host encoding to C by @gaborbernat in #493
  • ♻️ refactor(url): consolidate registrable-domain into the url module by @gaborbernat in #494
  • 🔒 fix(url): guard registrable-domain against non-ASCII host OOB read by @gaborbernat in #495
  • ⚡ perf(extract): port date parsing to C by @gaborbernat in #496
  • 📝 docs: add a how-this-was-built transparency note by @gaborbernat in #497
  • 🔧 build(bench): add --pgo flag to build the release-representative baseline by @gaborbernat in #498
  • ♻️ refactor(url): route the sanitizer allowlist through the unified parser by @gaborbernat in #499
  • 🔒 docs(security): real security policy and pinned-table provenance by @gaborbernat in #500
  • 🔒 fix(url): harden IDNA punycode decoder against overflow and OOB by @gaborbernat in #501
  • 🔒 test(clean): run DOMPurify's XSS corpus against the sanitizer by @gaborbernat in #502
  • 🔒 test(fuzz): ASan/UBSan fuzz harnesses for the untrusted-input entry points by @gaborbernat in #503
  • 🔒 fix(parse): cap tree nesting depth to prevent stack overflow (DoS) by @gaborbernat in #505
  • 🔒 test(parse): full html5lib-tests differential conformance oracle by @gaborbernat in #504
  • 🔒 fix(clean): per-node namespace-reachability check (mXSS defense-in-depth) by @gaborbernat in #507
  • 📝 docs(security): document single-pass sanitization by @gaborbernat in #506
  • 🔒 fix(tokenize): O(n) attribute-name deduplication to close a parse DoS by @gaborbernat in #508
  • ⚡ perf(query): linear single-pass :has() evaluation by @gaborbernat in #509
  • 🔒 build(data): SHA-256 provenance for the IDNA table generator by @gaborbernat in #510
  • ♻️ refactor(c): overflow-safe shared vec.h buffer growth by @gaborbernat in #511
  • ♻️ refactor(c): shared ascii.h and reconcile whitespace/case drift by @gaborbernat in #512
  • ♻️ refactor(tests): mirror src layout + quality by @gaborbernat in #513
  • 📝 docs(project): present shipped code as turbohtml's own by @gaborbernat in #514
  • ♻️ refactor(project): fold match into the query namespace by @gaborbernat in #515
  • 📄 chore: fold NOTICE into LICENSE by @gaborbernat in #516
  • 🧹 chore: whole-tree simp + no-slop sweep by @gaborbernat in #517
  • 📝 docs(changelog): collapse 0.4.0→1.0.0 into a 1.0 overview by @gaborbernat in #518
  • 📦 perf(build): shrink the release wheel by @gaborbernat in #519
  • 📊 docs(bench): final 1.0 benchmark numbers (PGO+LTO) by @gaborbernat in #520
  • 🐛 fix(serialize): skip null memcpy on empty run by @gaborbernat in #521
  • ✨ feat(bench): cover every op each library shares by @gaborbernat in #523
  • 📊 feat(bench): measure minifier size, add industry tools, close coverage gaps by @gaborbernat in #524
  • 📝 docs(changelog): fold the last fragment into the 1.0 overview by @gaborbernat in #525

New Contributors

Full Changelog: 0.4.0...1.0.0