Skip to content

Relax <select> parser #10557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 26 commits into
base: main
Choose a base branch
from
Open

Relax <select> parser #10557

wants to merge 26 commits into from

Conversation

josepharhar
Copy link
Contributor

@josepharhar josepharhar commented Aug 13, 2024

This patch makes the parser allow additional tags in <select> besides <option>, <optgroup>, and <hr>, mostly by removing the "in select" and "in select in table" parser modes.

In order to replicate the behavior where opening a <select> tag within another open <select> tag inserts a </select> close tag, a traversal through the stack of open elements was added which I borrowed from the <button> part of the parser.

This patch also changes the processing model to make <select> look through all its descendants in the DOM tree for <option> elements, rather than just children and optgroup children which conform to the content model. This is needed for compat reasons because there are websites which put other tags in between their <select> and <option>s which would break with this parser change unless we also update this processing model. More context here and here.

Fixes #10310

(See WHATWG Working Mode: Changes for more details.)


/form-elements.html ( diff )
/index.html ( diff )
/infrastructure.html ( diff )
/parsing.html ( diff )

@annevk
Copy link
Member

annevk commented Aug 14, 2024

Why does this also make a bunch of changes to the processing model of select elements? If that needs to be part of this change that should really be better motivated in the commit message.

@josepharhar
Copy link
Contributor Author

Why does this also make a bunch of changes to the processing model of select elements? If that needs to be part of this change that should really be better motivated in the commit message.

The changes in the processing model do support the new proposed content model for select, but they also mitigate compat issues for websites which are already putting tags in between <select> and <option> in their HTML.

For example, without these changes to the processing model, the following <select> would not register as having any options at all:

<select>
  <div>
    <option>...</option>
    ...
  </div>
</select>

In my compat analysis, I found a lot of websites which are doing this, so in order to ship the parser changes separately from customizable select in chrome, I plan to ship the parser changes and these processing model changes together, because otherwise there would be too much breakage.

I'm happy to put them in a separate PR if you want, or keep them here and update the commit message (sorry for not putting it in there). Which would you prefer?

@annevk
Copy link
Member

annevk commented Aug 16, 2024

Thanks! Given that rationale I think it's good to couple the changes, but that should be in the commit message as well.

@zcorpan
Copy link
Member

zcorpan commented Aug 19, 2024

This doesn't define optional tags for </option> and </optgroup> correctly.

The definition for "have a particular element in select scope" may be needed for that, but should be changed to be similar to "have a particular element in button scope" (but for select).

In particular, allow these without a parse error:

<select>
<optgroup>
<option>
<optgroup>
</select>

The second <optgroup> should pop the option and optgroup

<select>
<option><p>
<option>
</select>

This should generate implied end tags and pop the option.

<select>
<optgroup>
<hr>
<option>
<hr>
</select>

The hrs should pop the optgroup and option in a select.

See how the parser deals with <ruby><rtc><rt>, I think that can be used as a model for select.

@josepharhar
Copy link
Contributor Author

Thanks! Given that rationale I think it's good to couple the changes, but that should be in the commit message as well.

Done.

This doesn't define optional tags for </option> and </optgroup> correctly.

@zcorpan thanks for the feedback! I did some experimenting and added some logic to the start tags for option, optgroup, and hr. What do you think?

@zcorpan
Copy link
Member

zcorpan commented Aug 22, 2024

  • Need to check that a select element is in scope so that parsing of option/optgroup tags outside of select doesn't change. Example <option><p>1</option>2<option>3
  • Need to add select to the "in scope" list so that <div><select></div><option> doesn't close the select.
  • Should report parse errors when unexpected elements are popped. Compare with what the spec does for rt/rtc in ruby. Example <select><option><span>1</option>2<option>3

@josepharhar
Copy link
Contributor Author

  • Need to check that a select element is in scope so that parsing of option/optgroup tags outside of select doesn't change. Example <option><p>1</option>2<option>3

I thought that this is covered by "While the stack of open elements has an option element in select scope". What exactly should I change?

  • Need to add select to the "in scope" list so that <div><select></div><option> doesn't close the select.

Done

  • Should report parse errors when unexpected elements are popped. Compare with what the spec does for rt/rtc in ruby. Example <select><option><span>1</option>2<option>3

I'm guessing this is from "If the current node is not now a rtc element or a ruby element, this is a parse error," right?

Should I add "If the current node is not now an option element, this is a parse error" after "While the stack of open elements has an option element in select scope, pop an element from the stack of open elements"?

@zcorpan
Copy link
Member

zcorpan commented Aug 23, 2024

The "in select scope" I think should be removed altogether since it assumes the stack will not have other elements when in a select, which is no longer the case. Use the normal "in scope" instead.

High-level of what I think should happen: when parsing option or optgroup start tag: check that select is in scope, check that option or optgroup is in scope, generate implied end tags (except for optgroup when handling <option>), check the current node or the stack of open elements again, report a parse error if appropriate, then pop elements off the stack until the option or optgroup has been popped, then insert the new element.

I can look into this more next week and suggest more specific changes.

@josepharhar
Copy link
Contributor Author

High-level of what I think should happen: when parsing option or optgroup start tag: check that select is in scope, check that option or optgroup is in scope, generate implied end tags (except for optgroup when handling <option>), check the current node or the stack of open elements again, report a parse error if appropriate, then pop elements off the stack until the option or optgroup has been popped, then insert the new element.

Thanks! I gave this a try

@josepharhar
Copy link
Contributor Author

@zcorpan how does the latest text look?

Copy link
Member

@zcorpan zcorpan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the option/optgroup cases are right after these changes.

@josepharhar
Copy link
Contributor Author

This doesn't define optional tags for </option> and </optgroup> correctly.

I was talking to @mfreed7 about the changes we've made in the PR so far, and I feel like I couldn't provide a good explanation for why we are making the parser not support cases like these:

  • <hr> inside an <option>
  • nested <optgroup>s

Is it just compat reasons? Is there a good justification?

This patch makes the parser allow additional tags in <select> besides
<option>, <optgroup>, and <hr>, mostly by removing the "in select" and
"in select in table" parser modes.

In order to replicate the behavior where opening a <select> tag within
another open <select> tag inserts a </select> close tag, a traversal
through the stack of open elements was added which I borrowed from the
<button> part of the parser.

This will need test changes to be implemented in html5lib.

Fixes whatwg#10310
@zcorpan
Copy link
Member

zcorpan commented Sep 11, 2024

It would be a breaking change from what is conforming HTML today, and break compat for sites that omit </option> and </optgroup> tags. It's the same as why we can't allow certain elements in p.

chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Feb 21, 2025
This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
aarongable pushed a commit to chromium/chromium that referenced this pull request Feb 24, 2025
This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Feb 24, 2025
This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request Feb 24, 2025
This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}
source Outdated

<li><p>Pop elements from the <span>stack of open elements</span> until a <code>select</code>
element has been popped from the stack.</p></li>
</ol>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So in the fragment case you could still end up with an input element child? That will result in mutation XSS or some such, right? cc @zcorpan @mozfreddyb

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the context element of the fragment parsing algorithm or any of the context element's ancestors is a select element, then yes you would end up with an input inside a select. If the select is inside the fragment being parsed, then it will not be inside the select.

software hixie ch_utilities_js_live-dom-viewer_

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that might cause some XSS-related issues downstream, but then all parser changes like these do... 😕

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I add a fragment parsing context element check for input elements, would that fix this? I'd do the same thing as I did for select elements here: 9d86011

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we dropped the input element on the floor in the fragment case it might fix it I suppose.

I think we should be very careful that we don't introduce any new opportunities where serialize-then-parse results in a different tree (assuming the original tree was the result of parsing + innerHTML calls).

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @zcorpan here. I don't believe this change introduces any new mXSS technique (even if some mutations exist).

I would even go as far as saying that this change could be security-positive. Internally at Google we're aware of HTML sanitizer bypass existing because of the fact that the sanitizer wasn't aware of the special parsing rules for <select> (so the discrepancy could be exploited). When this change is in effect, the bypass doesn't work anymore because <select> is now parsed as any other tag.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds like a conclusion from an incomplete understanding of the current state of this change. Because it's still not parsed quite like any other tag.

I will grant that I'm not sure we need to do it, but I think it would be good if innerHTML could not be used to generate a different tree here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issue is that there are a lot of cases where innerHTML can be used to create a DOM that is mutated upon serialize-parse, and we likely can't start to drop elements without breaking the web. Doing so only for input in select but not input in option in select or h2 in h1 or any foster-parented content in table, and so on, doesn't seem like it solves anything. So my conclusion is that we should let input be inserted.

Also see #10310 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There could be a web compat reason to drop input though, similar to #10310 (comment)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @zcorpan, we have been keeping <input> in <select> for assigning to select.innerHTML in stable chrome for a while now with no issues, so I am comfortable keeping it that way.

Thanks for the analysis @securityMB!

@lukewarlow
Copy link
Member

Might be a silly question but does the check for HTML fragment parsing algorithm mean that this doesn't apply if you're parsing in an XHTML document?

@annevk
Copy link
Member

annevk commented Feb 28, 2025

@lukewarlow XHTML documents use the XML parser.

moz-v2v-gh pushed a commit to mozilla/gecko-dev that referenced this pull request Feb 28, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882
i3roly pushed a commit to i3roly/firefox-dynasty that referenced this pull request Feb 28, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified that referenced this pull request Mar 1, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <tcaptanchromium.org>
Commit-Queue: Joey Arhar <jarharchromium.org>
Commit-Queue: Traian Captan <tcaptanchromium.org>
Auto-Submit: Joey Arhar <jarharchromium.org>
Cr-Commit-Position: refs/heads/main{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882

UltraBlame original commit: 5aa6c6ca474e72bba059f5cca3e877950d617dc7
gecko-dev-updater pushed a commit to marco-c/gecko-dev-comments-removed that referenced this pull request Mar 1, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <tcaptanchromium.org>
Commit-Queue: Joey Arhar <jarharchromium.org>
Commit-Queue: Traian Captan <tcaptanchromium.org>
Auto-Submit: Joey Arhar <jarharchromium.org>
Cr-Commit-Position: refs/heads/main{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882

UltraBlame original commit: 5aa6c6ca474e72bba059f5cca3e877950d617dc7
gecko-dev-updater pushed a commit to marco-c/gecko-dev-wordified-and-comments-removed that referenced this pull request Mar 1, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <tcaptanchromium.org>
Commit-Queue: Joey Arhar <jarharchromium.org>
Commit-Queue: Traian Captan <tcaptanchromium.org>
Auto-Submit: Joey Arhar <jarharchromium.org>
Cr-Commit-Position: refs/heads/main{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882

UltraBlame original commit: 5aa6c6ca474e72bba059f5cca3e877950d617dc7
jamienicol pushed a commit to jamienicol/gecko that referenced this pull request Mar 5, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882
glandium pushed a commit to mozilla-firefox/firefox that referenced this pull request Apr 1, 2025
…Relaxation, a=testonly

Automatic update from web-platform-tests
Update select-value WPT for SelectParserRelaxation

This was pointed out here:
whatwg/html#10557 (comment)

Change-Id: I443ddd71e48ecefe598a7be07d0705a68db36d02
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6077768
Reviewed-by: Traian Captan <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Commit-Queue: Traian Captan <[email protected]>
Auto-Submit: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1423995}

--

wpt-commits: dcbc65f3c164e3a47ed07a0d216f8a7fc5b0d54d
wpt-pr: 50882
@josepharhar
Copy link
Contributor Author

This spec PR currently disallows <input> in <select> by looking up all of the ancestor nodes when inserting an <input>. Is anyone opposed to changing this to only looking at the parent node rather than all of the ancestors?

This would allow this to be parsed without the parser changing things:

<select>
  <div>
    <input>

@annevk
Copy link
Member

annevk commented May 2, 2025

That would mean <select><option><input> and <select><optgroup><input> would also not pop the option and optgroup elements or the select element? In principle that seems reasonable, though it's not entirely clear if it's worth the potential web compatibility fallout as we don't have a plan to make it accessible, right?

@josepharhar
Copy link
Contributor Author

Good point, I'd be OK with doing the <input> closing <select> logic when encountering <input>, <option>, or <optgroup> as the parent node.

@annevk
Copy link
Member

annevk commented May 2, 2025

That doesn't answer the more significant question. (And I don't really have the data to pick between that and what you suggested. I was just pointing out a consequence.)

@josepharhar
Copy link
Contributor Author

I have UseCounters for this which haven't reached stable yet, and I am working with accessibility folks on this right now. I can share more once the UseCounters hit stable and when I get more accessibility feedback.

aarongable pushed a commit to chromium/chromium that referenced this pull request May 13, 2025
Based on feedback in the <select> parser relaxation HTML spec PR, we
might want to close the <select> on these tags when parsing an <input>:
whatwg/html#10557 (comment)

If the UseCounter shows that usage is low enough, then we could remove
this behavior.

Bug: 402429384
Change-Id: Id8809d40162c7fd06a446e146d65aaf6258d2e5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6506819
Reviewed-by: Mason Freed <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1459707}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request May 13, 2025
Based on feedback in the <select> parser relaxation HTML spec PR, we
might want to close the <select> on these tags when parsing an <input>:
whatwg/html#10557 (comment)

If the UseCounter shows that usage is low enough, then we could remove
this behavior.

Bug: 402429384
Change-Id: Id8809d40162c7fd06a446e146d65aaf6258d2e5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6506819
Reviewed-by: Mason Freed <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1459707}
chromium-wpt-export-bot pushed a commit to web-platform-tests/wpt that referenced this pull request May 13, 2025
Based on feedback in the <select> parser relaxation HTML spec PR, we
might want to close the <select> on these tags when parsing an <input>:
whatwg/html#10557 (comment)

If the UseCounter shows that usage is low enough, then we could remove
this behavior.

Bug: 402429384
Change-Id: Id8809d40162c7fd06a446e146d65aaf6258d2e5b
Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6506819
Reviewed-by: Mason Freed <[email protected]>
Commit-Queue: Joey Arhar <[email protected]>
Cr-Commit-Position: refs/heads/main@{#1459707}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
do not merge yet Pull request must not be merged per rationale in comment normative change topic: parser topic: select The <select> element
Development

Successfully merging this pull request may close these issues.

HTML parser changes for customizable <select>