In CSS tokenization, some tokens have associated data (values, units, integer/number flags…). The bad-string-token has no associated data and WordPress\DataLiberation\CSS\CSSProcessor::get_token_value() should return null on bad string tokens.
It currently returns the part of the string up to the breaking codepoint. For example:
$p = WordPress\DataLiberation\CSS\CSSProcessor::create( "'str\ning'" );
$p->next_token();
assert( $p->get_token_type() === WordPress\DataLiberation\CSS\CSSProcessor::TOKEN_BAD_STRING );
assert( $p->get_token_value() === null, 'Expected null token value, got ' . var_export( $p->get_token_value(), true ) );
Will throw AssertionError: Expected null token value, got 'str'.
There's a relevant note in the parsing section about the preserved tokens:
Note: The tokens <}-token>s, <)-token>s, <]-token>, <bad-string-token>, and <bad-url-token> are always parse errors, but they are preserved in the token stream by this specification to allow other specs, such as Media Queries, to define more fine-grained error-handling than just dropping an entire declaration or block.
Note that bad-url-token returns null token value, as expected.
In CSS tokenization, some tokens have associated data (values, units, integer/number flags…). The bad-string-token has no associated data and
WordPress\DataLiberation\CSS\CSSProcessor::get_token_value()should returnnullon bad string tokens.It currently returns the part of the string up to the breaking codepoint. For example:
Will throw
AssertionError: Expected null token value, got 'str'.There's a relevant note in the parsing section about the preserved tokens:
Note that
bad-url-tokenreturnsnulltoken value, as expected.