Parse canonical DATETIME and DATE without sscanf - #1458
Open
nuclearspike wants to merge 1 commit into
Open
Conversation
### Motivation / Background Casting a text-protocol DATETIME or DATE runs `sscanf` once per value. On a 20,000-row result with one DATETIME column, `sscanf` accounted for 653 of 10,176 stack samples (~6% of wall time) in a macOS `sample` profile -- more than the Time construction it feeds. `sscanf` is doing general-purpose format interpretation for the canonical shape MySQL normally emits: `YYYY-MM-DD HH:MM:SS[.ffffff]`. ### Detail Adds a strict parser for exactly that canonical form (and `YYYY-MM-DD` for DATE), with the original `sscanf` retained as the fallback for anything that does not match byte-for-byte -- unusual widths, unexpected separators, anything a proxy or an older server might emit. The fallback keeps the previous semantics intact rather than trying to reproduce them. The fast parser validates shape, not meaning. A canonical-shaped but semantically invalid value such as `0000-00-00 00:00:00` is accepted by the parser and then handled by the same downstream code as before, which is what keeps the two paths equivalent -- the parser is not the place where zero dates or out-of-range components were ever rejected. Microsecond handling is the subtle part: `sscanf` captures the fractional digits as a string that `msec_char_to_uint` then interprets left-aligned, so ".5" means 500000 microseconds, not 5. The fast path applies the same left-aligned scaling directly, and `parsed_msec` tells the call site not to re-run `msec_char_to_uint` over an untouched buffer. ### On the specs The DATETIME spec casts each literal to the `DATETIME(N)` that actually puts N fractional digits on the wire. This matters: a `DATETIME(6)` cast normalises `...56.5` to `...56.500000`, so a spec that casts everything to `DATETIME(6)` never hands the parser a short fractional part and does not test what it appears to. Each case therefore also reads the value back with `cast: false` and asserts the exact bytes the parser was given, so the premise cannot rot silently. What the specs do not cover: the `sscanf` fallback itself. The existing integration suite, running against a stock MySQL server, does not naturally produce a non-canonical DATETIME on the wire to drive it with. The fallback is retained for producers that are not a stock MySQL server (proxies, older servers, MariaDB), and that path is unchanged from the current code rather than reimplemented -- but it is reasoned, not exercised here. Verified separately that these specs exercise the new parser rather than silently falling through to `sscanf`: with the fast path's microsecond output deliberately corrupted, the fractional-widths spec fails. ### Benchmark 20,000 rows, single DATETIME column, `as: :array` with `database_timezone: :utc`, timing materialisation only (the query is issued outside the measured region). Each sample is the min of 9 runs in one process; 9 samples per arm, arms interleaved. | | median | range | |---|---|---| | master | 9.60 ms | 9.24 - 9.98 | | patch | 7.43 ms | 7.25 - 8.57 | -22.6%, distributions non-overlapping. ### Where this was tested Ruby 3.3.10, MySQL 9.6.0 (Homebrew, libmysqlclient.24), macOS 26.5 arm64, Apple clang 21. The rspec suite was run there and is green apart from three failures already present on the base commit: `client_spec.rb:101`, `:729`, and `:740` -- the reconnect group. Not tested: MariaDB, Windows, 32-bit platforms, other Ruby versions, or any Linux CI matrix. Nothing in this commit is platform-specific -- it is byte inspection of an ASCII string -- but the claim that the server always emits the canonical form is a MySQL claim, and MariaDB in particular is unverified here. That is the case the retained `sscanf` fallback exists for.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation / Background
Casting a text-protocol DATETIME or DATE runs
sscanfonce per value. On a20,000-row result with one DATETIME column,
sscanfaccounted for 653 of 10,176stack samples (~6% of wall time) in a macOS
sampleprofile -- more than theTime construction it feeds.
sscanfis doing general-purpose format interpretation for the canonical shapeMySQL normally emits:
YYYY-MM-DD HH:MM:SS[.ffffff].Detail
Adds a strict parser for exactly that canonical form (and
YYYY-MM-DDforDATE), with the original
sscanfretained as the fallback for anything thatdoes not match byte-for-byte -- unusual widths, unexpected separators, anything
a proxy or an older server might emit. The fallback keeps the previous semantics
intact rather than trying to reproduce them.
The fast parser validates shape, not meaning. A canonical-shaped but
semantically invalid value such as
0000-00-00 00:00:00is accepted by theparser and then handled by the same downstream code as before, which is what
keeps the two paths equivalent -- the parser is not the place where zero dates
or out-of-range components were ever rejected.
Microsecond handling is the subtle part:
sscanfcaptures the fractional digitsas a string that
msec_char_to_uintthen interprets left-aligned, so ".5" means500000 microseconds, not 5. The fast path applies the same left-aligned scaling
directly, and
parsed_msectells the call site not to re-runmsec_char_to_uintover an untouched buffer.On the specs
The DATETIME spec casts each literal to the
DATETIME(N)that actually puts Nfractional digits on the wire. This matters: a
DATETIME(6)cast normalises...56.5to...56.500000, so a spec that casts everything toDATETIME(6)never hands the parser a short fractional part and does not test what it appears
to. Each case therefore also reads the value back with
cast: falseand assertsthe exact bytes the parser was given, so the premise cannot rot silently.
What the specs do not cover: the
sscanffallback itself. The existingintegration suite, running against a stock MySQL server, does not naturally
produce a non-canonical DATETIME on the wire to drive it with. The fallback is retained for
producers that are not a stock MySQL server (proxies, older servers, MariaDB),
and that path is unchanged from the current code rather than reimplemented --
but it is reasoned, not exercised here.
Verified separately that these specs exercise the new parser rather than
silently falling through to
sscanf: with the fast path's microsecond outputdeliberately corrupted, the fractional-widths spec fails.
Benchmark
20,000 rows, single DATETIME column,
as: :arraywithdatabase_timezone: :utc, timing materialisation only (the query is issuedoutside the measured region). Each sample is the min of 9 runs in one process;
9 samples per arm, arms interleaved.
-22.6%, distributions non-overlapping.
Where this was tested
Ruby 3.3.10, MySQL 9.6.0 (Homebrew, libmysqlclient.24), macOS 26.5 arm64,
Apple clang 21. The rspec suite was run there and is green apart from three failures already
present on the base commit:
client_spec.rb:101,:729, and:740-- thereconnect group.
Not tested: MariaDB, Windows, 32-bit platforms, other Ruby versions, or any
Linux CI matrix. Nothing in this commit is platform-specific -- it is byte
inspection of an ASCII string -- but the claim that the server always emits
the canonical form is a MySQL claim, and MariaDB in particular is unverified
here. That is the case the retained
sscanffallback exists for.🤖 Generated with Claude Code