Conversation
michaelknap
left a comment
There was a problem hiding this comment.
Thanks for the quick fix! I think the current patch fixes the original <31-digit
hang, but it still leaves an integer overflow in denom.
The patch keeps denom as a signed int:
int sign = 1, denom = 1, denom_digits = 0, part = 0;and rejects only after more than 15 fractional digits:
if (++denom_digits > 15) {
PyErr_SetString(DataError, "too many fractional digits");
return NULL;
}
denom *= 10;On common platforms where int is 32-bit, denom *= 10 overflows at the 10th
fractional digit:
1 digit -> 10
6 digits -> 1000000
9 digits -> 1000000000
10 digits -> 10000000000 // overflows 32-bit int
So inputs with roughly 10-15 fractional digits can still execute signed integer
overflow before the new DataError path is reached. That may no longer trigger
the original infinite loop, but it still leaves undefined behavior and can
produce incorrect interval parsing.
A simple test case for this remaining issue is:
import psycopg2.extensions as ext
ext.INTERVAL(b"0:0:0." + b"0" * 12, None)I would suggest guarding before multiplication, for example:
if (part == 6) {
if (denom > INT_MAX / 10) {
PyErr_SetString(DataError, "too many fractional digits");
return NULL;
}
denom *= 10;
}Alternatively, if the intended behavior is to allow up to 15 fractional digits,
denom should be widened to a 64-bit type such as PY_LONG_LONG, and the code
should still guard before multiplying.
Hope this helps! :)
|
Hmm, you are right, I picked 15 to "stay into a 16 bits signed int" but those are binary digits, not decimal digits. Probably better to detect the overflow in the loop. I dropped exception raising. Dropping digits is enough. |
Thank you for quick updates and I'm sorry for my late reply. I think quietly truncating extra fractional digits is a reasonable behavior, but The overflow happens here before the comparison can detect it: int new_denom = denom * 10;Because The comparison also does not reliably detect wraparound. For example, on common The wrapped value is still greater than the old denominator, so the overflowed If the desired behavior is to quietly truncate extra fractional digits, I think if (part == 6) {
if (denom <= INT_MAX / 10) {
denom *= 10;
}
}This may require including `<limits.h> for INT_MAX. |
Close #1835 Thank you very much Michael Knap for reporting the bug and making sure that the solution is correct.
|
@michaelknap you are right: I had assumed a two's complement but the behaviour is defined to be... undefined. So checking for overflow as you suggest seems the right call. Pushed and rebased. |
michaelknap
left a comment
There was a problem hiding this comment.
Thank you for the changes and for all the work that goes into this module :) All the best.
Close #1835
@michaelknap does it seem ok?