Reduce number of bytes examined by convert_one_string_to_scalar().

Previously, convert_one_string_to_scalar() would examine up to 20 bytes of
the input string, producing a scalar conversion with theoretical precision
far greater than is of any possible use considering the other limitations
on the accuracy of the resulting selectivity estimate.  (I think this
choice might pre-date the caller-level logic that strips any common prefix
of the strings; before that, there could have been value in scanning the
strings far enough to use all the precision available in a double.)

Aside from wasting cycles to little purpose, this choice meant that the
"denom" variable could grow to as much as 256^21 = 3.74e50, which could
overflow in some non-IEEE float arithmetics.  While we don't really support
any machines with non-IEEE arithmetic anymore, this still seems like quite
an unnecessary platform dependency.  Limit the scan to 12 bytes instead,
thus limiting "denom" to 256^13 = 2.03e31, a value more likely to be
computable everywhere.

Per testing by Greg Stark, which showed overflow failures in our standard
regression tests on VAX.
This commit is contained in:
Tom Lane 2015-08-23 15:15:47 -04:00
parent 44ed65a545
commit aad663a0b4
1 changed files with 9 additions and 3 deletions

View File

@ -3861,10 +3861,16 @@ convert_one_string_to_scalar(char *value, int rangelo, int rangehi)
return 0.0; /* empty string has scalar value 0 */
/*
* Since base is at least 10, need not consider more than about 20 chars
* There seems little point in considering more than a dozen bytes from
* the string. Since base is at least 10, that will give us nominal
* resolution of at least 12 decimal digits, which is surely far more
* precision than this estimation technique has got anyway (especially in
* non-C locales). Also, even with the maximum possible base of 256, this
* ensures denom cannot grow larger than 256^13 = 2.03e31, which will not
* overflow on any known machine.
*/
if (slen > 20)
slen = 20;
if (slen > 12)
slen = 12;
/* Convert initial characters to fraction */
base = rangehi - rangelo + 1;