rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Apply auto-vectorization to the inner loop of div_var_fast(). 

This loop is basically the same as the inner loop of mul_var(), which
was auto-vectorized in commit 8870917623, but the compiler will only
consider auto-vectorizing the div_var_fast() loop if the assignment
target div[qi + i] is replaced by div_qi[i], where div_qi = &div[qi].

Additionally, since the compiler doesn't know that qdigit is
guaranteed to fit in a 16-bit NumericDigit, cast it to NumericDigit
before multiplying to make the resulting auto-vectorized code more
efficient (avoiding unnecessary multiplication of the high 16 bits).

While at it, per suggestion from Tom Lane, change var1digit in
mul_var() to be a NumericDigit rather than an int for the same
reason. This actually makes no difference with modern gcc, but it
might help other compilers generate more efficient assembly.

Dean Rasheed, reviewed by Tom Lane.

Discussion: https://postgr.es/m/CAEZATCVwsBi-ND-t82Cuuh1=8ee6jdOpzsmGN+CUZB6yjLg9jw@mail.gmail.com
This commit is contained in:
Dean Rasheed 2022-02-27 10:15:46 +00:00
parent 6b04abdfc5
commit e3d41d08a1
1 changed files with 11 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
src/backend/utils/adt/numeric.c
View File

@ -8323,7 +8323,7 @@ mul_var(const NumericVar *var1, const NumericVar *var2, NumericVar *result,
*/
for (i1 = Min(var1ndigits - 1, res_ndigits - 3); i1 >= 0; i1--)
{
int var1digit = var1digits[i1];
NumericDigit var1digit = var1digits[i1];
if (var1digit == 0)
continue;
@ -8908,13 +8908,22 @@ div_var_fast(const NumericVar *var1, const NumericVar *var2,
* which would make the new value simply div[qi] mod vardigits[0].
* The lower-order terms in qdigit can change this result by not
* more than about twice INT_MAX/NBASE, so overflow is impossible.
*
* This inner loop is the performance bottleneck for division, so
* code it in the same way as the inner loop of mul_var() so that
* it can be auto-vectorized. We cast qdigit to NumericDigit
* before multiplying to allow the compiler to generate more
* efficient code (using 16-bit multiplication), which is safe
* since we know that the quotient digit is off by at most one, so
* there is no overflow risk.
*/
if (qdigit != 0)
{
int istop = Min(var2ndigits, div_ndigits - qi + 1);
int *div_qi = &div[qi];
for (i = 0; i < istop; i++)
div[qi + i] -= qdigit * var2digits[i];
div_qi[i] -= ((NumericDigit) qdigit) * var2digits[i];
}
}