mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-10-02 18:21:17 +02:00
Be more wary about 32-bit integer overflow in pg_stat_statements.
We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files. It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:
1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits. We still need overflow guards on its use, but
this helps.
2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize. If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.
3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms. It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.
Per report from Bruno da Silva. I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place. But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.
(See also commit 8bbe4cbd9
, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)
This issue is old, and is primarily a problem for old platforms,
so back-patch.
Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
This commit is contained in:
parent
331f8b851c
commit
6b67db10c3
@ -408,7 +408,7 @@ _PG_init(void)
|
|||||||
&pgss_max,
|
&pgss_max,
|
||||||
5000,
|
5000,
|
||||||
100,
|
100,
|
||||||
INT_MAX,
|
INT_MAX / 2,
|
||||||
PGC_POSTMASTER,
|
PGC_POSTMASTER,
|
||||||
0,
|
0,
|
||||||
NULL,
|
NULL,
|
||||||
@ -2053,6 +2053,18 @@ qtext_store(const char *query, int query_len,
|
|||||||
|
|
||||||
*query_offset = off;
|
*query_offset = off;
|
||||||
|
|
||||||
|
/*
|
||||||
|
* Don't allow the file to grow larger than what qtext_load_file can
|
||||||
|
* (theoretically) handle. This has been seen to be reachable on 32-bit
|
||||||
|
* platforms.
|
||||||
|
*/
|
||||||
|
if (unlikely(query_len >= MaxAllocHugeSize - off))
|
||||||
|
{
|
||||||
|
errno = EFBIG; /* not quite right, but it'll do */
|
||||||
|
fd = -1;
|
||||||
|
goto error;
|
||||||
|
}
|
||||||
|
|
||||||
/* Now write the data into the successfully-reserved part of the file */
|
/* Now write the data into the successfully-reserved part of the file */
|
||||||
fd = OpenTransientFile(PGSS_TEXT_FILE, O_RDWR | O_CREAT | PG_BINARY);
|
fd = OpenTransientFile(PGSS_TEXT_FILE, O_RDWR | O_CREAT | PG_BINARY);
|
||||||
if (fd < 0)
|
if (fd < 0)
|
||||||
@ -2238,8 +2250,14 @@ need_gc_qtexts(void)
|
|||||||
SpinLockRelease(&s->mutex);
|
SpinLockRelease(&s->mutex);
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Don't proceed if file does not exceed 512 bytes per possible entry */
|
/*
|
||||||
if (extent < 512 * pgss_max)
|
* Don't proceed if file does not exceed 512 bytes per possible entry.
|
||||||
|
*
|
||||||
|
* Here and in the next test, 32-bit machines have overflow hazards if
|
||||||
|
* pgss_max and/or mean_query_len are large. Force the multiplications
|
||||||
|
* and comparisons to be done in uint64 arithmetic to forestall trouble.
|
||||||
|
*/
|
||||||
|
if ((uint64) extent < (uint64) 512 * pgss_max)
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
/*
|
/*
|
||||||
@ -2249,7 +2267,7 @@ need_gc_qtexts(void)
|
|||||||
* query length in order to prevent garbage collection from thrashing
|
* query length in order to prevent garbage collection from thrashing
|
||||||
* uselessly.
|
* uselessly.
|
||||||
*/
|
*/
|
||||||
if (extent < pgss->mean_query_len * pgss_max * 2)
|
if ((uint64) extent < (uint64) pgss->mean_query_len * pgss_max * 2)
|
||||||
return false;
|
return false;
|
||||||
|
|
||||||
return true;
|
return true;
|
||||||
|
Loading…
Reference in New Issue
Block a user