2009-01-04 23:19:59 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* pg_stat_statements.c
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
* Track statement planning and execution times as well as resource
|
|
|
|
* usage across a whole database cluster.
|
2009-01-04 23:19:59 +01:00
|
|
|
*
|
2020-07-05 15:37:57 +02:00
|
|
|
* Execution costs are totaled for each distinct source query, and kept in
|
2014-05-06 18:12:18 +02:00
|
|
|
* a shared hashtable. (We track only as many distinct queries as will fit
|
2012-03-29 03:00:31 +02:00
|
|
|
* in the designated amount of shared memory.)
|
|
|
|
*
|
2021-04-07 19:06:47 +02:00
|
|
|
* Starting in Postgres 9.2, this module normalized query entries. As of
|
|
|
|
* Postgres 14, the normalization is done by the core if compute_query_id is
|
|
|
|
* enabled, or optionally by third-party modules.
|
2012-03-29 03:00:31 +02:00
|
|
|
*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* To facilitate presenting entries to users, we create "representative" query
|
2017-03-28 02:14:36 +02:00
|
|
|
* strings in which constants are replaced with parameter symbols ($n), to
|
|
|
|
* make it clearer what a normalized entry can represent. To save on shared
|
|
|
|
* memory, and to avoid having to truncate oversized query strings, we store
|
|
|
|
* these strings in a temporary external query-texts file. Offsets into this
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* file are kept in shared memory.
|
|
|
|
*
|
2009-01-04 23:19:59 +01:00
|
|
|
* Note about locking issues: to create or delete an entry in the shared
|
|
|
|
* hashtable, one must hold pgss->lock exclusively. Modifying any field
|
|
|
|
* in an entry except the counters requires the same. To look up an entry,
|
|
|
|
* one must hold the lock shared. To read or update the counters within
|
|
|
|
* an entry, one must hold the lock shared or exclusive (so the entry doesn't
|
|
|
|
* disappear!) and also take the entry's mutex spinlock.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* The shared state variable pgss->extent (the next free spot in the external
|
|
|
|
* query-text file) should be accessed only while holding either the
|
|
|
|
* pgss->mutex spinlock, or exclusive lock on pgss->lock. We use the mutex to
|
|
|
|
* allow reserving file space while holding only shared lock on pgss->lock.
|
|
|
|
* Rewriting the entire external query-text file, eg for garbage collection,
|
|
|
|
* requires holding pgss->lock exclusively; this allows individual entries
|
|
|
|
* in the file to be read or written while holding only shared lock.
|
2009-01-04 23:19:59 +01:00
|
|
|
*
|
|
|
|
*
|
2022-01-08 01:04:57 +01:00
|
|
|
* Copyright (c) 2008-2022, PostgreSQL Global Development Group
|
2009-01-04 23:19:59 +01:00
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* contrib/pg_stat_statements/pg_stat_statements.c
|
2009-01-04 23:19:59 +01:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#include "postgres.h"
|
|
|
|
|
2015-03-28 14:22:51 +01:00
|
|
|
#include <math.h>
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
#include <sys/stat.h>
|
2009-01-04 23:19:59 +01:00
|
|
|
#include <unistd.h>
|
|
|
|
|
2021-04-08 17:16:01 +02:00
|
|
|
#include "access/parallel.h"
|
2017-03-30 20:18:53 +02:00
|
|
|
#include "catalog/pg_authid.h"
|
2020-02-27 04:55:41 +01:00
|
|
|
#include "common/hashfn.h"
|
2009-01-04 23:19:59 +01:00
|
|
|
#include "executor/instrument.h"
|
2010-01-08 01:38:20 +01:00
|
|
|
#include "funcapi.h"
|
2022-04-08 13:51:01 +02:00
|
|
|
#include "jit/jit.h"
|
2009-01-04 23:19:59 +01:00
|
|
|
#include "mb/pg_wchar.h"
|
|
|
|
#include "miscadmin.h"
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
#include "optimizer/planner.h"
|
2012-03-29 03:00:31 +02:00
|
|
|
#include "parser/analyze.h"
|
|
|
|
#include "parser/parsetree.h"
|
|
|
|
#include "parser/scanner.h"
|
2017-01-14 22:17:30 +01:00
|
|
|
#include "parser/scansup.h"
|
2014-02-03 15:19:49 +01:00
|
|
|
#include "pgstat.h"
|
2009-01-04 23:19:59 +01:00
|
|
|
#include "storage/fd.h"
|
|
|
|
#include "storage/ipc.h"
|
2021-04-03 05:01:14 +02:00
|
|
|
#include "storage/lwlock.h"
|
|
|
|
#include "storage/shmem.h"
|
2011-09-04 07:13:16 +02:00
|
|
|
#include "storage/spin.h"
|
2009-12-15 21:04:49 +01:00
|
|
|
#include "tcop/utility.h"
|
2018-04-15 02:12:14 +02:00
|
|
|
#include "utils/acl.h"
|
2009-01-04 23:19:59 +01:00
|
|
|
#include "utils/builtins.h"
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
#include "utils/queryjumble.h"
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
#include "utils/memutils.h"
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
#include "utils/timestamp.h"
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
PG_MODULE_MAGIC;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Location of permanent stats file (valid when database is shut down) */
|
2014-06-04 05:09:45 +02:00
|
|
|
#define PGSS_DUMP_FILE PGSTAT_STAT_PERMANENT_DIRECTORY "/pg_stat_statements.stat"
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
2022-04-07 06:29:46 +02:00
|
|
|
* Location of external query text file.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
2014-02-03 15:19:49 +01:00
|
|
|
#define PGSS_TEXT_FILE PG_STAT_TMP_DIR "/pgss_query_texts.stat"
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
/* Magic number identifying the stats file format */
|
2022-04-08 06:12:07 +02:00
|
|
|
static const uint32 PGSS_FILE_HEADER = 0x20220408;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
2013-12-07 18:06:02 +01:00
|
|
|
/* PostgreSQL major version number, changes in which invalidate all entries */
|
|
|
|
static const uint32 PGSS_PG_MAJOR_VERSION = PG_VERSION_NUM / 100;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* XXX: Should USAGE_EXEC reflect execution time and/or buffer usage? */
|
|
|
|
#define USAGE_EXEC(duration) (1.0)
|
|
|
|
#define USAGE_INIT (1.0) /* including initial planning */
|
2012-04-08 21:49:47 +02:00
|
|
|
#define ASSUMED_MEDIAN_INIT (10.0) /* initial assumed median usage */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
#define ASSUMED_LENGTH_INIT 1024 /* initial assumed mean query length */
|
2009-01-04 23:19:59 +01:00
|
|
|
#define USAGE_DECREASE_FACTOR (0.99) /* decreased every entry_dealloc */
|
2012-04-08 21:49:47 +02:00
|
|
|
#define STICKY_DECREASE_FACTOR (0.50) /* factor for sticky entries */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
#define USAGE_DEALLOC_PERCENT 5 /* free this % of entries at once */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
#define IS_STICKY(c) ((c.calls[PGSS_PLAN] + c.calls[PGSS_EXEC]) == 0)
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
/*
|
|
|
|
* Utility statements that pgss_ProcessUtility and pgss_post_parse_analyze
|
|
|
|
* ignores.
|
|
|
|
*/
|
|
|
|
#define PGSS_HANDLED_UTILITY(n) (!IsA(n, ExecuteStmt) && \
|
|
|
|
!IsA(n, PrepareStmt) && \
|
|
|
|
!IsA(n, DeallocateStmt))
|
|
|
|
|
2013-12-07 18:06:02 +01:00
|
|
|
/*
|
|
|
|
* Extension version number, for supporting older extension versions' objects
|
|
|
|
*/
|
|
|
|
typedef enum pgssVersion
|
|
|
|
{
|
|
|
|
PGSS_V1_0 = 0,
|
|
|
|
PGSS_V1_1,
|
2015-03-27 20:43:22 +01:00
|
|
|
PGSS_V1_2,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PGSS_V1_3,
|
2021-04-08 10:23:10 +02:00
|
|
|
PGSS_V1_8,
|
2022-04-08 06:12:07 +02:00
|
|
|
PGSS_V1_9,
|
|
|
|
PGSS_V1_10
|
2013-12-07 18:06:02 +01:00
|
|
|
} pgssVersion;
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
typedef enum pgssStoreKind
|
|
|
|
{
|
|
|
|
PGSS_INVALID = -1,
|
|
|
|
|
|
|
|
/*
|
|
|
|
* PGSS_PLAN and PGSS_EXEC must be respectively 0 and 1 as they're used to
|
|
|
|
* reference the underlying values in the arrays in the Counters struct,
|
|
|
|
* and this order is required in pg_stat_statements_internal().
|
|
|
|
*/
|
|
|
|
PGSS_PLAN = 0,
|
|
|
|
PGSS_EXEC,
|
|
|
|
|
|
|
|
PGSS_NUMKIND /* Must be last value of this enum */
|
|
|
|
} pgssStoreKind;
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
2012-03-29 03:00:31 +02:00
|
|
|
* Hashtable key that defines the identity of a hashtable entry. We separate
|
|
|
|
* queries by user and by database even if they are otherwise identical.
|
2017-10-20 15:40:17 +02:00
|
|
|
*
|
2021-04-08 10:23:10 +02:00
|
|
|
* If you add a new key to this struct, make sure to teach pgss_store() to
|
|
|
|
* zero the padding bytes. Otherwise, things will break, because pgss_hash is
|
|
|
|
* created using HASH_BLOBS, and thus tag_hash is used to hash this.
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
typedef struct pgssHashKey
|
|
|
|
{
|
|
|
|
Oid userid; /* user OID */
|
|
|
|
Oid dbid; /* database OID */
|
2017-10-12 01:52:46 +02:00
|
|
|
uint64 queryid; /* query identifier */
|
2021-04-08 10:23:10 +02:00
|
|
|
bool toplevel; /* query executed at top level */
|
2009-01-04 23:19:59 +01:00
|
|
|
} pgssHashKey;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* The actual stats counters kept within pgssEntry.
|
|
|
|
*/
|
|
|
|
typedef struct Counters
|
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
int64 calls[PGSS_NUMKIND]; /* # of times planned/executed */
|
|
|
|
double total_time[PGSS_NUMKIND]; /* total planning/execution time,
|
|
|
|
* in msec */
|
|
|
|
double min_time[PGSS_NUMKIND]; /* minimum planning/execution time in
|
|
|
|
* msec */
|
|
|
|
double max_time[PGSS_NUMKIND]; /* maximum planning/execution time in
|
|
|
|
* msec */
|
|
|
|
double mean_time[PGSS_NUMKIND]; /* mean planning/execution time in
|
|
|
|
* msec */
|
|
|
|
double sum_var_time[PGSS_NUMKIND]; /* sum of variances in
|
|
|
|
* planning/execution time in msec */
|
2010-02-26 03:01:40 +01:00
|
|
|
int64 rows; /* total # of retrieved or affected rows */
|
2012-06-10 21:20:04 +02:00
|
|
|
int64 shared_blks_hit; /* # of shared buffer hits */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
int64 shared_blks_read; /* # of shared disk blocks read */
|
2012-02-23 02:33:05 +01:00
|
|
|
int64 shared_blks_dirtied; /* # of shared disk blocks dirtied */
|
2010-02-26 03:01:40 +01:00
|
|
|
int64 shared_blks_written; /* # of shared disk blocks written */
|
2012-06-10 21:20:04 +02:00
|
|
|
int64 local_blks_hit; /* # of local buffer hits */
|
|
|
|
int64 local_blks_read; /* # of local disk blocks read */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
int64 local_blks_dirtied; /* # of local disk blocks dirtied */
|
|
|
|
int64 local_blks_written; /* # of local disk blocks written */
|
2012-06-10 21:20:04 +02:00
|
|
|
int64 temp_blks_read; /* # of temp blocks read */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
int64 temp_blks_written; /* # of temp blocks written */
|
2022-04-08 06:12:07 +02:00
|
|
|
double blk_read_time; /* time spent reading blocks, in msec */
|
|
|
|
double blk_write_time; /* time spent writing blocks, in msec */
|
|
|
|
double temp_blk_read_time; /* time spent reading temp blocks, in msec */
|
|
|
|
double temp_blk_write_time; /* time spent writing temp blocks, in
|
|
|
|
* msec */
|
2012-06-10 21:20:04 +02:00
|
|
|
double usage; /* usage factor */
|
2020-04-05 04:04:04 +02:00
|
|
|
int64 wal_records; /* # of WAL records generated */
|
2020-05-05 04:30:53 +02:00
|
|
|
int64 wal_fpi; /* # of WAL full page images generated */
|
2020-12-24 09:05:49 +01:00
|
|
|
uint64 wal_bytes; /* total amount of WAL generated in bytes */
|
2022-04-08 13:51:01 +02:00
|
|
|
int64 jit_functions; /* total number of JIT functions emitted */
|
|
|
|
double jit_generation_time; /* total time to generate jit code */
|
|
|
|
int64 jit_inlining_count; /* number of times inlining time has been
|
|
|
|
* > 0 */
|
|
|
|
double jit_inlining_time; /* total time to inline jit code */
|
|
|
|
int64 jit_optimization_count; /* number of times optimization time
|
|
|
|
* has been > 0 */
|
|
|
|
double jit_optimization_time; /* total time to optimize jit code */
|
|
|
|
int64 jit_emission_count; /* number of times emission time has been
|
|
|
|
* > 0 */
|
|
|
|
double jit_emission_time; /* total time to emit jit code */
|
2009-01-04 23:19:59 +01:00
|
|
|
} Counters;
|
|
|
|
|
2020-11-26 13:18:05 +01:00
|
|
|
/*
|
|
|
|
* Global statistics for pg_stat_statements
|
|
|
|
*/
|
|
|
|
typedef struct pgssGlobalStats
|
|
|
|
{
|
|
|
|
int64 dealloc; /* # of times entries were deallocated */
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
TimestampTz stats_reset; /* timestamp with all stats reset */
|
2020-11-26 13:18:05 +01:00
|
|
|
} pgssGlobalStats;
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* Statistics per statement
|
|
|
|
*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Note: in event of a failure in garbage collection of the query text file,
|
|
|
|
* we reset query_offset to zero and query_len to -1. This will be seen as
|
|
|
|
* an invalid state by qtext_fetch().
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
typedef struct pgssEntry
|
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
pgssHashKey key; /* hash key of entry - MUST BE FIRST */
|
2009-01-04 23:19:59 +01:00
|
|
|
Counters counters; /* the statistics for this query */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Size query_offset; /* query text offset in external file */
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
int query_len; /* # of valid bytes in query string, or -1 */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
int encoding; /* query text encoding */
|
2009-01-04 23:19:59 +01:00
|
|
|
slock_t mutex; /* protects the counters only */
|
|
|
|
} pgssEntry;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Global shared state
|
|
|
|
*/
|
|
|
|
typedef struct pgssSharedState
|
|
|
|
{
|
2014-01-27 17:07:44 +01:00
|
|
|
LWLock *lock; /* protects hashtable search/modification */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
double cur_median_usage; /* current median usage in hashtable */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Size mean_query_len; /* current mean entry text length */
|
|
|
|
slock_t mutex; /* protects following fields only: */
|
|
|
|
Size extent; /* current extent of query file */
|
|
|
|
int n_writers; /* number of active writers to query file */
|
|
|
|
int gc_count; /* query file garbage collection cycle count */
|
2020-11-26 13:18:05 +01:00
|
|
|
pgssGlobalStats stats; /* global statistics for pgss */
|
2009-01-04 23:19:59 +01:00
|
|
|
} pgssSharedState;
|
|
|
|
|
|
|
|
/*---- Local variables ----*/
|
|
|
|
|
2012-03-29 22:42:09 +02:00
|
|
|
/* Current nesting depth of ExecutorRun+ProcessUtility calls */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
static int exec_nested_level = 0;
|
|
|
|
|
|
|
|
/* Current nesting depth of planner calls */
|
|
|
|
static int plan_nested_level = 0;
|
2009-06-11 16:49:15 +02:00
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/* Saved hook values in case of unload */
|
2022-05-13 15:31:06 +02:00
|
|
|
static shmem_request_hook_type prev_shmem_request_hook = NULL;
|
2009-06-11 16:49:15 +02:00
|
|
|
static shmem_startup_hook_type prev_shmem_startup_hook = NULL;
|
2012-03-29 03:00:31 +02:00
|
|
|
static post_parse_analyze_hook_type prev_post_parse_analyze_hook = NULL;
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
static planner_hook_type prev_planner_hook = NULL;
|
2009-06-11 16:49:15 +02:00
|
|
|
static ExecutorStart_hook_type prev_ExecutorStart = NULL;
|
|
|
|
static ExecutorRun_hook_type prev_ExecutorRun = NULL;
|
2011-02-27 19:43:29 +01:00
|
|
|
static ExecutorFinish_hook_type prev_ExecutorFinish = NULL;
|
2009-06-11 16:49:15 +02:00
|
|
|
static ExecutorEnd_hook_type prev_ExecutorEnd = NULL;
|
2009-12-15 21:04:49 +01:00
|
|
|
static ProcessUtility_hook_type prev_ProcessUtility = NULL;
|
2009-06-11 16:49:15 +02:00
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/* Links to shared memory state */
|
2009-06-11 16:49:15 +02:00
|
|
|
static pgssSharedState *pgss = NULL;
|
|
|
|
static HTAB *pgss_hash = NULL;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/*---- GUC variables ----*/
|
|
|
|
|
|
|
|
typedef enum
|
|
|
|
{
|
|
|
|
PGSS_TRACK_NONE, /* track no statements */
|
|
|
|
PGSS_TRACK_TOP, /* only top level statements */
|
2009-12-15 21:04:49 +01:00
|
|
|
PGSS_TRACK_ALL /* all statements, including nested ones */
|
2017-06-21 20:39:04 +02:00
|
|
|
} PGSSTrackLevel;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2010-01-08 01:38:20 +01:00
|
|
|
static const struct config_enum_entry track_options[] =
|
|
|
|
{
|
2009-01-04 23:19:59 +01:00
|
|
|
{"none", PGSS_TRACK_NONE, false},
|
|
|
|
{"top", PGSS_TRACK_TOP, false},
|
|
|
|
{"all", PGSS_TRACK_ALL, false},
|
|
|
|
{NULL, 0, false}
|
|
|
|
};
|
|
|
|
|
|
|
|
static int pgss_max; /* max # statements to track */
|
|
|
|
static int pgss_track; /* tracking level */
|
2010-02-26 03:01:40 +01:00
|
|
|
static bool pgss_track_utility; /* whether to track utility commands */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
static bool pgss_track_planning; /* whether to track planning duration */
|
2009-01-04 23:19:59 +01:00
|
|
|
static bool pgss_save; /* whether to save stats across shutdown */
|
|
|
|
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
#define pgss_enabled(level) \
|
2021-04-08 17:16:01 +02:00
|
|
|
(!IsParallelWorker() && \
|
2009-01-04 23:19:59 +01:00
|
|
|
(pgss_track == PGSS_TRACK_ALL || \
|
2021-04-08 17:16:01 +02:00
|
|
|
(pgss_track == PGSS_TRACK_TOP && (level) == 0)))
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
#define record_gc_qtexts() \
|
|
|
|
do { \
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss; \
|
|
|
|
SpinLockAcquire(&s->mutex); \
|
|
|
|
s->gc_count++; \
|
|
|
|
SpinLockRelease(&s->mutex); \
|
|
|
|
} while(0)
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*---- Function declarations ----*/
|
|
|
|
|
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_reset);
|
2019-01-11 04:20:09 +01:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_reset_1_7);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_1_2);
|
2015-03-27 20:43:22 +01:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_1_3);
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_1_8);
|
2021-04-08 15:15:17 +02:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_1_9);
|
2022-04-08 06:12:07 +02:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_1_10);
|
2009-01-04 23:19:59 +01:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements);
|
2020-11-26 13:18:05 +01:00
|
|
|
PG_FUNCTION_INFO_V1(pg_stat_statements_info);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2022-05-13 15:31:06 +02:00
|
|
|
static void pgss_shmem_request(void);
|
2009-01-04 23:19:59 +01:00
|
|
|
static void pgss_shmem_startup(void);
|
|
|
|
static void pgss_shmem_shutdown(int code, Datum arg);
|
2021-04-07 19:06:47 +02:00
|
|
|
static void pgss_post_parse_analyze(ParseState *pstate, Query *query,
|
|
|
|
JumbleState *jstate);
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
static PlannedStmt *pgss_planner(Query *parse,
|
|
|
|
const char *query_string,
|
|
|
|
int cursorOptions,
|
|
|
|
ParamListInfo boundParams);
|
2009-01-04 23:19:59 +01:00
|
|
|
static void pgss_ExecutorStart(QueryDesc *queryDesc, int eflags);
|
|
|
|
static void pgss_ExecutorRun(QueryDesc *queryDesc,
|
2019-05-22 19:04:48 +02:00
|
|
|
ScanDirection direction,
|
|
|
|
uint64 count, bool execute_once);
|
2011-02-27 19:43:29 +01:00
|
|
|
static void pgss_ExecutorFinish(QueryDesc *queryDesc);
|
2009-01-04 23:19:59 +01:00
|
|
|
static void pgss_ExecutorEnd(QueryDesc *queryDesc);
|
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
|
|
|
static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
bool readOnlyTree,
|
2019-05-22 19:04:48 +02:00
|
|
|
ProcessUtilityContext context, ParamListInfo params,
|
|
|
|
QueryEnvironment *queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
DestReceiver *dest, QueryCompletion *qc);
|
2017-10-12 01:52:46 +02:00
|
|
|
static void pgss_store(const char *query, uint64 queryId,
|
2019-05-22 19:04:48 +02:00
|
|
|
int query_location, int query_len,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
pgssStoreKind kind,
|
2019-05-22 19:04:48 +02:00
|
|
|
double total_time, uint64 rows,
|
|
|
|
const BufferUsage *bufusage,
|
2020-04-05 04:04:04 +02:00
|
|
|
const WalUsage *walusage,
|
2022-04-08 13:51:01 +02:00
|
|
|
const struct JitInstrumentation *jitusage,
|
2021-04-07 19:06:47 +02:00
|
|
|
JumbleState *jstate);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
static void pg_stat_statements_internal(FunctionCallInfo fcinfo,
|
2019-05-22 19:04:48 +02:00
|
|
|
pgssVersion api_version,
|
|
|
|
bool showtext);
|
2009-06-11 16:49:15 +02:00
|
|
|
static Size pgss_memsize(void);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
static pgssEntry *entry_alloc(pgssHashKey *key, Size query_offset, int query_len,
|
2019-05-22 19:04:48 +02:00
|
|
|
int encoding, bool sticky);
|
2009-01-04 23:19:59 +01:00
|
|
|
static void entry_dealloc(void);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
static bool qtext_store(const char *query, int query_len,
|
2019-05-22 19:04:48 +02:00
|
|
|
Size *query_offset, int *gc_count);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
static char *qtext_load_file(Size *buffer_size);
|
|
|
|
static char *qtext_fetch(Size query_offset, int query_len,
|
2019-05-22 19:04:48 +02:00
|
|
|
char *buffer, Size buffer_size);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
static bool need_gc_qtexts(void);
|
|
|
|
static void gc_qtexts(void);
|
2019-01-11 04:20:09 +01:00
|
|
|
static void entry_reset(Oid userid, Oid dbid, uint64 queryid);
|
2021-04-07 19:06:47 +02:00
|
|
|
static char *generate_normalized_query(JumbleState *jstate, const char *query,
|
2020-09-08 10:08:46 +02:00
|
|
|
int query_loc, int *query_len_p);
|
2021-04-07 19:06:47 +02:00
|
|
|
static void fill_in_constant_lengths(JumbleState *jstate, const char *query,
|
2019-05-22 19:04:48 +02:00
|
|
|
int query_loc);
|
2012-03-29 03:00:31 +02:00
|
|
|
static int comp_location(const void *a, const void *b);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Module load callback
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
_PG_init(void)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* In order to create our shared memory area, we have to be loaded via
|
2009-06-11 16:49:15 +02:00
|
|
|
* shared_preload_libraries. If not, fall out without hooking into any of
|
|
|
|
* the main system. (We don't throw error here because it seems useful to
|
|
|
|
* allow the pg_stat_statements functions to be created even when the
|
|
|
|
* module isn't active. The functions must protect themselves against
|
|
|
|
* being called then, however.)
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
if (!process_shared_preload_libraries_in_progress)
|
|
|
|
return;
|
|
|
|
|
2021-05-15 20:13:09 +02:00
|
|
|
/*
|
|
|
|
* Inform the postmaster that we want to enable query_id calculation if
|
|
|
|
* compute_query_id is set to auto.
|
|
|
|
*/
|
|
|
|
EnableQueryId();
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* Define (or redefine) custom GUC variables.
|
|
|
|
*/
|
|
|
|
DefineCustomIntVariable("pg_stat_statements.max",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
"Sets the maximum number of statements tracked by pg_stat_statements.",
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL,
|
|
|
|
&pgss_max,
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
5000,
|
2009-01-04 23:19:59 +01:00
|
|
|
100,
|
Be more wary about 32-bit integer overflow in pg_stat_statements.
We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files. It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:
1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits. We still need overflow guards on its use, but
this helps.
2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize. If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.
3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms. It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.
Per report from Bruno da Silva. I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place. But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.
(See also commit 8bbe4cbd9, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)
This issue is old, and is primarily a problem for old platforms,
so back-patch.
Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
2022-08-03 00:05:34 +02:00
|
|
|
INT_MAX / 2,
|
2009-01-04 23:19:59 +01:00
|
|
|
PGC_POSTMASTER,
|
|
|
|
0,
|
|
|
|
NULL,
|
2011-04-07 06:11:01 +02:00
|
|
|
NULL,
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL);
|
|
|
|
|
|
|
|
DefineCustomEnumVariable("pg_stat_statements.track",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
"Selects which statements are tracked by pg_stat_statements.",
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL,
|
|
|
|
&pgss_track,
|
|
|
|
PGSS_TRACK_TOP,
|
|
|
|
track_options,
|
|
|
|
PGC_SUSET,
|
|
|
|
0,
|
|
|
|
NULL,
|
2011-04-07 06:11:01 +02:00
|
|
|
NULL,
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL);
|
|
|
|
|
2009-12-15 21:04:49 +01:00
|
|
|
DefineCustomBoolVariable("pg_stat_statements.track_utility",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
"Selects whether utility commands are tracked by pg_stat_statements.",
|
2009-12-15 21:04:49 +01:00
|
|
|
NULL,
|
|
|
|
&pgss_track_utility,
|
|
|
|
true,
|
|
|
|
PGC_SUSET,
|
|
|
|
0,
|
|
|
|
NULL,
|
2011-04-07 06:11:01 +02:00
|
|
|
NULL,
|
2009-12-15 21:04:49 +01:00
|
|
|
NULL);
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
DefineCustomBoolVariable("pg_stat_statements.track_planning",
|
|
|
|
"Selects whether planning duration is tracked by pg_stat_statements.",
|
|
|
|
NULL,
|
|
|
|
&pgss_track_planning,
|
2020-07-03 04:35:22 +02:00
|
|
|
false,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PGC_SUSET,
|
|
|
|
0,
|
|
|
|
NULL,
|
|
|
|
NULL,
|
|
|
|
NULL);
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
DefineCustomBoolVariable("pg_stat_statements.save",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
"Save pg_stat_statements statistics across server shutdowns.",
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL,
|
|
|
|
&pgss_save,
|
|
|
|
true,
|
|
|
|
PGC_SIGHUP,
|
|
|
|
0,
|
|
|
|
NULL,
|
2011-04-07 06:11:01 +02:00
|
|
|
NULL,
|
2009-01-04 23:19:59 +01:00
|
|
|
NULL);
|
|
|
|
|
2022-02-21 20:10:15 +01:00
|
|
|
MarkGUCPrefixReserved("pg_stat_statements");
|
2009-01-05 14:35:38 +01:00
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* Install hooks.
|
|
|
|
*/
|
2022-05-13 15:31:06 +02:00
|
|
|
prev_shmem_request_hook = shmem_request_hook;
|
|
|
|
shmem_request_hook = pgss_shmem_request;
|
2009-01-04 23:19:59 +01:00
|
|
|
prev_shmem_startup_hook = shmem_startup_hook;
|
|
|
|
shmem_startup_hook = pgss_shmem_startup;
|
2012-03-29 03:00:31 +02:00
|
|
|
prev_post_parse_analyze_hook = post_parse_analyze_hook;
|
|
|
|
post_parse_analyze_hook = pgss_post_parse_analyze;
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
prev_planner_hook = planner_hook;
|
|
|
|
planner_hook = pgss_planner;
|
2009-01-04 23:19:59 +01:00
|
|
|
prev_ExecutorStart = ExecutorStart_hook;
|
|
|
|
ExecutorStart_hook = pgss_ExecutorStart;
|
|
|
|
prev_ExecutorRun = ExecutorRun_hook;
|
|
|
|
ExecutorRun_hook = pgss_ExecutorRun;
|
2011-02-27 19:43:29 +01:00
|
|
|
prev_ExecutorFinish = ExecutorFinish_hook;
|
|
|
|
ExecutorFinish_hook = pgss_ExecutorFinish;
|
2009-01-04 23:19:59 +01:00
|
|
|
prev_ExecutorEnd = ExecutorEnd_hook;
|
|
|
|
ExecutorEnd_hook = pgss_ExecutorEnd;
|
2009-12-15 21:04:49 +01:00
|
|
|
prev_ProcessUtility = ProcessUtility_hook;
|
|
|
|
ProcessUtility_hook = pgss_ProcessUtility;
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
2022-05-13 15:31:06 +02:00
|
|
|
/*
|
|
|
|
* shmem_request hook: request additional shared resources. We'll allocate or
|
|
|
|
* attach to the shared resources in pgss_shmem_startup().
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_shmem_request(void)
|
|
|
|
{
|
|
|
|
if (prev_shmem_request_hook)
|
|
|
|
prev_shmem_request_hook();
|
|
|
|
|
|
|
|
RequestAddinShmemSpace(pgss_memsize());
|
|
|
|
RequestNamedLWLockTranche("pg_stat_statements", 1);
|
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* shmem_startup hook: allocate or attach to shared memory,
|
|
|
|
* then load any pre-existing statistics from file.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Also create and load the query-texts file, which is expected to exist
|
|
|
|
* (even if empty) while the module is enabled.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_shmem_startup(void)
|
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
bool found;
|
2009-01-04 23:19:59 +01:00
|
|
|
HASHCTL info;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
FILE *file = NULL;
|
|
|
|
FILE *qfile = NULL;
|
2009-01-04 23:19:59 +01:00
|
|
|
uint32 header;
|
|
|
|
int32 num;
|
2013-12-07 18:06:02 +01:00
|
|
|
int32 pgver;
|
2009-01-04 23:19:59 +01:00
|
|
|
int32 i;
|
|
|
|
int buffer_size;
|
|
|
|
char *buffer = NULL;
|
|
|
|
|
|
|
|
if (prev_shmem_startup_hook)
|
|
|
|
prev_shmem_startup_hook();
|
|
|
|
|
|
|
|
/* reset in case this is a restart within the postmaster */
|
|
|
|
pgss = NULL;
|
|
|
|
pgss_hash = NULL;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Create or attach to the shared memory state, including hash table
|
|
|
|
*/
|
|
|
|
LWLockAcquire(AddinShmemInitLock, LW_EXCLUSIVE);
|
|
|
|
|
|
|
|
pgss = ShmemInitStruct("pg_stat_statements",
|
|
|
|
sizeof(pgssSharedState),
|
|
|
|
&found);
|
|
|
|
|
|
|
|
if (!found)
|
2009-06-11 16:49:15 +02:00
|
|
|
{
|
2009-01-04 23:19:59 +01:00
|
|
|
/* First time through ... */
|
2016-02-04 22:43:04 +01:00
|
|
|
pgss->lock = &(GetNamedLWLockTranche("pg_stat_statements"))->lock;
|
2012-04-08 21:49:47 +02:00
|
|
|
pgss->cur_median_usage = ASSUMED_MEDIAN_INIT;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
pgss->mean_query_len = ASSUMED_LENGTH_INIT;
|
|
|
|
SpinLockInit(&pgss->mutex);
|
|
|
|
pgss->extent = 0;
|
|
|
|
pgss->n_writers = 0;
|
|
|
|
pgss->gc_count = 0;
|
2020-11-26 13:18:05 +01:00
|
|
|
pgss->stats.dealloc = 0;
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
pgss->stats.stats_reset = GetCurrentTimestamp();
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
info.keysize = sizeof(pgssHashKey);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
info.entrysize = sizeof(pgssEntry);
|
2009-01-04 23:19:59 +01:00
|
|
|
pgss_hash = ShmemInitHash("pg_stat_statements hash",
|
|
|
|
pgss_max, pgss_max,
|
|
|
|
&info,
|
2017-10-12 01:52:46 +02:00
|
|
|
HASH_ELEM | HASH_BLOBS);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
LWLockRelease(AddinShmemInitLock);
|
|
|
|
|
|
|
|
/*
|
2009-06-11 16:49:15 +02:00
|
|
|
* If we're in the postmaster (or a standalone backend...), set up a shmem
|
|
|
|
* exit hook to dump the statistics to disk.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
if (!IsUnderPostmaster)
|
|
|
|
on_shmem_exit(pgss_shmem_shutdown, (Datum) 0);
|
|
|
|
|
|
|
|
/*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Done if some other process already completed our initialization.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (found)
|
2009-01-04 23:19:59 +01:00
|
|
|
return;
|
|
|
|
|
2009-07-27 06:09:55 +02:00
|
|
|
/*
|
|
|
|
* Note: we don't bother with locks here, because there should be no other
|
|
|
|
* processes running when this code is reached.
|
|
|
|
*/
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
/* Unlink query text file possibly left over from crash */
|
|
|
|
unlink(PGSS_TEXT_FILE);
|
|
|
|
|
|
|
|
/* Allocate new query text temp file */
|
|
|
|
qfile = AllocateFile(PGSS_TEXT_FILE, PG_BINARY_W);
|
|
|
|
if (qfile == NULL)
|
|
|
|
goto write_error;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If we were told not to load old statistics, we're done. (Note we do
|
|
|
|
* not try to unlink any old dump file in this case. This seems a bit
|
|
|
|
* questionable but it's the historical behavior.)
|
|
|
|
*/
|
|
|
|
if (!pgss_save)
|
|
|
|
{
|
|
|
|
FreeFile(qfile);
|
|
|
|
return;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Attempt to load old statistics from the dump file.
|
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
file = AllocateFile(PGSS_DUMP_FILE, PG_BINARY_R);
|
|
|
|
if (file == NULL)
|
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (errno != ENOENT)
|
|
|
|
goto read_error;
|
|
|
|
/* No existing persisted stats file, so we're done */
|
|
|
|
FreeFile(qfile);
|
|
|
|
return;
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
buffer_size = 2048;
|
2009-01-04 23:19:59 +01:00
|
|
|
buffer = (char *) palloc(buffer_size);
|
|
|
|
|
|
|
|
if (fread(&header, sizeof(uint32), 1, file) != 1 ||
|
2013-12-07 18:06:02 +01:00
|
|
|
fread(&pgver, sizeof(uint32), 1, file) != 1 ||
|
2009-01-04 23:19:59 +01:00
|
|
|
fread(&num, sizeof(int32), 1, file) != 1)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
goto read_error;
|
|
|
|
|
|
|
|
if (header != PGSS_FILE_HEADER ||
|
|
|
|
pgver != PGSS_PG_MAJOR_VERSION)
|
|
|
|
goto data_error;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
for (i = 0; i < num; i++)
|
|
|
|
{
|
|
|
|
pgssEntry temp;
|
2009-06-11 16:49:15 +02:00
|
|
|
pgssEntry *entry;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Size query_offset;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (fread(&temp, sizeof(pgssEntry), 1, file) != 1)
|
|
|
|
goto read_error;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* Encoding is the only field we can easily sanity-check */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (!PG_VALID_BE_ENCODING(temp.encoding))
|
|
|
|
goto data_error;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Resize buffer as needed */
|
2012-03-29 03:00:31 +02:00
|
|
|
if (temp.query_len >= buffer_size)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
buffer_size = Max(buffer_size * 2, temp.query_len + 1);
|
|
|
|
buffer = repalloc(buffer, buffer_size);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (fread(buffer, 1, temp.query_len + 1, file) != temp.query_len + 1)
|
|
|
|
goto read_error;
|
|
|
|
|
|
|
|
/* Should have a trailing null, but let's make sure */
|
2012-03-29 03:00:31 +02:00
|
|
|
buffer[temp.query_len] = '\0';
|
|
|
|
|
|
|
|
/* Skip loading "sticky" entries */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (IS_STICKY(temp.counters))
|
2012-03-29 03:00:31 +02:00
|
|
|
continue;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Store the query text */
|
|
|
|
query_offset = pgss->extent;
|
|
|
|
if (fwrite(buffer, 1, temp.query_len + 1, qfile) != temp.query_len + 1)
|
|
|
|
goto write_error;
|
|
|
|
pgss->extent += temp.query_len + 1;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* make the hashtable entry (discards old entries if too many) */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
entry = entry_alloc(&temp.key, query_offset, temp.query_len,
|
|
|
|
temp.encoding,
|
|
|
|
false);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* copy in the actual stats */
|
|
|
|
entry->counters = temp.counters;
|
|
|
|
}
|
|
|
|
|
2020-11-26 13:18:05 +01:00
|
|
|
/* Read global statistics for pg_stat_statements */
|
|
|
|
if (fread(&pgss->stats, sizeof(pgssGlobalStats), 1, file) != 1)
|
|
|
|
goto read_error;
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
pfree(buffer);
|
|
|
|
FreeFile(file);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
FreeFile(qfile);
|
2012-05-27 10:54:31 +02:00
|
|
|
|
|
|
|
/*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Remove the persisted stats file so it's not included in
|
2019-06-19 14:38:23 +02:00
|
|
|
* backups/replication standbys, etc. A new file will be written on next
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* shutdown.
|
|
|
|
*
|
|
|
|
* Note: it's okay if the PGSS_TEXT_FILE is included in a basebackup,
|
|
|
|
* because we remove that file on startup; it acts inversely to
|
|
|
|
* PGSS_DUMP_FILE, in that it is only supposed to be around when the
|
|
|
|
* server is running, whereas PGSS_DUMP_FILE is only supposed to be around
|
2014-05-06 18:12:18 +02:00
|
|
|
* when the server is not running. Leaving the file creates no danger of
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* a newly restored database having a spurious record of execution costs,
|
|
|
|
* which is what we're really concerned about here.
|
2012-05-27 10:54:31 +02:00
|
|
|
*/
|
|
|
|
unlink(PGSS_DUMP_FILE);
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
return;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
read_error:
|
2009-01-04 23:19:59 +01:00
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not read file \"%s\": %m",
|
2009-01-04 23:19:59 +01:00
|
|
|
PGSS_DUMP_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
goto fail;
|
|
|
|
data_error:
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("ignoring invalid data in file \"%s\"",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_DUMP_FILE)));
|
|
|
|
goto fail;
|
|
|
|
write_error:
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
fail:
|
2009-01-04 23:19:59 +01:00
|
|
|
if (buffer)
|
|
|
|
pfree(buffer);
|
|
|
|
if (file)
|
|
|
|
FreeFile(file);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (qfile)
|
|
|
|
FreeFile(qfile);
|
2009-01-04 23:19:59 +01:00
|
|
|
/* If possible, throw away the bogus file; ignore any error */
|
|
|
|
unlink(PGSS_DUMP_FILE);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Don't unlink PGSS_TEXT_FILE here; it should always be around while the
|
|
|
|
* server is running with pg_stat_statements enabled
|
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* shmem_shutdown hook: Dump statistics into file.
|
|
|
|
*
|
|
|
|
* Note: we don't bother with acquiring lock, because there should be no
|
|
|
|
* other processes running when this is called.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_shmem_shutdown(int code, Datum arg)
|
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
FILE *file;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
char *qbuffer = NULL;
|
|
|
|
Size qbuffer_size = 0;
|
2009-06-11 16:49:15 +02:00
|
|
|
HASH_SEQ_STATUS hash_seq;
|
|
|
|
int32 num_entries;
|
|
|
|
pgssEntry *entry;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* Don't try to dump during a crash. */
|
|
|
|
if (code)
|
|
|
|
return;
|
|
|
|
|
|
|
|
/* Safety check ... shouldn't get here unless shmem is set up. */
|
|
|
|
if (!pgss || !pgss_hash)
|
|
|
|
return;
|
|
|
|
|
|
|
|
/* Don't dump if told not to. */
|
|
|
|
if (!pgss_save)
|
|
|
|
return;
|
|
|
|
|
2012-05-27 10:54:31 +02:00
|
|
|
file = AllocateFile(PGSS_DUMP_FILE ".tmp", PG_BINARY_W);
|
2009-01-04 23:19:59 +01:00
|
|
|
if (file == NULL)
|
|
|
|
goto error;
|
|
|
|
|
|
|
|
if (fwrite(&PGSS_FILE_HEADER, sizeof(uint32), 1, file) != 1)
|
|
|
|
goto error;
|
2013-12-07 18:06:02 +01:00
|
|
|
if (fwrite(&PGSS_PG_MAJOR_VERSION, sizeof(uint32), 1, file) != 1)
|
|
|
|
goto error;
|
2009-01-04 23:19:59 +01:00
|
|
|
num_entries = hash_get_num_entries(pgss_hash);
|
|
|
|
if (fwrite(&num_entries, sizeof(int32), 1, file) != 1)
|
|
|
|
goto error;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
qbuffer = qtext_load_file(&qbuffer_size);
|
|
|
|
if (qbuffer == NULL)
|
|
|
|
goto error;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* When serializing to disk, we store query texts immediately after their
|
2014-05-06 18:12:18 +02:00
|
|
|
* entry data. Any orphaned query texts are thereby excluded.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
2012-03-29 03:00:31 +02:00
|
|
|
int len = entry->query_len;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
char *qstr = qtext_fetch(entry->query_offset, len,
|
|
|
|
qbuffer, qbuffer_size);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (qstr == NULL)
|
|
|
|
continue; /* Ignore any entries with bogus texts */
|
|
|
|
|
|
|
|
if (fwrite(entry, sizeof(pgssEntry), 1, file) != 1 ||
|
|
|
|
fwrite(qstr, 1, len + 1, file) != len + 1)
|
|
|
|
{
|
|
|
|
/* note: we assume hash_seq_term won't change errno */
|
|
|
|
hash_seq_term(&hash_seq);
|
2009-01-04 23:19:59 +01:00
|
|
|
goto error;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
2020-11-26 13:18:05 +01:00
|
|
|
/* Dump global statistics for pg_stat_statements */
|
|
|
|
if (fwrite(&pgss->stats, sizeof(pgssGlobalStats), 1, file) != 1)
|
|
|
|
goto error;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
free(qbuffer);
|
|
|
|
qbuffer = NULL;
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
if (FreeFile(file))
|
|
|
|
{
|
|
|
|
file = NULL;
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
2012-05-27 10:54:31 +02:00
|
|
|
/*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Rename file into place, so we atomically replace any old one.
|
2012-05-27 10:54:31 +02:00
|
|
|
*/
|
2016-03-10 03:53:53 +01:00
|
|
|
(void) durable_rename(PGSS_DUMP_FILE ".tmp", PGSS_DUMP_FILE, LOG);
|
2012-05-27 10:54:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Unlink query-texts file; it's not needed while shutdown */
|
|
|
|
unlink(PGSS_TEXT_FILE);
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
return;
|
|
|
|
|
|
|
|
error:
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
2012-06-10 21:20:04 +02:00
|
|
|
PGSS_DUMP_FILE ".tmp")));
|
2022-06-16 21:50:56 +02:00
|
|
|
free(qbuffer);
|
2009-01-04 23:19:59 +01:00
|
|
|
if (file)
|
|
|
|
FreeFile(file);
|
2012-05-27 10:54:31 +02:00
|
|
|
unlink(PGSS_DUMP_FILE ".tmp");
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
unlink(PGSS_TEXT_FILE);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
/*
|
|
|
|
* Post-parse-analysis hook: mark query with a queryId
|
|
|
|
*/
|
|
|
|
static void
|
2021-04-07 19:06:47 +02:00
|
|
|
pgss_post_parse_analyze(ParseState *pstate, Query *query, JumbleState *jstate)
|
2012-03-29 03:00:31 +02:00
|
|
|
{
|
2014-04-21 19:28:07 +02:00
|
|
|
if (prev_post_parse_analyze_hook)
|
2021-04-07 19:06:47 +02:00
|
|
|
prev_post_parse_analyze_hook(pstate, query, jstate);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
/* Safety check... */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (!pgss || !pgss_hash || !pgss_enabled(exec_nested_level))
|
2012-03-29 03:00:31 +02:00
|
|
|
return;
|
|
|
|
|
2012-03-29 21:32:50 +02:00
|
|
|
/*
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
* Clear queryId for prepared statements related utility, as those will
|
|
|
|
* inherit from the underlying statement's one (except DEALLOCATE which is
|
|
|
|
* entirely untracked).
|
2012-03-29 21:32:50 +02:00
|
|
|
*/
|
2012-03-29 03:00:31 +02:00
|
|
|
if (query->utilityStmt)
|
2012-03-29 21:32:50 +02:00
|
|
|
{
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
if (pgss_track_utility && !PGSS_HANDLED_UTILITY(query->utilityStmt))
|
|
|
|
query->queryId = UINT64CONST(0);
|
2012-03-29 03:00:31 +02:00
|
|
|
return;
|
2012-03-29 21:32:50 +02:00
|
|
|
}
|
2012-03-29 03:00:31 +02:00
|
|
|
|
2012-03-29 21:32:50 +02:00
|
|
|
/*
|
2021-04-07 19:06:47 +02:00
|
|
|
* If query jumbling were able to identify any ignorable constants, we
|
|
|
|
* immediately create a hash table entry for the query, so that we can
|
|
|
|
* record the normalized form of the query string. If there were no such
|
|
|
|
* constants, the normalized string would be the same as the query text
|
|
|
|
* anyway, so there's no need for an early entry.
|
2012-03-29 21:32:50 +02:00
|
|
|
*/
|
2021-04-07 19:06:47 +02:00
|
|
|
if (jstate && jstate->clocations_count > 0)
|
2012-03-29 03:00:31 +02:00
|
|
|
pgss_store(pstate->p_sourcetext,
|
|
|
|
query->queryId,
|
2017-01-14 22:17:30 +01:00
|
|
|
query->stmt_location,
|
|
|
|
query->stmt_len,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PGSS_INVALID,
|
2012-03-29 03:00:31 +02:00
|
|
|
0,
|
|
|
|
0,
|
2012-04-09 17:16:04 +02:00
|
|
|
NULL,
|
2020-04-05 04:04:04 +02:00
|
|
|
NULL,
|
2022-04-08 13:51:01 +02:00
|
|
|
NULL,
|
2021-04-07 19:06:47 +02:00
|
|
|
jstate);
|
2012-03-29 03:00:31 +02:00
|
|
|
}
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
/*
|
|
|
|
* Planner hook: forward to regular planner, but measure planning time
|
|
|
|
* if needed.
|
|
|
|
*/
|
|
|
|
static PlannedStmt *
|
|
|
|
pgss_planner(Query *parse,
|
|
|
|
const char *query_string,
|
|
|
|
int cursorOptions,
|
|
|
|
ParamListInfo boundParams)
|
|
|
|
{
|
|
|
|
PlannedStmt *result;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We can't process the query if no query_string is provided, as
|
|
|
|
* pgss_store needs it. We also ignore query without queryid, as it would
|
|
|
|
* be treated as a utility statement, which may not be the case.
|
|
|
|
*
|
|
|
|
* Note that planner_hook can be called from the planner itself, so we
|
|
|
|
* have a specific nesting level for the planner. However, utility
|
|
|
|
* commands containing optimizable statements can also call the planner,
|
|
|
|
* same for regular DML (for instance for underlying foreign key queries).
|
|
|
|
* So testing the planner nesting level only is not enough to detect real
|
|
|
|
* top level planner call.
|
|
|
|
*/
|
|
|
|
if (pgss_enabled(plan_nested_level + exec_nested_level)
|
|
|
|
&& pgss_track_planning && query_string
|
|
|
|
&& parse->queryId != UINT64CONST(0))
|
|
|
|
{
|
|
|
|
instr_time start;
|
|
|
|
instr_time duration;
|
|
|
|
BufferUsage bufusage_start,
|
|
|
|
bufusage;
|
2020-04-05 04:04:04 +02:00
|
|
|
WalUsage walusage_start,
|
|
|
|
walusage;
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
|
|
|
|
/* We need to track buffer usage as the planner can access them. */
|
|
|
|
bufusage_start = pgBufferUsage;
|
2020-04-05 04:04:04 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Similarly the planner could write some WAL records in some cases
|
|
|
|
* (e.g. setting a hint bit with those being WAL-logged)
|
|
|
|
*/
|
|
|
|
walusage_start = pgWalUsage;
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
INSTR_TIME_SET_CURRENT(start);
|
|
|
|
|
|
|
|
plan_nested_level++;
|
|
|
|
PG_TRY();
|
|
|
|
{
|
|
|
|
if (prev_planner_hook)
|
|
|
|
result = prev_planner_hook(parse, query_string, cursorOptions,
|
|
|
|
boundParams);
|
|
|
|
else
|
|
|
|
result = standard_planner(parse, query_string, cursorOptions,
|
|
|
|
boundParams);
|
|
|
|
}
|
|
|
|
PG_FINALLY();
|
|
|
|
{
|
|
|
|
plan_nested_level--;
|
|
|
|
}
|
|
|
|
PG_END_TRY();
|
|
|
|
|
|
|
|
INSTR_TIME_SET_CURRENT(duration);
|
|
|
|
INSTR_TIME_SUBTRACT(duration, start);
|
|
|
|
|
|
|
|
/* calc differences of buffer counters. */
|
|
|
|
memset(&bufusage, 0, sizeof(BufferUsage));
|
|
|
|
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
|
|
|
|
|
2020-04-05 04:04:04 +02:00
|
|
|
/* calc differences of WAL counters. */
|
|
|
|
memset(&walusage, 0, sizeof(WalUsage));
|
|
|
|
WalUsageAccumDiff(&walusage, &pgWalUsage, &walusage_start);
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
pgss_store(query_string,
|
|
|
|
parse->queryId,
|
|
|
|
parse->stmt_location,
|
|
|
|
parse->stmt_len,
|
|
|
|
PGSS_PLAN,
|
|
|
|
INSTR_TIME_GET_MILLISEC(duration),
|
|
|
|
0,
|
|
|
|
&bufusage,
|
2020-04-05 04:04:04 +02:00
|
|
|
&walusage,
|
2022-04-08 13:51:01 +02:00
|
|
|
NULL,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
NULL);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
if (prev_planner_hook)
|
|
|
|
result = prev_planner_hook(parse, query_string, cursorOptions,
|
|
|
|
boundParams);
|
|
|
|
else
|
|
|
|
result = standard_planner(parse, query_string, cursorOptions,
|
|
|
|
boundParams);
|
|
|
|
}
|
|
|
|
|
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* ExecutorStart hook: start up tracking if needed
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_ExecutorStart(QueryDesc *queryDesc, int eflags)
|
|
|
|
{
|
|
|
|
if (prev_ExecutorStart)
|
|
|
|
prev_ExecutorStart(queryDesc, eflags);
|
|
|
|
else
|
|
|
|
standard_ExecutorStart(queryDesc, eflags);
|
|
|
|
|
2012-03-29 21:32:50 +02:00
|
|
|
/*
|
|
|
|
* If query has queryId zero, don't track it. This prevents double
|
|
|
|
* counting of optimizable statements that are directly contained in
|
|
|
|
* utility statements.
|
|
|
|
*/
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (pgss_enabled(exec_nested_level) && queryDesc->plannedstmt->queryId != UINT64CONST(0))
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
|
|
|
/*
|
2009-06-11 16:49:15 +02:00
|
|
|
* Set up to track total elapsed time in ExecutorRun. Make sure the
|
|
|
|
* space is allocated in the per-query context so it will go away at
|
|
|
|
* ExecutorEnd.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
if (queryDesc->totaltime == NULL)
|
|
|
|
{
|
|
|
|
MemoryContext oldcxt;
|
|
|
|
|
|
|
|
oldcxt = MemoryContextSwitchTo(queryDesc->estate->es_query_cxt);
|
2021-05-12 07:00:00 +02:00
|
|
|
queryDesc->totaltime = InstrAlloc(1, INSTRUMENT_ALL, false);
|
2009-01-04 23:19:59 +01:00
|
|
|
MemoryContextSwitchTo(oldcxt);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* ExecutorRun hook: all we need do is track nesting depth
|
|
|
|
*/
|
|
|
|
static void
|
2017-03-23 18:05:48 +01:00
|
|
|
pgss_ExecutorRun(QueryDesc *queryDesc, ScanDirection direction, uint64 count,
|
|
|
|
bool execute_once)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level++;
|
2009-01-04 23:19:59 +01:00
|
|
|
PG_TRY();
|
|
|
|
{
|
|
|
|
if (prev_ExecutorRun)
|
2017-03-23 18:05:48 +01:00
|
|
|
prev_ExecutorRun(queryDesc, direction, count, execute_once);
|
2009-01-04 23:19:59 +01:00
|
|
|
else
|
2017-03-23 18:05:48 +01:00
|
|
|
standard_ExecutorRun(queryDesc, direction, count, execute_once);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
2019-11-01 11:09:52 +01:00
|
|
|
PG_FINALLY();
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level--;
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
PG_END_TRY();
|
2011-02-27 19:43:29 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* ExecutorFinish hook: all we need do is track nesting depth
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_ExecutorFinish(QueryDesc *queryDesc)
|
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level++;
|
2011-02-27 19:43:29 +01:00
|
|
|
PG_TRY();
|
|
|
|
{
|
|
|
|
if (prev_ExecutorFinish)
|
|
|
|
prev_ExecutorFinish(queryDesc);
|
|
|
|
else
|
|
|
|
standard_ExecutorFinish(queryDesc);
|
|
|
|
}
|
2019-11-01 11:09:52 +01:00
|
|
|
PG_FINALLY();
|
2011-02-27 19:43:29 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level--;
|
2011-02-27 19:43:29 +01:00
|
|
|
}
|
|
|
|
PG_END_TRY();
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* ExecutorEnd hook: store results if needed
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
pgss_ExecutorEnd(QueryDesc *queryDesc)
|
|
|
|
{
|
2017-10-12 01:52:46 +02:00
|
|
|
uint64 queryId = queryDesc->plannedstmt->queryId;
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (queryId != UINT64CONST(0) && queryDesc->totaltime &&
|
|
|
|
pgss_enabled(exec_nested_level))
|
2012-03-29 21:32:50 +02:00
|
|
|
{
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
2009-06-11 16:49:15 +02:00
|
|
|
* Make sure stats accumulation is done. (Note: it's okay if several
|
|
|
|
* levels of hook all do this.)
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
InstrEndLoop(queryDesc->totaltime);
|
|
|
|
|
|
|
|
pgss_store(queryDesc->sourceText,
|
2012-03-29 03:00:31 +02:00
|
|
|
queryId,
|
2017-01-14 22:17:30 +01:00
|
|
|
queryDesc->plannedstmt->stmt_location,
|
|
|
|
queryDesc->plannedstmt->stmt_len,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PGSS_EXEC,
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
queryDesc->totaltime->total * 1000.0, /* convert to msec */
|
2010-01-08 01:38:20 +01:00
|
|
|
queryDesc->estate->es_processed,
|
2012-03-29 03:00:31 +02:00
|
|
|
&queryDesc->totaltime->bufusage,
|
2020-04-05 04:04:04 +02:00
|
|
|
&queryDesc->totaltime->walusage,
|
2022-04-08 13:51:01 +02:00
|
|
|
queryDesc->estate->es_jit ? &queryDesc->estate->es_jit->instr : NULL,
|
2012-03-29 03:00:31 +02:00
|
|
|
NULL);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
if (prev_ExecutorEnd)
|
|
|
|
prev_ExecutorEnd(queryDesc);
|
|
|
|
else
|
|
|
|
standard_ExecutorEnd(queryDesc);
|
|
|
|
}
|
|
|
|
|
2009-12-15 21:04:49 +01:00
|
|
|
/*
|
|
|
|
* ProcessUtility hook
|
|
|
|
*/
|
|
|
|
static void
|
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
|
|
|
pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
bool readOnlyTree,
|
2017-04-01 06:17:18 +02:00
|
|
|
ProcessUtilityContext context,
|
|
|
|
ParamListInfo params, QueryEnvironment *queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
DestReceiver *dest, QueryCompletion *qc)
|
2009-12-15 21:04:49 +01:00
|
|
|
{
|
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
|
|
|
Node *parsetree = pstmt->utilityStmt;
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
uint64 saved_queryId = pstmt->queryId;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Force utility statements to get queryId zero. We do this even in cases
|
|
|
|
* where the statement contains an optimizable statement for which a
|
|
|
|
* queryId could be derived (such as EXPLAIN or DECLARE CURSOR). For such
|
2021-05-12 19:14:10 +02:00
|
|
|
* cases, runtime control will first go through ProcessUtility and then
|
|
|
|
* the executor, and we don't want the executor hooks to do anything,
|
|
|
|
* since we are already measuring the statement's costs at the utility
|
|
|
|
* level.
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
*
|
|
|
|
* Note that this is only done if pg_stat_statements is enabled and
|
|
|
|
* configured to track utility statements, in the unlikely possibility
|
|
|
|
* that user configured another extension to handle utility statements
|
|
|
|
* only.
|
|
|
|
*/
|
|
|
|
if (pgss_enabled(exec_nested_level) && pgss_track_utility)
|
|
|
|
pstmt->queryId = UINT64CONST(0);
|
Change representation of statement lists, and add statement location info.
This patch makes several changes that improve the consistency of
representation of lists of statements. It's always been the case
that the output of parse analysis is a list of Query nodes, whatever
the types of the individual statements in the list. This patch brings
similar consistency to the outputs of raw parsing and planning steps:
* The output of raw parsing is now always a list of RawStmt nodes;
the statement-type-dependent nodes are one level down from that.
* The output of pg_plan_queries() is now always a list of PlannedStmt
nodes, even for utility statements. In the case of a utility statement,
"planning" just consists of wrapping a CMD_UTILITY PlannedStmt around
the utility node. This list representation is now used in Portal and
CachedPlan plan lists, replacing the former convention of intermixing
PlannedStmts with bare utility-statement nodes.
Now, every list of statements has a consistent head-node type depending
on how far along it is in processing. This allows changing many places
that formerly used generic "Node *" pointers to use a more specific
pointer type, thus reducing the number of IsA() tests and casts needed,
as well as improving code clarity.
Also, the post-parse-analysis representation of DECLARE CURSOR is changed
so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained
SELECT remains a child of the DeclareCursorStmt rather than getting flipped
around to be the other way. It's now true for both Query and PlannedStmt
that utilityStmt is non-null if and only if commandType is CMD_UTILITY.
That allows simplifying a lot of places that were testing both fields.
(I think some of those were just defensive programming, but in many places,
it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.)
Because PlannedStmt carries a canSetTag field, we're also able to get rid
of some ad-hoc rules about how to reconstruct canSetTag for a bare utility
statement; specifically, the assumption that a utility is canSetTag if and
only if it's the only one in its list. While I see no near-term need for
relaxing that restriction, it's nice to get rid of the ad-hocery.
The API of ProcessUtility() is changed so that what it's passed is the
wrapper PlannedStmt not just the bare utility statement. This will affect
all users of ProcessUtility_hook, but the changes are pretty trivial; see
the affected contrib modules for examples of the minimum change needed.
(Most compilers should give pointer-type-mismatch warnings for uncorrected
code.)
There's also a change in the API of ExplainOneQuery_hook, to pass through
cursorOptions instead of expecting hook functions to know what to pick.
This is needed because of the DECLARE CURSOR changes, but really should
have been done in 9.6; it's unlikely that any extant hook functions
know about using CURSOR_OPT_PARALLEL_OK.
Finally, teach gram.y to save statement boundary locations in RawStmt
nodes, and pass those through to Query and PlannedStmt nodes. This allows
more intelligent handling of cases where a source query string contains
multiple statements. This patch doesn't actually do anything with the
information, but a follow-on patch will. (Passing this information through
cleanly is the true motivation for these changes; while I think this is all
good cleanup, it's unlikely we'd have bothered without this end goal.)
catversion bump because addition of location fields to struct Query
affects stored rules.
This patch is by me, but it owes a good deal to Fabien Coelho who did
a lot of preliminary work on the problem, and also reviewed the patch.
Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
|
|
|
|
2012-03-29 22:42:09 +02:00
|
|
|
/*
|
2012-06-10 21:20:04 +02:00
|
|
|
* If it's an EXECUTE statement, we don't track it and don't increment the
|
|
|
|
* nesting level. This allows the cycles to be charged to the underlying
|
|
|
|
* PREPARE instead (by the Executor hooks), which is much more useful.
|
2012-03-29 22:42:09 +02:00
|
|
|
*
|
|
|
|
* We also don't track execution of PREPARE. If we did, we would get one
|
|
|
|
* hash table entry for the PREPARE (with hash calculated from the query
|
|
|
|
* string), and then a different one with the same query string (but hash
|
|
|
|
* calculated from the query tree) would be used to accumulate costs of
|
|
|
|
* ensuing EXECUTEs. This would be confusing, and inconsistent with other
|
|
|
|
* cases where planning time is not included at all.
|
2014-08-25 18:13:24 +02:00
|
|
|
*
|
|
|
|
* Likewise, we don't track execution of DEALLOCATE.
|
2012-03-29 22:42:09 +02:00
|
|
|
*/
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (pgss_track_utility && pgss_enabled(exec_nested_level) &&
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
PGSS_HANDLED_UTILITY(parsetree))
|
2009-12-15 21:04:49 +01:00
|
|
|
{
|
|
|
|
instr_time start;
|
|
|
|
instr_time duration;
|
2014-05-27 04:23:29 +02:00
|
|
|
uint64 rows;
|
2012-03-29 03:00:31 +02:00
|
|
|
BufferUsage bufusage_start,
|
|
|
|
bufusage;
|
2020-04-05 04:04:04 +02:00
|
|
|
WalUsage walusage_start,
|
|
|
|
walusage;
|
2009-12-15 21:04:49 +01:00
|
|
|
|
2012-03-27 21:17:22 +02:00
|
|
|
bufusage_start = pgBufferUsage;
|
2020-04-05 04:04:04 +02:00
|
|
|
walusage_start = pgWalUsage;
|
2009-12-15 21:04:49 +01:00
|
|
|
INSTR_TIME_SET_CURRENT(start);
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level++;
|
2009-12-15 21:04:49 +01:00
|
|
|
PG_TRY();
|
|
|
|
{
|
|
|
|
if (prev_ProcessUtility)
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
prev_ProcessUtility(pstmt, queryString, readOnlyTree,
|
2017-04-01 06:17:18 +02:00
|
|
|
context, params, queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
dest, qc);
|
2009-12-15 21:04:49 +01:00
|
|
|
else
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
|
2017-04-01 06:17:18 +02:00
|
|
|
context, params, queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
dest, qc);
|
2009-12-15 21:04:49 +01:00
|
|
|
}
|
2019-11-01 11:09:52 +01:00
|
|
|
PG_FINALLY();
|
2009-12-15 21:04:49 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
exec_nested_level--;
|
2009-12-15 21:04:49 +01:00
|
|
|
}
|
|
|
|
PG_END_TRY();
|
|
|
|
|
|
|
|
INSTR_TIME_SET_CURRENT(duration);
|
|
|
|
INSTR_TIME_SUBTRACT(duration, start);
|
|
|
|
|
2020-07-29 16:21:55 +02:00
|
|
|
/*
|
2020-11-12 03:26:55 +01:00
|
|
|
* Track the total number of rows retrieved or affected by the utility
|
|
|
|
* statements of COPY, FETCH, CREATE TABLE AS, CREATE MATERIALIZED
|
|
|
|
* VIEW, REFRESH MATERIALIZED VIEW and SELECT INTO.
|
2020-07-29 16:21:55 +02:00
|
|
|
*/
|
|
|
|
rows = (qc && (qc->commandTag == CMDTAG_COPY ||
|
|
|
|
qc->commandTag == CMDTAG_FETCH ||
|
2020-11-12 03:26:55 +01:00
|
|
|
qc->commandTag == CMDTAG_SELECT ||
|
|
|
|
qc->commandTag == CMDTAG_REFRESH_MATERIALIZED_VIEW)) ?
|
2020-07-29 16:21:55 +02:00
|
|
|
qc->nprocessed : 0;
|
2009-12-15 21:04:49 +01:00
|
|
|
|
2010-01-08 01:38:20 +01:00
|
|
|
/* calc differences of buffer counters. */
|
2020-03-30 05:15:26 +02:00
|
|
|
memset(&bufusage, 0, sizeof(BufferUsage));
|
|
|
|
BufferUsageAccumDiff(&bufusage, &pgBufferUsage, &bufusage_start);
|
2010-01-08 01:38:20 +01:00
|
|
|
|
2020-04-05 04:04:04 +02:00
|
|
|
/* calc differences of WAL counters. */
|
|
|
|
memset(&walusage, 0, sizeof(WalUsage));
|
|
|
|
WalUsageAccumDiff(&walusage, &pgWalUsage, &walusage_start);
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
pgss_store(queryString,
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
saved_queryId,
|
2017-01-14 22:17:30 +01:00
|
|
|
pstmt->stmt_location,
|
|
|
|
pstmt->stmt_len,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
PGSS_EXEC,
|
2012-04-28 22:03:57 +02:00
|
|
|
INSTR_TIME_GET_MILLISEC(duration),
|
2012-03-29 03:00:31 +02:00
|
|
|
rows,
|
|
|
|
&bufusage,
|
2020-04-05 04:04:04 +02:00
|
|
|
&walusage,
|
2022-04-08 13:51:01 +02:00
|
|
|
NULL,
|
2012-03-29 03:00:31 +02:00
|
|
|
NULL);
|
2009-12-15 21:04:49 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
if (prev_ProcessUtility)
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
prev_ProcessUtility(pstmt, queryString, readOnlyTree,
|
2017-04-01 06:17:18 +02:00
|
|
|
context, params, queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
dest, qc);
|
2009-12-15 21:04:49 +01:00
|
|
|
else
|
Centralize the logic for protective copying of utility statements.
In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards. However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions. Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree. But that's
prone to errors of omission. Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.
In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.
Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary. It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing. In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees. Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.
(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only. Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)
Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
2021-06-18 17:22:58 +02:00
|
|
|
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
|
2017-04-01 06:17:18 +02:00
|
|
|
context, params, queryEnv,
|
2020-03-02 22:19:51 +01:00
|
|
|
dest, qc);
|
2009-12-15 21:04:49 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* Store some statistics for a statement.
|
2012-03-29 03:00:31 +02:00
|
|
|
*
|
|
|
|
* If jstate is not NULL then we're trying to create an entry for which
|
|
|
|
* we have no statistics as yet; we just want to record the normalized
|
2020-04-05 04:04:04 +02:00
|
|
|
* query string. total_time, rows, bufusage and walusage are ignored in this
|
|
|
|
* case.
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
*
|
|
|
|
* If kind is PGSS_PLAN or PGSS_EXEC, its value is used as the array position
|
|
|
|
* for the arrays in the Counters field.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
static void
|
2017-10-12 01:52:46 +02:00
|
|
|
pgss_store(const char *query, uint64 queryId,
|
2017-01-14 22:17:30 +01:00
|
|
|
int query_location, int query_len,
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
pgssStoreKind kind,
|
2012-03-29 03:00:31 +02:00
|
|
|
double total_time, uint64 rows,
|
|
|
|
const BufferUsage *bufusage,
|
2020-04-05 04:04:04 +02:00
|
|
|
const WalUsage *walusage,
|
2022-04-08 13:51:01 +02:00
|
|
|
const struct JitInstrumentation *jitusage,
|
2021-04-07 19:06:47 +02:00
|
|
|
JumbleState *jstate)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
pgssHashKey key;
|
|
|
|
pgssEntry *entry;
|
2012-03-29 03:00:31 +02:00
|
|
|
char *norm_query = NULL;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
int encoding = GetDatabaseEncoding();
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
Assert(query != NULL);
|
|
|
|
|
|
|
|
/* Safety check... */
|
|
|
|
if (!pgss || !pgss_hash)
|
|
|
|
return;
|
|
|
|
|
2017-01-14 22:17:30 +01:00
|
|
|
/*
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
* Nothing to do if compute_query_id isn't enabled and no other module
|
|
|
|
* computed a query identifier.
|
2017-01-14 22:17:30 +01:00
|
|
|
*/
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
if (queryId == UINT64CONST(0))
|
|
|
|
return;
|
2017-01-14 22:17:30 +01:00
|
|
|
|
|
|
|
/*
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
* Confine our attention to the relevant part of the string, if the query
|
|
|
|
* is a portion of a multi-statement source string, and update query
|
|
|
|
* location and length if needed.
|
2017-01-14 22:17:30 +01:00
|
|
|
*/
|
Make use of in-core query id added by commit 5fd9dfa5f5
Use the in-core query id computation for pg_stat_activity,
log_line_prefix, and EXPLAIN VERBOSE.
Similar to other fields in pg_stat_activity, only the queryid from the
top level statements are exposed, and if the backends status isn't
active then the queryid from the last executed statements is displayed.
Add a %Q placeholder to include the queryid in log_line_prefix, which
will also only expose top level statements.
For EXPLAIN VERBOSE, if a query identifier has been computed, either by
enabling compute_query_id or using a third-party module, display it.
Bump catalog version.
Discussion: https://postgr.es/m/20210407125726.tkvjdbw76hxnpwfi@nol
Author: Julien Rouhaud
Reviewed-by: Alvaro Herrera, Nitin Jadhav, Zhihong Yu
2021-04-07 20:03:56 +02:00
|
|
|
query = CleanQuerytext(query, &query_location, &query_len);
|
2019-01-11 04:20:09 +01:00
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/* Set up key for hashtable search */
|
2021-04-08 10:23:10 +02:00
|
|
|
|
|
|
|
/* memset() is required when pgssHashKey is without padding only */
|
|
|
|
memset(&key, 0, sizeof(pgssHashKey));
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
key.userid = GetUserId();
|
|
|
|
key.dbid = MyDatabaseId;
|
2012-03-29 03:00:31 +02:00
|
|
|
key.queryid = queryId;
|
2021-04-08 10:23:10 +02:00
|
|
|
key.toplevel = (exec_nested_level == 0);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* Lookup the hash table entry with shared lock. */
|
|
|
|
LWLockAcquire(pgss->lock, LW_SHARED);
|
|
|
|
|
|
|
|
entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_FIND, NULL);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
2012-04-09 17:16:04 +02:00
|
|
|
/* Create new entry, if not present */
|
2009-01-04 23:19:59 +01:00
|
|
|
if (!entry)
|
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Size query_offset;
|
|
|
|
int gc_count;
|
|
|
|
bool stored;
|
|
|
|
bool do_gc;
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
/*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Create a new, normalized query string if caller asked. We don't
|
|
|
|
* need to hold the lock while doing this work. (Note: in any case,
|
|
|
|
* it's possible that someone else creates a duplicate hashtable entry
|
|
|
|
* in the interval where we don't hold the lock below. That case is
|
|
|
|
* handled by entry_alloc.)
|
2012-03-29 03:00:31 +02:00
|
|
|
*/
|
|
|
|
if (jstate)
|
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
LWLockRelease(pgss->lock);
|
2012-03-29 03:00:31 +02:00
|
|
|
norm_query = generate_normalized_query(jstate, query,
|
2017-01-14 22:17:30 +01:00
|
|
|
query_location,
|
2020-09-08 10:08:46 +02:00
|
|
|
&query_len);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
LWLockAcquire(pgss->lock, LW_SHARED);
|
|
|
|
}
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Append new query text to file with only shared lock held */
|
|
|
|
stored = qtext_store(norm_query ? norm_query : query, query_len,
|
|
|
|
&query_offset, &gc_count);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* Determine whether we need to garbage collect external query texts
|
|
|
|
* while the shared lock is still held. This micro-optimization
|
|
|
|
* avoids taking the time to decide this while holding exclusive lock.
|
|
|
|
*/
|
|
|
|
do_gc = need_gc_qtexts();
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Need exclusive lock to make a new hashtable entry - promote */
|
|
|
|
LWLockRelease(pgss->lock);
|
|
|
|
LWLockAcquire(pgss->lock, LW_EXCLUSIVE);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* A garbage collection may have occurred while we weren't holding the
|
|
|
|
* lock. In the unlikely event that this happens, the query text we
|
|
|
|
* stored above will have been garbage collected, so write it again.
|
|
|
|
* This should be infrequent enough that doing it while holding
|
|
|
|
* exclusive lock isn't a performance problem.
|
|
|
|
*/
|
|
|
|
if (!stored || pgss->gc_count != gc_count)
|
|
|
|
stored = qtext_store(norm_query ? norm_query : query, query_len,
|
|
|
|
&query_offset, NULL);
|
|
|
|
|
|
|
|
/* If we failed to write to the text file, give up */
|
|
|
|
if (!stored)
|
|
|
|
goto done;
|
|
|
|
|
|
|
|
/* OK to create a new hashtable entry */
|
|
|
|
entry = entry_alloc(&key, query_offset, query_len, encoding,
|
|
|
|
jstate != NULL);
|
|
|
|
|
|
|
|
/* If needed, perform garbage collection while exclusive lock held */
|
|
|
|
if (do_gc)
|
|
|
|
gc_qtexts();
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
2012-04-09 17:16:04 +02:00
|
|
|
/* Increment the counts, except when jstate is not NULL */
|
|
|
|
if (!jstate)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
2012-04-09 17:16:04 +02:00
|
|
|
/*
|
|
|
|
* Grab the spinlock while updating the counters (see comment about
|
|
|
|
* locking rules at the head of the file)
|
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
volatile pgssEntry *e = (volatile pgssEntry *) entry;
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
Assert(kind == PGSS_PLAN || kind == PGSS_EXEC);
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
SpinLockAcquire(&e->mutex);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
2012-04-09 17:16:04 +02:00
|
|
|
/* "Unstick" entry if it was previously sticky */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (IS_STICKY(e->counters))
|
2012-04-09 17:16:04 +02:00
|
|
|
e->counters.usage = USAGE_INIT;
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
e->counters.calls[kind] += 1;
|
|
|
|
e->counters.total_time[kind] += total_time;
|
|
|
|
|
|
|
|
if (e->counters.calls[kind] == 1)
|
2015-03-27 20:43:22 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
e->counters.min_time[kind] = total_time;
|
|
|
|
e->counters.max_time[kind] = total_time;
|
|
|
|
e->counters.mean_time[kind] = total_time;
|
2015-03-27 20:43:22 +01:00
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/*
|
2015-05-24 03:35:49 +02:00
|
|
|
* Welford's method for accurately computing variance. See
|
|
|
|
* <http://www.johndcook.com/blog/standard_deviation/>
|
2015-03-27 20:43:22 +01:00
|
|
|
*/
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
double old_mean = e->counters.mean_time[kind];
|
2015-03-27 20:43:22 +01:00
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
e->counters.mean_time[kind] +=
|
|
|
|
(total_time - old_mean) / e->counters.calls[kind];
|
|
|
|
e->counters.sum_var_time[kind] +=
|
|
|
|
(total_time - old_mean) * (total_time - e->counters.mean_time[kind]);
|
2015-03-27 20:43:22 +01:00
|
|
|
|
|
|
|
/* calculate min and max time */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (e->counters.min_time[kind] > total_time)
|
|
|
|
e->counters.min_time[kind] = total_time;
|
|
|
|
if (e->counters.max_time[kind] < total_time)
|
|
|
|
e->counters.max_time[kind] = total_time;
|
2015-03-27 20:43:22 +01:00
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
e->counters.rows += rows;
|
2010-01-08 01:38:20 +01:00
|
|
|
e->counters.shared_blks_hit += bufusage->shared_blks_hit;
|
|
|
|
e->counters.shared_blks_read += bufusage->shared_blks_read;
|
2012-02-23 02:33:05 +01:00
|
|
|
e->counters.shared_blks_dirtied += bufusage->shared_blks_dirtied;
|
2010-01-08 01:38:20 +01:00
|
|
|
e->counters.shared_blks_written += bufusage->shared_blks_written;
|
|
|
|
e->counters.local_blks_hit += bufusage->local_blks_hit;
|
|
|
|
e->counters.local_blks_read += bufusage->local_blks_read;
|
2012-02-23 02:33:05 +01:00
|
|
|
e->counters.local_blks_dirtied += bufusage->local_blks_dirtied;
|
2010-01-08 01:38:20 +01:00
|
|
|
e->counters.local_blks_written += bufusage->local_blks_written;
|
|
|
|
e->counters.temp_blks_read += bufusage->temp_blks_read;
|
|
|
|
e->counters.temp_blks_written += bufusage->temp_blks_written;
|
2012-04-30 00:13:33 +02:00
|
|
|
e->counters.blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage->blk_read_time);
|
|
|
|
e->counters.blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage->blk_write_time);
|
2022-04-08 06:12:07 +02:00
|
|
|
e->counters.temp_blk_read_time += INSTR_TIME_GET_MILLISEC(bufusage->temp_blk_read_time);
|
|
|
|
e->counters.temp_blk_write_time += INSTR_TIME_GET_MILLISEC(bufusage->temp_blk_write_time);
|
2012-04-28 22:03:57 +02:00
|
|
|
e->counters.usage += USAGE_EXEC(total_time);
|
2020-04-05 04:04:04 +02:00
|
|
|
e->counters.wal_records += walusage->wal_records;
|
2020-05-05 04:30:53 +02:00
|
|
|
e->counters.wal_fpi += walusage->wal_fpi;
|
2020-04-05 04:04:04 +02:00
|
|
|
e->counters.wal_bytes += walusage->wal_bytes;
|
2022-04-08 13:51:01 +02:00
|
|
|
if (jitusage)
|
|
|
|
{
|
|
|
|
e->counters.jit_functions += jitusage->created_functions;
|
|
|
|
e->counters.jit_generation_time += INSTR_TIME_GET_MILLISEC(jitusage->generation_counter);
|
|
|
|
|
|
|
|
if (INSTR_TIME_GET_MILLISEC(jitusage->inlining_counter))
|
|
|
|
e->counters.jit_inlining_count++;
|
|
|
|
e->counters.jit_inlining_time += INSTR_TIME_GET_MILLISEC(jitusage->inlining_counter);
|
|
|
|
|
|
|
|
if (INSTR_TIME_GET_MILLISEC(jitusage->optimization_counter))
|
|
|
|
e->counters.jit_optimization_count++;
|
|
|
|
e->counters.jit_optimization_time += INSTR_TIME_GET_MILLISEC(jitusage->optimization_counter);
|
|
|
|
|
|
|
|
if (INSTR_TIME_GET_MILLISEC(jitusage->emission_counter))
|
|
|
|
e->counters.jit_emission_count++;
|
|
|
|
e->counters.jit_emission_time += INSTR_TIME_GET_MILLISEC(jitusage->emission_counter);
|
|
|
|
}
|
2012-03-29 03:00:31 +02:00
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
SpinLockRelease(&e->mutex);
|
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
done:
|
2009-01-04 23:19:59 +01:00
|
|
|
LWLockRelease(pgss->lock);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* We postpone this clean-up until we're out of the lock */
|
2012-03-29 03:00:31 +02:00
|
|
|
if (norm_query)
|
|
|
|
pfree(norm_query);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2019-01-11 04:20:09 +01:00
|
|
|
* Reset statement statistics corresponding to userid, dbid, and queryid.
|
|
|
|
*/
|
|
|
|
Datum
|
|
|
|
pg_stat_statements_reset_1_7(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
Oid userid;
|
|
|
|
Oid dbid;
|
|
|
|
uint64 queryid;
|
|
|
|
|
|
|
|
userid = PG_GETARG_OID(0);
|
|
|
|
dbid = PG_GETARG_OID(1);
|
|
|
|
queryid = (uint64) PG_GETARG_INT64(2);
|
|
|
|
|
|
|
|
entry_reset(userid, dbid, queryid);
|
|
|
|
|
|
|
|
PG_RETURN_VOID();
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Reset statement statistics.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
Datum
|
|
|
|
pg_stat_statements_reset(PG_FUNCTION_ARGS)
|
|
|
|
{
|
2019-01-11 04:20:09 +01:00
|
|
|
entry_reset(0, 0, 0);
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
PG_RETURN_VOID();
|
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Number of output arguments (columns) for various API versions */
|
2012-02-23 02:33:05 +01:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_0 14
|
2013-12-07 18:06:02 +01:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_1 18
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_2 19
|
2015-03-27 20:43:22 +01:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_3 23
|
2020-04-05 04:04:04 +02:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_8 32
|
2021-04-08 15:15:17 +02:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_9 33
|
2022-04-08 13:51:01 +02:00
|
|
|
#define PG_STAT_STATEMENTS_COLS_V1_10 43
|
|
|
|
#define PG_STAT_STATEMENTS_COLS 43 /* maximum of above */
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Retrieve statement statistics.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*
|
|
|
|
* The SQL API of this function has changed multiple times, and will likely
|
|
|
|
* do so again in future. To support the case where a newer version of this
|
|
|
|
* loadable module is being used with an old SQL declaration of the function,
|
|
|
|
* we continue to support the older API versions. For 1.2 and later, the
|
|
|
|
* expected API version is identified by embedding it in the C name of the
|
|
|
|
* function. Unfortunately we weren't bright enough to do that for 1.1.
|
|
|
|
*/
|
2022-04-08 06:12:07 +02:00
|
|
|
Datum
|
|
|
|
pg_stat_statements_1_10(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
bool showtext = PG_GETARG_BOOL(0);
|
|
|
|
|
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_10, showtext);
|
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
2021-04-08 10:23:10 +02:00
|
|
|
Datum
|
2021-04-08 15:15:17 +02:00
|
|
|
pg_stat_statements_1_9(PG_FUNCTION_ARGS)
|
2021-04-08 10:23:10 +02:00
|
|
|
{
|
|
|
|
bool showtext = PG_GETARG_BOOL(0);
|
|
|
|
|
2021-04-08 15:15:17 +02:00
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_9, showtext);
|
2021-04-08 10:23:10 +02:00
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
Datum
|
|
|
|
pg_stat_statements_1_8(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
bool showtext = PG_GETARG_BOOL(0);
|
|
|
|
|
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_8, showtext);
|
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
2015-03-27 20:43:22 +01:00
|
|
|
Datum
|
|
|
|
pg_stat_statements_1_3(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
bool showtext = PG_GETARG_BOOL(0);
|
|
|
|
|
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_3, showtext);
|
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Datum
|
|
|
|
pg_stat_statements_1_2(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
bool showtext = PG_GETARG_BOOL(0);
|
|
|
|
|
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_2, showtext);
|
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Legacy entry point for pg_stat_statements() API versions 1.0 and 1.1.
|
|
|
|
* This can be removed someday, perhaps.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
Datum
|
|
|
|
pg_stat_statements(PG_FUNCTION_ARGS)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
{
|
|
|
|
/* If it's really API 1.1, we'll figure that out below */
|
|
|
|
pg_stat_statements_internal(fcinfo, PGSS_V1_0, true);
|
|
|
|
|
|
|
|
return (Datum) 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Common code for all versions of pg_stat_statements() */
|
|
|
|
static void
|
|
|
|
pg_stat_statements_internal(FunctionCallInfo fcinfo,
|
|
|
|
pgssVersion api_version,
|
|
|
|
bool showtext)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
ReturnSetInfo *rsinfo = (ReturnSetInfo *) fcinfo->resultinfo;
|
|
|
|
Oid userid = GetUserId();
|
2017-03-30 20:18:53 +02:00
|
|
|
bool is_allowed_role = false;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
char *qbuffer = NULL;
|
|
|
|
Size qbuffer_size = 0;
|
|
|
|
Size extent = 0;
|
|
|
|
int gc_count = 0;
|
2009-06-11 16:49:15 +02:00
|
|
|
HASH_SEQ_STATUS hash_seq;
|
|
|
|
pgssEntry *entry;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2022-05-12 21:17:30 +02:00
|
|
|
/*
|
|
|
|
* Superusers or roles with the privileges of pg_read_all_stats members
|
|
|
|
* are allowed
|
|
|
|
*/
|
2022-03-28 21:10:04 +02:00
|
|
|
is_allowed_role = has_privs_of_role(userid, ROLE_PG_READ_ALL_STATS);
|
2017-03-30 20:18:53 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* hash table must exist already */
|
2009-01-04 23:19:59 +01:00
|
|
|
if (!pgss || !pgss_hash)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
|
|
|
|
errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
|
|
|
|
|
Simplify SRFs using materialize mode in contrib/ modules
9e98583 introduced a helper to centralize building their needed state
(tuplestore, tuple descriptors, etc.), checking for any errors. This
commit updates all places of contrib/ that can be switched to use
SetSingleFuncCall() as a drop-in replacement, resulting in the removal
of a lot of boilerplate code in all the modules updated by this commit.
Per analysis, some places remain as they are:
- pg_logdir_ls() in adminpack/ uses historically TYPEFUNC_RECORD as
return type, and I suspect that changing it may cause issues at run-time
with some of its past versions, down to 1.0.
- dblink/ uses a wrapper function doing exactly the work of
SetSingleFuncCall(). Here the switch should be possible, but rather
invasive so it does not seem the extra backpatch maintenance cost.
- tablefunc/, similarly, uses multiple helper functions with portions of
SetSingleFuncCall() spread across the code paths of this module.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_bvDPJoL9mH6eYwvBpPtTGQwbDzfJbCM-OjkSZDu5yTPg@mail.gmail.com
2022-03-08 02:12:22 +01:00
|
|
|
SetSingleFuncCall(fcinfo, 0);
|
2013-12-07 18:06:02 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* Check we have the expected number of output arguments. Aside from
|
|
|
|
* being a good safety check, we need a kluge here to detect API version
|
|
|
|
* 1.1, which was wedged into the code in an ill-considered way.
|
|
|
|
*/
|
Simplify SRFs using materialize mode in contrib/ modules
9e98583 introduced a helper to centralize building their needed state
(tuplestore, tuple descriptors, etc.), checking for any errors. This
commit updates all places of contrib/ that can be switched to use
SetSingleFuncCall() as a drop-in replacement, resulting in the removal
of a lot of boilerplate code in all the modules updated by this commit.
Per analysis, some places remain as they are:
- pg_logdir_ls() in adminpack/ uses historically TYPEFUNC_RECORD as
return type, and I suspect that changing it may cause issues at run-time
with some of its past versions, down to 1.0.
- dblink/ uses a wrapper function doing exactly the work of
SetSingleFuncCall(). Here the switch should be possible, but rather
invasive so it does not seem the extra backpatch maintenance cost.
- tablefunc/, similarly, uses multiple helper functions with portions of
SetSingleFuncCall() spread across the code paths of this module.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_bvDPJoL9mH6eYwvBpPtTGQwbDzfJbCM-OjkSZDu5yTPg@mail.gmail.com
2022-03-08 02:12:22 +01:00
|
|
|
switch (rsinfo->setDesc->natts)
|
2013-12-07 18:06:02 +01:00
|
|
|
{
|
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_0:
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (api_version != PGSS_V1_0)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
2013-12-07 18:06:02 +01:00
|
|
|
break;
|
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_1:
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* pg_stat_statements() should have told us 1.0 */
|
|
|
|
if (api_version != PGSS_V1_0)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
|
|
|
api_version = PGSS_V1_1;
|
2013-12-07 18:06:02 +01:00
|
|
|
break;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_2:
|
|
|
|
if (api_version != PGSS_V1_2)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
2013-12-07 18:06:02 +01:00
|
|
|
break;
|
2015-03-27 20:43:22 +01:00
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_3:
|
|
|
|
if (api_version != PGSS_V1_3)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
|
|
|
break;
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_8:
|
|
|
|
if (api_version != PGSS_V1_8)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
|
|
|
break;
|
2021-04-08 15:15:17 +02:00
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_9:
|
|
|
|
if (api_version != PGSS_V1_9)
|
2021-04-08 10:23:10 +02:00
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
|
|
|
break;
|
2022-04-08 06:12:07 +02:00
|
|
|
case PG_STAT_STATEMENTS_COLS_V1_10:
|
|
|
|
if (api_version != PGSS_V1_10)
|
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
|
|
|
break;
|
2013-12-07 18:06:02 +01:00
|
|
|
default:
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
elog(ERROR, "incorrect number of output arguments");
|
2013-12-07 18:06:02 +01:00
|
|
|
}
|
2010-01-08 01:38:20 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* We'd like to load the query text file (if needed) while not holding any
|
2014-05-06 18:12:18 +02:00
|
|
|
* lock on pgss->lock. In the worst case we'll have to do this again
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* after we have the lock, but it's unlikely enough to make this a win
|
2014-05-06 18:12:18 +02:00
|
|
|
* despite occasional duplicated work. We need to reload if anybody
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* writes to the file (either a retail qtext_store(), or a garbage
|
|
|
|
* collection) between this point and where we've gotten shared lock. If
|
|
|
|
* a qtext_store is actually in progress when we look, we might as well
|
|
|
|
* skip the speculative load entirely.
|
|
|
|
*/
|
|
|
|
if (showtext)
|
|
|
|
{
|
|
|
|
int n_writers;
|
|
|
|
|
|
|
|
/* Take the mutex so we can examine variables */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
extent = s->extent;
|
|
|
|
n_writers = s->n_writers;
|
|
|
|
gc_count = s->gc_count;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
|
|
|
/* No point in loading file now if there are active writers */
|
|
|
|
if (n_writers == 0)
|
|
|
|
qbuffer = qtext_load_file(&qbuffer_size);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Get shared lock, load or reload the query text file if we must, and
|
|
|
|
* iterate over the hashtable entries.
|
|
|
|
*
|
|
|
|
* With a large hash table, we might be holding the lock rather longer
|
|
|
|
* than one could wish. However, this only blocks creation of new hash
|
|
|
|
* table entries, and the larger the hash table the less likely that is to
|
|
|
|
* be needed. So we can hope this is okay. Perhaps someday we'll decide
|
|
|
|
* we need to partition the hash table to limit the time spent holding any
|
|
|
|
* one lock.
|
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
LWLockAcquire(pgss->lock, LW_SHARED);
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (showtext)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Here it is safe to examine extent and gc_count without taking the
|
|
|
|
* mutex. Note that although other processes might change
|
|
|
|
* pgss->extent just after we look at it, the strings they then write
|
|
|
|
* into the file cannot yet be referenced in the hashtable, so we
|
|
|
|
* don't care whether we see them or not.
|
|
|
|
*
|
|
|
|
* If qtext_load_file fails, we just press on; we'll return NULL for
|
|
|
|
* every query text.
|
|
|
|
*/
|
|
|
|
if (qbuffer == NULL ||
|
|
|
|
pgss->extent != extent ||
|
|
|
|
pgss->gc_count != gc_count)
|
|
|
|
{
|
2022-06-16 21:50:56 +02:00
|
|
|
free(qbuffer);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
qbuffer = qtext_load_file(&qbuffer_size);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
Datum values[PG_STAT_STATEMENTS_COLS];
|
|
|
|
bool nulls[PG_STAT_STATEMENTS_COLS];
|
|
|
|
int i = 0;
|
|
|
|
Counters tmp;
|
2015-03-28 18:56:37 +01:00
|
|
|
double stddev;
|
2013-12-08 11:59:07 +01:00
|
|
|
int64 queryid = entry->key.queryid;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
memset(values, 0, sizeof(values));
|
|
|
|
memset(nulls, 0, sizeof(nulls));
|
|
|
|
|
|
|
|
values[i++] = ObjectIdGetDatum(entry->key.userid);
|
|
|
|
values[i++] = ObjectIdGetDatum(entry->key.dbid);
|
2021-04-08 15:15:17 +02:00
|
|
|
if (api_version >= PGSS_V1_9)
|
2021-04-08 10:23:10 +02:00
|
|
|
values[i++] = BoolGetDatum(entry->key.toplevel);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2017-03-30 20:18:53 +02:00
|
|
|
if (is_allowed_role || entry->key.userid == userid)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (api_version >= PGSS_V1_2)
|
2013-12-08 11:59:07 +01:00
|
|
|
values[i++] = Int64GetDatumFast(queryid);
|
2013-12-07 18:06:02 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (showtext)
|
|
|
|
{
|
|
|
|
char *qstr = qtext_fetch(entry->query_offset,
|
|
|
|
entry->query_len,
|
|
|
|
qbuffer,
|
|
|
|
qbuffer_size);
|
|
|
|
|
|
|
|
if (qstr)
|
|
|
|
{
|
|
|
|
char *enc;
|
|
|
|
|
2014-02-23 22:59:05 +01:00
|
|
|
enc = pg_any_to_server(qstr,
|
|
|
|
entry->query_len,
|
|
|
|
entry->encoding);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
values[i++] = CStringGetTextDatum(enc);
|
|
|
|
|
|
|
|
if (enc != qstr)
|
|
|
|
pfree(enc);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Just return a null if we fail to find the text */
|
|
|
|
nulls[i++] = true;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Query text not requested */
|
|
|
|
nulls[i++] = true;
|
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
else
|
2013-12-07 18:06:02 +01:00
|
|
|
{
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Don't show queryid */
|
|
|
|
if (api_version >= PGSS_V1_2)
|
2013-12-07 18:06:02 +01:00
|
|
|
nulls[i++] = true;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* Don't show query text, but hint as to the reason for not doing
|
|
|
|
* so if it was requested
|
|
|
|
*/
|
|
|
|
if (showtext)
|
|
|
|
values[i++] = CStringGetTextDatum("<insufficient privilege>");
|
|
|
|
else
|
|
|
|
nulls[i++] = true;
|
2013-12-07 18:06:02 +01:00
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* copy counters to a local variable to keep locking time short */
|
|
|
|
{
|
|
|
|
volatile pgssEntry *e = (volatile pgssEntry *) entry;
|
|
|
|
|
|
|
|
SpinLockAcquire(&e->mutex);
|
|
|
|
tmp = e->counters;
|
|
|
|
SpinLockRelease(&e->mutex);
|
|
|
|
}
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
/* Skip entry if unexecuted (ie, it's a pending "sticky" entry) */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (IS_STICKY(tmp))
|
2012-03-29 03:00:31 +02:00
|
|
|
continue;
|
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
/* Note that we rely on PGSS_PLAN being 0 and PGSS_EXEC being 1. */
|
|
|
|
for (int kind = 0; kind < PGSS_NUMKIND; kind++)
|
2015-03-27 20:43:22 +01:00
|
|
|
{
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (kind == PGSS_EXEC || api_version >= PGSS_V1_8)
|
|
|
|
{
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.calls[kind]);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.total_time[kind]);
|
|
|
|
}
|
2015-05-24 03:35:49 +02:00
|
|
|
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if ((kind == PGSS_EXEC && api_version >= PGSS_V1_3) ||
|
|
|
|
api_version >= PGSS_V1_8)
|
|
|
|
{
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.min_time[kind]);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.max_time[kind]);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.mean_time[kind]);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Note we are calculating the population variance here, not
|
|
|
|
* the sample variance, as we have data for the whole
|
|
|
|
* population, so Bessel's correction is not used, and we
|
|
|
|
* don't divide by tmp.calls - 1.
|
|
|
|
*/
|
|
|
|
if (tmp.calls[kind] > 1)
|
|
|
|
stddev = sqrt(tmp.sum_var_time[kind] / tmp.calls[kind]);
|
|
|
|
else
|
|
|
|
stddev = 0.0;
|
|
|
|
values[i++] = Float8GetDatumFast(stddev);
|
|
|
|
}
|
2015-03-27 20:43:22 +01:00
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.rows);
|
2010-01-08 01:38:20 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.shared_blks_hit);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.shared_blks_read);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (api_version >= PGSS_V1_1)
|
2012-02-23 02:33:05 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.shared_blks_dirtied);
|
2010-01-08 01:38:20 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.shared_blks_written);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.local_blks_hit);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.local_blks_read);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (api_version >= PGSS_V1_1)
|
2012-02-23 02:33:05 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.local_blks_dirtied);
|
2010-01-08 01:38:20 +01:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.local_blks_written);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.temp_blks_read);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.temp_blks_written);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (api_version >= PGSS_V1_1)
|
2012-03-27 21:17:22 +02:00
|
|
|
{
|
2012-04-30 00:13:33 +02:00
|
|
|
values[i++] = Float8GetDatumFast(tmp.blk_read_time);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.blk_write_time);
|
2012-03-27 21:17:22 +02:00
|
|
|
}
|
2022-04-08 06:12:07 +02:00
|
|
|
if (api_version >= PGSS_V1_10)
|
|
|
|
{
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.temp_blk_read_time);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.temp_blk_write_time);
|
|
|
|
}
|
2020-04-05 04:04:04 +02:00
|
|
|
if (api_version >= PGSS_V1_8)
|
|
|
|
{
|
|
|
|
char buf[256];
|
|
|
|
Datum wal_bytes;
|
|
|
|
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.wal_records);
|
2020-05-05 04:30:53 +02:00
|
|
|
values[i++] = Int64GetDatumFast(tmp.wal_fpi);
|
2020-04-05 04:04:04 +02:00
|
|
|
|
|
|
|
snprintf(buf, sizeof buf, UINT64_FORMAT, tmp.wal_bytes);
|
|
|
|
|
|
|
|
/* Convert to numeric. */
|
|
|
|
wal_bytes = DirectFunctionCall3(numeric_in,
|
|
|
|
CStringGetDatum(buf),
|
|
|
|
ObjectIdGetDatum(0),
|
|
|
|
Int32GetDatum(-1));
|
|
|
|
values[i++] = wal_bytes;
|
|
|
|
}
|
2022-04-08 13:51:01 +02:00
|
|
|
if (api_version >= PGSS_V1_10)
|
|
|
|
{
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.jit_functions);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.jit_generation_time);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.jit_inlining_count);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.jit_inlining_time);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.jit_optimization_count);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.jit_optimization_time);
|
|
|
|
values[i++] = Int64GetDatumFast(tmp.jit_emission_count);
|
|
|
|
values[i++] = Float8GetDatumFast(tmp.jit_emission_time);
|
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
Assert(i == (api_version == PGSS_V1_0 ? PG_STAT_STATEMENTS_COLS_V1_0 :
|
|
|
|
api_version == PGSS_V1_1 ? PG_STAT_STATEMENTS_COLS_V1_1 :
|
|
|
|
api_version == PGSS_V1_2 ? PG_STAT_STATEMENTS_COLS_V1_2 :
|
2015-03-27 20:43:22 +01:00
|
|
|
api_version == PGSS_V1_3 ? PG_STAT_STATEMENTS_COLS_V1_3 :
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
api_version == PGSS_V1_8 ? PG_STAT_STATEMENTS_COLS_V1_8 :
|
2021-04-08 15:15:17 +02:00
|
|
|
api_version == PGSS_V1_9 ? PG_STAT_STATEMENTS_COLS_V1_9 :
|
2022-04-08 06:12:07 +02:00
|
|
|
api_version == PGSS_V1_10 ? PG_STAT_STATEMENTS_COLS_V1_10 :
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
-1 /* fail if you forget to update this assert */ ));
|
2009-01-04 23:19:59 +01:00
|
|
|
|
Simplify SRFs using materialize mode in contrib/ modules
9e98583 introduced a helper to centralize building their needed state
(tuplestore, tuple descriptors, etc.), checking for any errors. This
commit updates all places of contrib/ that can be switched to use
SetSingleFuncCall() as a drop-in replacement, resulting in the removal
of a lot of boilerplate code in all the modules updated by this commit.
Per analysis, some places remain as they are:
- pg_logdir_ls() in adminpack/ uses historically TYPEFUNC_RECORD as
return type, and I suspect that changing it may cause issues at run-time
with some of its past versions, down to 1.0.
- dblink/ uses a wrapper function doing exactly the work of
SetSingleFuncCall(). Here the switch should be possible, but rather
invasive so it does not seem the extra backpatch maintenance cost.
- tablefunc/, similarly, uses multiple helper functions with portions of
SetSingleFuncCall() spread across the code paths of this module.
Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_bvDPJoL9mH6eYwvBpPtTGQwbDzfJbCM-OjkSZDu5yTPg@mail.gmail.com
2022-03-08 02:12:22 +01:00
|
|
|
tuplestore_putvalues(rsinfo->setResult, rsinfo->setDesc, values, nulls);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
LWLockRelease(pgss->lock);
|
|
|
|
|
2022-06-16 21:50:56 +02:00
|
|
|
free(qbuffer);
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
/* Number of output arguments (columns) for pg_stat_statements_info */
|
|
|
|
#define PG_STAT_STATEMENTS_INFO_COLS 2
|
|
|
|
|
2020-11-26 13:18:05 +01:00
|
|
|
/*
|
|
|
|
* Return statistics of pg_stat_statements.
|
|
|
|
*/
|
|
|
|
Datum
|
|
|
|
pg_stat_statements_info(PG_FUNCTION_ARGS)
|
|
|
|
{
|
|
|
|
pgssGlobalStats stats;
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
TupleDesc tupdesc;
|
2022-07-16 08:42:15 +02:00
|
|
|
Datum values[PG_STAT_STATEMENTS_INFO_COLS] = {0};
|
|
|
|
bool nulls[PG_STAT_STATEMENTS_INFO_COLS] = {0};
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
|
2021-01-28 08:22:34 +01:00
|
|
|
if (!pgss || !pgss_hash)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
|
|
|
|
errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
|
|
|
|
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
/* Build a tuple descriptor for our result type */
|
|
|
|
if (get_call_result_type(fcinfo, NULL, &tupdesc) != TYPEFUNC_COMPOSITE)
|
|
|
|
elog(ERROR, "return type must be a row type");
|
|
|
|
|
2020-11-26 13:18:05 +01:00
|
|
|
/* Read global statistics for pg_stat_statements */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
stats = s->stats;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
values[0] = Int64GetDatum(stats.dealloc);
|
|
|
|
values[1] = TimestampTzGetDatum(stats.stats_reset);
|
|
|
|
|
|
|
|
PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
|
2020-11-26 13:18:05 +01:00
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
|
|
|
* Estimate shared memory space needed.
|
|
|
|
*/
|
|
|
|
static Size
|
|
|
|
pgss_memsize(void)
|
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
Size size;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
size = MAXALIGN(sizeof(pgssSharedState));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
size = add_size(size, hash_estimate_size(pgss_max, sizeof(pgssEntry)));
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
return size;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Allocate a new hashtable entry.
|
|
|
|
* caller must hold an exclusive lock on pgss->lock
|
|
|
|
*
|
2012-03-29 03:00:31 +02:00
|
|
|
* "query" need not be null-terminated; we rely on query_len instead
|
|
|
|
*
|
2012-04-09 17:16:04 +02:00
|
|
|
* If "sticky" is true, make the new entry artificially sticky so that it will
|
|
|
|
* probably still be there when the query finishes execution. We do this by
|
|
|
|
* giving it a median usage value rather than the normal value. (Strictly
|
|
|
|
* speaking, query strings are normalized on a best effort basis, though it
|
|
|
|
* would be difficult to demonstrate this even under artificial conditions.)
|
|
|
|
*
|
2009-01-04 23:19:59 +01:00
|
|
|
* Note: despite needing exclusive lock, it's not an error for the target
|
2014-05-06 18:12:18 +02:00
|
|
|
* entry to already exist. This is because pgss_store releases and
|
2009-01-04 23:19:59 +01:00
|
|
|
* reacquires lock after failing to find a match; so someone else could
|
|
|
|
* have made the entry while we waited to get exclusive lock.
|
|
|
|
*/
|
|
|
|
static pgssEntry *
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
entry_alloc(pgssHashKey *key, Size query_offset, int query_len, int encoding,
|
|
|
|
bool sticky)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
|
|
|
pgssEntry *entry;
|
2009-06-11 16:49:15 +02:00
|
|
|
bool found;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
/* Make space if needed */
|
|
|
|
while (hash_get_num_entries(pgss_hash) >= pgss_max)
|
|
|
|
entry_dealloc();
|
|
|
|
|
|
|
|
/* Find or create an entry with desired hash code */
|
|
|
|
entry = (pgssEntry *) hash_search(pgss_hash, key, HASH_ENTER, &found);
|
|
|
|
|
|
|
|
if (!found)
|
|
|
|
{
|
|
|
|
/* New entry, initialize it */
|
|
|
|
|
|
|
|
/* reset the statistics */
|
|
|
|
memset(&entry->counters, 0, sizeof(Counters));
|
2012-04-09 17:16:04 +02:00
|
|
|
/* set the appropriate initial usage count */
|
|
|
|
entry->counters.usage = sticky ? pgss->cur_median_usage : USAGE_INIT;
|
2009-01-04 23:19:59 +01:00
|
|
|
/* re-initialize the mutex each time ... we assume no one using it */
|
|
|
|
SpinLockInit(&entry->mutex);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* ... and don't forget the query text metadata */
|
|
|
|
Assert(query_len >= 0);
|
|
|
|
entry->query_offset = query_offset;
|
2012-03-29 03:00:31 +02:00
|
|
|
entry->query_len = query_len;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
entry->encoding = encoding;
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
return entry;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* qsort comparator for sorting into increasing usage order
|
|
|
|
*/
|
|
|
|
static int
|
|
|
|
entry_cmp(const void *lhs, const void *rhs)
|
|
|
|
{
|
2012-03-29 03:00:31 +02:00
|
|
|
double l_usage = (*(pgssEntry *const *) lhs)->counters.usage;
|
|
|
|
double r_usage = (*(pgssEntry *const *) rhs)->counters.usage;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
if (l_usage < r_usage)
|
|
|
|
return -1;
|
|
|
|
else if (l_usage > r_usage)
|
|
|
|
return +1;
|
|
|
|
else
|
|
|
|
return 0;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
* Deallocate least-used entries.
|
|
|
|
*
|
2009-01-04 23:19:59 +01:00
|
|
|
* Caller must hold an exclusive lock on pgss->lock.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
entry_dealloc(void)
|
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
HASH_SEQ_STATUS hash_seq;
|
|
|
|
pgssEntry **entries;
|
|
|
|
pgssEntry *entry;
|
|
|
|
int nvictims;
|
|
|
|
int i;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
Size tottextlen;
|
|
|
|
int nvalidtexts;
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2012-04-08 21:49:47 +02:00
|
|
|
/*
|
|
|
|
* Sort entries by usage and deallocate USAGE_DEALLOC_PERCENT of them.
|
|
|
|
* While we're scanning the table, apply the decay factor to the usage
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
* values, and update the mean query length.
|
|
|
|
*
|
|
|
|
* Note that the mean query length is almost immediately obsolete, since
|
|
|
|
* we compute it before not after discarding the least-used entries.
|
|
|
|
* Hopefully, that doesn't affect the mean too much; it doesn't seem worth
|
|
|
|
* making two passes to get a more current result. Likewise, the new
|
|
|
|
* cur_median_usage includes the entries we're about to zap.
|
2012-04-08 21:49:47 +02:00
|
|
|
*/
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
entries = palloc(hash_get_num_entries(pgss_hash) * sizeof(pgssEntry *));
|
|
|
|
|
|
|
|
i = 0;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
tottextlen = 0;
|
|
|
|
nvalidtexts = 0;
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
entries[i++] = entry;
|
2012-04-08 21:49:47 +02:00
|
|
|
/* "Sticky" entries get a different usage decay rate. */
|
Allow pg_stat_statements to track planning statistics.
This commit makes pg_stat_statements support new GUC
pg_stat_statements.track_planning. If this option is enabled,
pg_stat_statements tracks the planning statistics of the statements,
e.g., the number of times the statement was planned, the total time
spent planning the statement, etc. This feature is useful to check
the statements that it takes a long time to plan. Previously since
pg_stat_statements tracked only the execution statistics, we could
not use that for the purpose.
The planning and execution statistics are stored at the end of
each phase separately. So there are not always one-to-one relationship
between them. For example, if the statement is successfully planned
but fails in the execution phase, only its planning statistics are stored.
This may cause the users to be able to see different pg_stat_statements
results from the previous version. To avoid this,
pg_stat_statements.track_planning needs to be disabled.
This commit bumps the version of pg_stat_statements to 1.8
since it changes the definition of pg_stat_statements function.
Author: Julien Rouhaud, Pascal Legrand, Thomas Munro, Fujii Masao
Reviewed-by: Sergei Kornilov, Tomas Vondra, Yoshikazu Imai, Haribabu Kommi, Tom Lane
Discussion: https://postgr.es/m/CAHGQGwFx_=DO-Gu-MfPW3VQ4qC7TfVdH2zHmvZfrGv6fQ3D-Tw@mail.gmail.com
Discussion: https://postgr.es/m/CAEepm=0e59Y_6Q_YXYCTHZkqOc6H2pJ54C_Xe=VFu50Aqqp_sA@mail.gmail.com
Discussion: https://postgr.es/m/DB6PR0301MB21352F6210E3B11934B0DCC790B00@DB6PR0301MB2135.eurprd03.prod.outlook.com
2020-04-02 04:20:19 +02:00
|
|
|
if (IS_STICKY(entry->counters))
|
2012-04-08 21:49:47 +02:00
|
|
|
entry->counters.usage *= STICKY_DECREASE_FACTOR;
|
|
|
|
else
|
|
|
|
entry->counters.usage *= USAGE_DECREASE_FACTOR;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* In the mean length computation, ignore dropped texts. */
|
|
|
|
if (entry->query_len >= 0)
|
|
|
|
{
|
|
|
|
tottextlen += entry->query_len + 1;
|
|
|
|
nvalidtexts++;
|
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* Sort into increasing order by usage */
|
2009-01-04 23:19:59 +01:00
|
|
|
qsort(entries, i, sizeof(pgssEntry *), entry_cmp);
|
2012-04-08 21:49:47 +02:00
|
|
|
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* Record the (approximate) median usage */
|
2012-04-08 21:49:47 +02:00
|
|
|
if (i > 0)
|
|
|
|
pgss->cur_median_usage = entries[i / 2]->counters.usage;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* Record the mean query length */
|
|
|
|
if (nvalidtexts > 0)
|
|
|
|
pgss->mean_query_len = tottextlen / nvalidtexts;
|
|
|
|
else
|
|
|
|
pgss->mean_query_len = ASSUMED_LENGTH_INIT;
|
2012-04-08 21:49:47 +02:00
|
|
|
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* Now zap an appropriate fraction of lowest-usage entries */
|
2009-01-04 23:19:59 +01:00
|
|
|
nvictims = Max(10, i * USAGE_DEALLOC_PERCENT / 100);
|
|
|
|
nvictims = Min(nvictims, i);
|
|
|
|
|
|
|
|
for (i = 0; i < nvictims; i++)
|
|
|
|
{
|
|
|
|
hash_search(pgss_hash, &entries[i]->key, HASH_REMOVE, NULL);
|
|
|
|
}
|
|
|
|
|
|
|
|
pfree(entries);
|
2020-11-26 13:18:05 +01:00
|
|
|
|
|
|
|
/* Increment the number of times entries are deallocated */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
s->stats.dealloc += 1;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
2017-01-14 22:17:30 +01:00
|
|
|
* Given a query string (not necessarily null-terminated), allocate a new
|
|
|
|
* entry in the external query text file and store the string there.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*
|
|
|
|
* If successful, returns true, and stores the new entry's offset in the file
|
2014-05-06 18:12:18 +02:00
|
|
|
* into *query_offset. Also, if gc_count isn't NULL, *gc_count is set to the
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* number of garbage collections that have occurred so far.
|
|
|
|
*
|
|
|
|
* On failure, returns false.
|
|
|
|
*
|
|
|
|
* At least a shared lock on pgss->lock must be held by the caller, so as
|
2014-05-06 18:12:18 +02:00
|
|
|
* to prevent a concurrent garbage collection. Share-lock-holding callers
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* should pass a gc_count pointer to obtain the number of garbage collections,
|
|
|
|
* so that they can recheck the count after obtaining exclusive lock to
|
|
|
|
* detect whether a garbage collection occurred (and removed this entry).
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
qtext_store(const char *query, int query_len,
|
|
|
|
Size *query_offset, int *gc_count)
|
|
|
|
{
|
|
|
|
Size off;
|
|
|
|
int fd;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We use a spinlock to protect extent/n_writers/gc_count, so that
|
|
|
|
* multiple processes may execute this function concurrently.
|
|
|
|
*/
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
off = s->extent;
|
|
|
|
s->extent += query_len + 1;
|
|
|
|
s->n_writers++;
|
|
|
|
if (gc_count)
|
|
|
|
*gc_count = s->gc_count;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
|
|
|
*query_offset = off;
|
|
|
|
|
Be more wary about 32-bit integer overflow in pg_stat_statements.
We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files. It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:
1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits. We still need overflow guards on its use, but
this helps.
2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize. If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.
3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms. It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.
Per report from Bruno da Silva. I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place. But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.
(See also commit 8bbe4cbd9, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)
This issue is old, and is primarily a problem for old platforms,
so back-patch.
Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
2022-08-03 00:05:34 +02:00
|
|
|
/*
|
|
|
|
* Don't allow the file to grow larger than what qtext_load_file can
|
|
|
|
* (theoretically) handle. This has been seen to be reachable on 32-bit
|
|
|
|
* platforms.
|
|
|
|
*/
|
|
|
|
if (unlikely(query_len >= MaxAllocHugeSize - off))
|
|
|
|
{
|
|
|
|
errno = EFBIG; /* not quite right, but it'll do */
|
|
|
|
fd = -1;
|
|
|
|
goto error;
|
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Now write the data into the successfully-reserved part of the file */
|
2017-09-23 15:49:22 +02:00
|
|
|
fd = OpenTransientFile(PGSS_TEXT_FILE, O_RDWR | O_CREAT | PG_BINARY);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (fd < 0)
|
|
|
|
goto error;
|
|
|
|
|
2022-09-29 02:12:11 +02:00
|
|
|
if (pg_pwrite(fd, query, query_len, off) != query_len)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
goto error;
|
2022-09-29 02:12:11 +02:00
|
|
|
if (pg_pwrite(fd, "\0", 1, off + query_len) != 1)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
goto error;
|
|
|
|
|
|
|
|
CloseTransientFile(fd);
|
|
|
|
|
|
|
|
/* Mark our write complete */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
s->n_writers--;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
|
|
|
return true;
|
|
|
|
|
|
|
|
error:
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
|
|
|
|
if (fd >= 0)
|
|
|
|
CloseTransientFile(fd);
|
|
|
|
|
|
|
|
/* Mark our write complete */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
s->n_writers--;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Read the external query text file into a malloc'd buffer.
|
|
|
|
*
|
|
|
|
* Returns NULL (without throwing an error) if unable to read, eg
|
|
|
|
* file not there or insufficient memory.
|
|
|
|
*
|
|
|
|
* On success, the buffer size is also returned into *buffer_size.
|
|
|
|
*
|
|
|
|
* This can be called without any lock on pgss->lock, but in that case
|
|
|
|
* the caller is responsible for verifying that the result is sane.
|
|
|
|
*/
|
|
|
|
static char *
|
|
|
|
qtext_load_file(Size *buffer_size)
|
|
|
|
{
|
|
|
|
char *buf;
|
|
|
|
int fd;
|
|
|
|
struct stat stat;
|
2021-11-01 00:13:48 +01:00
|
|
|
Size nread;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
2017-09-23 15:49:22 +02:00
|
|
|
fd = OpenTransientFile(PGSS_TEXT_FILE, O_RDONLY | PG_BINARY);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
if (fd < 0)
|
|
|
|
{
|
|
|
|
if (errno != ENOENT)
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not read file \"%s\": %m",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Get file length */
|
|
|
|
if (fstat(fd, &stat))
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not stat file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
CloseTransientFile(fd);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Allocate buffer; beware that off_t might be wider than size_t */
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
if (stat.st_size <= MaxAllocHugeSize)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
buf = (char *) malloc(stat.st_size);
|
|
|
|
else
|
|
|
|
buf = NULL;
|
|
|
|
if (buf == NULL)
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode(ERRCODE_OUT_OF_MEMORY),
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
errmsg("out of memory"),
|
2018-07-23 02:19:12 +02:00
|
|
|
errdetail("Could not allocate enough memory to read file \"%s\".",
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
CloseTransientFile(fd);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2021-11-01 00:13:48 +01:00
|
|
|
* OK, slurp in the file. Windows fails if we try to read more than
|
|
|
|
* INT_MAX bytes at once, and other platforms might not like that either,
|
|
|
|
* so read a very large file in 1GB segments.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
2021-11-01 00:13:48 +01:00
|
|
|
nread = 0;
|
|
|
|
while (nread < stat.st_size)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
{
|
2021-11-01 00:13:48 +01:00
|
|
|
int toread = Min(1024 * 1024 * 1024, stat.st_size - nread);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If we get a short read and errno doesn't get set, the reason is
|
|
|
|
* probably that garbage collection truncated the file since we did
|
|
|
|
* the fstat(), so we don't log a complaint --- but we don't return
|
|
|
|
* the data, either, since it's most likely corrupt due to concurrent
|
|
|
|
* writes from garbage collection.
|
|
|
|
*/
|
|
|
|
errno = 0;
|
|
|
|
if (read(fd, buf + nread, toread) != toread)
|
|
|
|
{
|
|
|
|
if (errno)
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
|
|
|
errmsg("could not read file \"%s\": %m",
|
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
free(buf);
|
|
|
|
CloseTransientFile(fd);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
nread += toread;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
}
|
|
|
|
|
2019-07-06 23:18:46 +02:00
|
|
|
if (CloseTransientFile(fd) != 0)
|
Tighten use of OpenTransientFile and CloseTransientFile
This fixes two sets of issues related to the use of transient files in
the backend:
1) OpenTransientFile() has been used in some code paths with read-write
flags while read-only is sufficient, so switch those calls to be
read-only where necessary. These have been reported by Joe Conway.
2) When opening transient files, it is up to the caller to close the
file descriptors opened. In error code paths, CloseTransientFile() gets
called to clean up things before issuing an error. However in normal
exit paths, a lot of callers of CloseTransientFile() never actually
reported errors, which could leave a file descriptor open without
knowing about it. This is an issue I complained about a couple of
times, but never had the courage to write and submit a patch, so here we
go.
Note that one frontend code path is impacted by this commit so as an
error is issued when fetching control file data, making backend and
frontend to be treated consistently.
Reported-by: Joe Conway, Michael Paquier
Author: Michael Paquier
Reviewed-by: Álvaro Herrera, Georgios Kokolatos, Joe Conway
Discussion: https://postgr.es/m/20190301023338.GD1348@paquier.xyz
Discussion: https://postgr.es/m/c49b69ec-e2f7-ff33-4f17-0eaa4f2cef27@joeconway.com
2019-03-09 00:50:55 +01:00
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
|
|
|
errmsg("could not close file \"%s\": %m", PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
2021-11-01 00:13:48 +01:00
|
|
|
*buffer_size = nread;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
return buf;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Locate a query text in the file image previously read by qtext_load_file().
|
|
|
|
*
|
|
|
|
* We validate the given offset/length, and return NULL if bogus. Otherwise,
|
|
|
|
* the result points to a null-terminated string within the buffer.
|
|
|
|
*/
|
|
|
|
static char *
|
|
|
|
qtext_fetch(Size query_offset, int query_len,
|
|
|
|
char *buffer, Size buffer_size)
|
|
|
|
{
|
|
|
|
/* File read failed? */
|
|
|
|
if (buffer == NULL)
|
|
|
|
return NULL;
|
|
|
|
/* Bogus offset/length? */
|
|
|
|
if (query_len < 0 ||
|
|
|
|
query_offset + query_len >= buffer_size)
|
|
|
|
return NULL;
|
|
|
|
/* As a further sanity check, make sure there's a trailing null */
|
|
|
|
if (buffer[query_offset + query_len] != '\0')
|
|
|
|
return NULL;
|
|
|
|
/* Looks OK */
|
|
|
|
return buffer + query_offset;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Do we need to garbage-collect the external query text file?
|
|
|
|
*
|
|
|
|
* Caller should hold at least a shared lock on pgss->lock.
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
need_gc_qtexts(void)
|
|
|
|
{
|
|
|
|
Size extent;
|
|
|
|
|
|
|
|
/* Read shared extent pointer */
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
extent = s->extent;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
Be more wary about 32-bit integer overflow in pg_stat_statements.
We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files. It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:
1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits. We still need overflow guards on its use, but
this helps.
2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize. If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.
3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms. It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.
Per report from Bruno da Silva. I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place. But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.
(See also commit 8bbe4cbd9, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)
This issue is old, and is primarily a problem for old platforms,
so back-patch.
Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
2022-08-03 00:05:34 +02:00
|
|
|
/*
|
|
|
|
* Don't proceed if file does not exceed 512 bytes per possible entry.
|
|
|
|
*
|
|
|
|
* Here and in the next test, 32-bit machines have overflow hazards if
|
|
|
|
* pgss_max and/or mean_query_len are large. Force the multiplications
|
|
|
|
* and comparisons to be done in uint64 arithmetic to forestall trouble.
|
|
|
|
*/
|
|
|
|
if ((uint64) extent < (uint64) 512 * pgss_max)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
return false;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Don't proceed if file is less than about 50% bloat. Nothing can or
|
|
|
|
* should be done in the event of unusually large query texts accounting
|
|
|
|
* for file's large size. We go to the trouble of maintaining the mean
|
|
|
|
* query length in order to prevent garbage collection from thrashing
|
|
|
|
* uselessly.
|
|
|
|
*/
|
Be more wary about 32-bit integer overflow in pg_stat_statements.
We've heard a couple of reports of people having trouble with
multi-gigabyte-sized query-texts files. It occurred to me that on
32-bit platforms, there could be an issue with integer overflow
of calculations associated with the total query text size.
Address that with several changes:
1. Limit pg_stat_statements.max to INT_MAX / 2 not INT_MAX.
The hashtable code will bound it to that anyway unless "long"
is 64 bits. We still need overflow guards on its use, but
this helps.
2. Add a check to prevent extending the query-texts file to
more than MaxAllocHugeSize. If it got that big, qtext_load_file
would certainly fail, so there's not much point in allowing it.
Without this, we'd need to consider whether extent, query_offset,
and related variables shouldn't be off_t not size_t.
3. Adjust the comparisons in need_gc_qtexts() to be done in 64-bit
arithmetic on all platforms. It appears possible that under duress
those multiplications could overflow 32 bits, yielding a false
conclusion that we need to garbage-collect the texts file, which
could lead to repeatedly garbage-collecting after every hash table
insertion.
Per report from Bruno da Silva. I'm not convinced that these
issues fully explain his problem; there may be some other bug that's
contributing to the query-texts file becoming so large in the first
place. But it did get that big, so #2 is a reasonable defense,
and #3 could explain the reported performance difficulties.
(See also commit 8bbe4cbd9, which addressed some related bugs.
The second Discussion: link is the thread that led up to that.)
This issue is old, and is primarily a problem for old platforms,
so back-patch.
Discussion: https://postgr.es/m/CAB+Nuk93fL1Q9eLOCotvLP07g7RAv4vbdrkm0cVQohDVMpAb9A@mail.gmail.com
Discussion: https://postgr.es/m/5601D354.5000703@BlueTreble.com
2022-08-03 00:05:34 +02:00
|
|
|
if ((uint64) extent < (uint64) pgss->mean_query_len * pgss_max * 2)
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
return false;
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Garbage-collect orphaned query texts in external file.
|
|
|
|
*
|
|
|
|
* This won't be called often in the typical case, since it's likely that
|
|
|
|
* there won't be too much churn, and besides, a similar compaction process
|
|
|
|
* occurs when serializing to disk at shutdown or as part of resetting.
|
|
|
|
* Despite this, it seems prudent to plan for the edge case where the file
|
|
|
|
* becomes unreasonably large, with no other method of compaction likely to
|
|
|
|
* occur in the foreseeable future.
|
|
|
|
*
|
|
|
|
* The caller must hold an exclusive lock on pgss->lock.
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
*
|
|
|
|
* At the first sign of trouble we unlink the query text file to get a clean
|
|
|
|
* slate (although existing statistics are retained), rather than risk
|
|
|
|
* thrashing by allowing the same problem case to recur indefinitely.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
|
|
|
static void
|
|
|
|
gc_qtexts(void)
|
|
|
|
{
|
|
|
|
char *qbuffer;
|
|
|
|
Size qbuffer_size;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
FILE *qfile = NULL;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
HASH_SEQ_STATUS hash_seq;
|
|
|
|
pgssEntry *entry;
|
|
|
|
Size extent;
|
|
|
|
int nentries;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* When called from pgss_store, some other session might have proceeded
|
|
|
|
* with garbage collection in the no-lock-held interim of lock strength
|
2014-05-06 18:12:18 +02:00
|
|
|
* escalation. Check once more that this is actually necessary.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
|
|
|
if (!need_gc_qtexts())
|
|
|
|
return;
|
|
|
|
|
|
|
|
/*
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
* Load the old texts file. If we fail (out of memory, for instance),
|
|
|
|
* invalidate query texts. Hopefully this is rare. It might seem better
|
|
|
|
* to leave things alone on an OOM failure, but the problem is that the
|
|
|
|
* file is only going to get bigger; hoping for a future non-OOM result is
|
|
|
|
* risky and can easily lead to complete denial of service.
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*/
|
|
|
|
qbuffer = qtext_load_file(&qbuffer_size);
|
|
|
|
if (qbuffer == NULL)
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
goto gc_fail;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* We overwrite the query texts file in place, so as to reduce the risk of
|
|
|
|
* an out-of-disk-space failure. Since the file is guaranteed not to get
|
|
|
|
* larger, this should always work on traditional filesystems; though we
|
|
|
|
* could still lose on copy-on-write filesystems.
|
|
|
|
*/
|
|
|
|
qfile = AllocateFile(PGSS_TEXT_FILE, PG_BINARY_W);
|
|
|
|
if (qfile == NULL)
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
goto gc_fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
extent = 0;
|
|
|
|
nentries = 0;
|
|
|
|
|
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
int query_len = entry->query_len;
|
|
|
|
char *qry = qtext_fetch(entry->query_offset,
|
|
|
|
query_len,
|
|
|
|
qbuffer,
|
|
|
|
qbuffer_size);
|
|
|
|
|
|
|
|
if (qry == NULL)
|
|
|
|
{
|
|
|
|
/* Trouble ... drop the text */
|
|
|
|
entry->query_offset = 0;
|
|
|
|
entry->query_len = -1;
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/* entry will not be counted in mean query length computation */
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
|
|
|
if (fwrite(qry, 1, query_len + 1, qfile) != query_len + 1)
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
hash_seq_term(&hash_seq);
|
|
|
|
goto gc_fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
entry->query_offset = extent;
|
|
|
|
extent += query_len + 1;
|
|
|
|
nentries++;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Truncate away any now-unused space. If this fails for some odd reason,
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* we log it, but there's no need to fail.
|
|
|
|
*/
|
|
|
|
if (ftruncate(fileno(qfile), extent) != 0)
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not truncate file \"%s\": %m",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
if (FreeFile(qfile))
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not write file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
qfile = NULL;
|
|
|
|
goto gc_fail;
|
|
|
|
}
|
|
|
|
|
|
|
|
elog(DEBUG1, "pgss gc of queries file shrunk size from %zu to %zu",
|
|
|
|
pgss->extent, extent);
|
|
|
|
|
|
|
|
/* Reset the shared extent pointer */
|
|
|
|
pgss->extent = extent;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Also update the mean query length, to be sure that need_gc_qtexts()
|
|
|
|
* won't still think we have a problem.
|
|
|
|
*/
|
|
|
|
if (nentries > 0)
|
|
|
|
pgss->mean_query_len = extent / nentries;
|
|
|
|
else
|
|
|
|
pgss->mean_query_len = ASSUMED_LENGTH_INIT;
|
|
|
|
|
|
|
|
free(qbuffer);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* OK, count a garbage collection cycle. (Note: even though we have
|
|
|
|
* exclusive lock on pgss->lock, we must take pgss->mutex for this, since
|
|
|
|
* other processes may examine gc_count while holding only the mutex.
|
|
|
|
* Also, we have to advance the count *after* we've rewritten the file,
|
|
|
|
* else other processes might not realize they read a stale file.)
|
|
|
|
*/
|
|
|
|
record_gc_qtexts();
|
|
|
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
gc_fail:
|
|
|
|
/* clean up resources */
|
|
|
|
if (qfile)
|
|
|
|
FreeFile(qfile);
|
2022-06-16 21:50:56 +02:00
|
|
|
free(qbuffer);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Since the contents of the external file are now uncertain, mark all
|
|
|
|
* hashtable entries as having invalid texts.
|
|
|
|
*/
|
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
entry->query_offset = 0;
|
|
|
|
entry->query_len = -1;
|
|
|
|
}
|
|
|
|
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
/*
|
|
|
|
* Destroy the query text file and create a new, empty one
|
|
|
|
*/
|
|
|
|
(void) unlink(PGSS_TEXT_FILE);
|
|
|
|
qfile = AllocateFile(PGSS_TEXT_FILE, PG_BINARY_W);
|
|
|
|
if (qfile == NULL)
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not recreate file \"%s\": %m",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Improve contrib/pg_stat_statements' handling of garbage collection failure.
If we can't read the query texts file (whether because out-of-memory, or
for some other reason), give up and reset the file to empty, discarding all
stored query texts, though not the statistics per se. We used to leave
things alone and hope for better luck next time, but the problem is that
the file is only going to get bigger and even harder to slurp into memory.
Better to do something that will get us out of trouble.
Likewise reset the file to empty for any other failure within gc_qtexts().
The previous behavior after a write error was to discard query texts but
not do anything to truncate the file, which is just weird.
Also, increase the maximum supported file size from MaxAllocSize to
MaxAllocHugeSize; this makes it more likely we'll be able to do a garbage
collection successfully.
Also, fix recalculation of mean_query_len within entry_dealloc() to match
the calculation in gc_qtexts(). The previous coding overlooked the
possibility of dropped texts (query_len == -1) and would underestimate the
mean of the remaining entries in such cases, thus possibly causing excess
garbage collection cycles.
In passing, add some errdetail to the log entry that complains about
insufficient memory to read the query texts file, which after all was
Jim Nasby's original complaint.
Back-patch to 9.4 where the current handling of query texts was
introduced.
Peter Geoghegan, rather editorialized upon by me
2015-10-04 23:58:29 +02:00
|
|
|
else
|
|
|
|
FreeFile(qfile);
|
|
|
|
|
|
|
|
/* Reset the shared extent pointer */
|
|
|
|
pgss->extent = 0;
|
|
|
|
|
|
|
|
/* Reset mean_query_len to match the new state */
|
|
|
|
pgss->mean_query_len = ASSUMED_LENGTH_INIT;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Bump the GC count even though we failed.
|
|
|
|
*
|
|
|
|
* This is needed to make concurrent readers of file without any lock on
|
|
|
|
* pgss->lock notice existence of new version of file. Once readers
|
|
|
|
* subsequently observe a change in GC count with pgss->lock held, that
|
|
|
|
* forces a safe reopen of file. Writers also require that we bump here,
|
|
|
|
* of course. (As required by locking protocol, readers and writers don't
|
|
|
|
* trust earlier file contents until gc_count is found unchanged after
|
|
|
|
* pgss->lock acquired in shared or exclusive mode respectively.)
|
|
|
|
*/
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
record_gc_qtexts();
|
|
|
|
}
|
|
|
|
|
2009-01-04 23:19:59 +01:00
|
|
|
/*
|
2019-01-11 04:20:09 +01:00
|
|
|
* Release entries corresponding to parameters passed.
|
2009-01-04 23:19:59 +01:00
|
|
|
*/
|
|
|
|
static void
|
2019-01-11 04:20:09 +01:00
|
|
|
entry_reset(Oid userid, Oid dbid, uint64 queryid)
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
2009-06-11 16:49:15 +02:00
|
|
|
HASH_SEQ_STATUS hash_seq;
|
|
|
|
pgssEntry *entry;
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
FILE *qfile;
|
2019-01-11 04:20:09 +01:00
|
|
|
long num_entries;
|
|
|
|
long num_remove = 0;
|
|
|
|
pgssHashKey key;
|
|
|
|
|
|
|
|
if (!pgss || !pgss_hash)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
|
|
|
|
errmsg("pg_stat_statements must be loaded via shared_preload_libraries")));
|
2009-01-04 23:19:59 +01:00
|
|
|
|
|
|
|
LWLockAcquire(pgss->lock, LW_EXCLUSIVE);
|
2019-01-11 04:20:09 +01:00
|
|
|
num_entries = hash_get_num_entries(pgss_hash);
|
2009-01-04 23:19:59 +01:00
|
|
|
|
2019-01-11 04:20:09 +01:00
|
|
|
if (userid != 0 && dbid != 0 && queryid != UINT64CONST(0))
|
|
|
|
{
|
|
|
|
/* If all the parameters are available, use the fast path. */
|
2021-04-08 10:23:10 +02:00
|
|
|
memset(&key, 0, sizeof(pgssHashKey));
|
2019-01-11 04:20:09 +01:00
|
|
|
key.userid = userid;
|
|
|
|
key.dbid = dbid;
|
|
|
|
key.queryid = queryid;
|
|
|
|
|
2021-04-08 10:23:10 +02:00
|
|
|
/* Remove the key if it exists, starting with the top-level entry */
|
|
|
|
key.toplevel = false;
|
|
|
|
entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_REMOVE, NULL);
|
|
|
|
if (entry) /* found */
|
|
|
|
num_remove++;
|
|
|
|
|
|
|
|
/* Also remove entries for top level statements */
|
|
|
|
key.toplevel = true;
|
|
|
|
|
2019-01-11 04:20:09 +01:00
|
|
|
/* Remove the key if exists */
|
|
|
|
entry = (pgssEntry *) hash_search(pgss_hash, &key, HASH_REMOVE, NULL);
|
|
|
|
if (entry) /* found */
|
|
|
|
num_remove++;
|
|
|
|
}
|
|
|
|
else if (userid != 0 || dbid != 0 || queryid != UINT64CONST(0))
|
2009-01-04 23:19:59 +01:00
|
|
|
{
|
2019-01-11 04:20:09 +01:00
|
|
|
/* Remove entries corresponding to valid parameters. */
|
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
if ((!userid || entry->key.userid == userid) &&
|
|
|
|
(!dbid || entry->key.dbid == dbid) &&
|
|
|
|
(!queryid || entry->key.queryid == queryid))
|
|
|
|
{
|
|
|
|
hash_search(pgss_hash, &entry->key, HASH_REMOVE, NULL);
|
|
|
|
num_remove++;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Remove all entries. */
|
|
|
|
hash_seq_init(&hash_seq, pgss_hash);
|
|
|
|
while ((entry = hash_seq_search(&hash_seq)) != NULL)
|
|
|
|
{
|
|
|
|
hash_search(pgss_hash, &entry->key, HASH_REMOVE, NULL);
|
|
|
|
num_remove++;
|
|
|
|
}
|
2009-01-04 23:19:59 +01:00
|
|
|
}
|
|
|
|
|
2019-01-11 04:20:09 +01:00
|
|
|
/* All entries are removed? */
|
|
|
|
if (num_entries != num_remove)
|
|
|
|
goto release_lock;
|
|
|
|
|
pg_stat_statements: Track time at which all statistics were last reset.
This commit adds "stats_reset" column into the pg_stat_statements_info
view. This column indicates the time at which all statistics in the
pg_stat_statements view were last reset.
Per discussion, this commit also changes pg_stat_statements_info code
so that "dealloc" column is reset at the same time as "stats_reset" is reset,
i.e., whenever all pg_stat_statements entries are removed, for the sake
of consistency. Previously "dealloc" was reset only when
pg_stat_statements_reset(0, 0, 0) is called and was not reset when
pg_stat_statements_reset() with non-zero value argument discards all
entries. This was confusing.
Author: Naoki Nakamichi, Yuki Seino
Reviewed-by: Yuki Seino, Kyotaro Horiguchi, Li Japin, Fujii Masao
Discussion: https://postgr.es/m/c102cf3180d0ee73c1c5a0f7f8558322@oss.nttdata.com
2020-12-18 02:49:58 +01:00
|
|
|
/*
|
|
|
|
* Reset global statistics for pg_stat_statements since all entries are
|
|
|
|
* removed.
|
|
|
|
*/
|
|
|
|
{
|
|
|
|
volatile pgssSharedState *s = (volatile pgssSharedState *) pgss;
|
|
|
|
TimestampTz stats_reset = GetCurrentTimestamp();
|
|
|
|
|
|
|
|
SpinLockAcquire(&s->mutex);
|
|
|
|
s->stats.dealloc = 0;
|
|
|
|
s->stats.stats_reset = stats_reset;
|
|
|
|
SpinLockRelease(&s->mutex);
|
|
|
|
}
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/*
|
|
|
|
* Write new empty query file, perhaps even creating a new one to recover
|
|
|
|
* if the file was missing.
|
|
|
|
*/
|
|
|
|
qfile = AllocateFile(PGSS_TEXT_FILE, PG_BINARY_W);
|
|
|
|
if (qfile == NULL)
|
|
|
|
{
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not create file \"%s\": %m",
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
PGSS_TEXT_FILE)));
|
|
|
|
goto done;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* If ftruncate fails, log it, but it's not a fatal problem */
|
|
|
|
if (ftruncate(fileno(qfile), 0) != 0)
|
|
|
|
ereport(LOG,
|
|
|
|
(errcode_for_file_access(),
|
2018-07-23 02:19:12 +02:00
|
|
|
errmsg("could not truncate file \"%s\": %m",
|
Phase 3 of pgindent updates.
Don't move parenthesized lines to the left, even if that means they
flow past the right margin.
By default, BSD indent lines up statement continuation lines that are
within parentheses so that they start just to the right of the preceding
left parenthesis. However, traditionally, if that resulted in the
continuation line extending to the right of the desired right margin,
then indent would push it left just far enough to not overrun the margin,
if it could do so without making the continuation line start to the left of
the current statement indent. That makes for a weird mix of indentations
unless one has been completely rigid about never violating the 80-column
limit.
This behavior has been pretty universally panned by Postgres developers.
Hence, disable it with indent's new -lpl switch, so that parenthesized
lines are always lined up with the preceding left paren.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
|
|
|
PGSS_TEXT_FILE)));
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
|
|
|
|
FreeFile(qfile);
|
|
|
|
|
|
|
|
done:
|
|
|
|
pgss->extent = 0;
|
|
|
|
/* This counts as a query text garbage collection for our purposes */
|
|
|
|
record_gc_qtexts();
|
|
|
|
|
2019-01-11 04:20:09 +01:00
|
|
|
release_lock:
|
2009-01-04 23:19:59 +01:00
|
|
|
LWLockRelease(pgss->lock);
|
|
|
|
}
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Generate a normalized version of the query string that will be used to
|
|
|
|
* represent all similar queries.
|
|
|
|
*
|
|
|
|
* Note that the normalized representation may well vary depending on
|
|
|
|
* just which "equivalent" query is used to create the hashtable entry.
|
|
|
|
* We assume this is OK.
|
|
|
|
*
|
2017-01-14 22:17:30 +01:00
|
|
|
* If query_loc > 0, then "query" has been advanced by that much compared to
|
|
|
|
* the original string start, so we need to translate the provided locations
|
|
|
|
* to compensate. (This lets us avoid re-scanning statements before the one
|
|
|
|
* of interest, so it's worth doing.)
|
|
|
|
*
|
2012-03-29 03:00:31 +02:00
|
|
|
* *query_len_p contains the input string length, and is updated with
|
2017-03-28 02:14:36 +02:00
|
|
|
* the result string length on exit. The resulting string might be longer
|
|
|
|
* or shorter depending on what happens with replacement of constants.
|
2012-03-29 03:00:31 +02:00
|
|
|
*
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* Returns a palloc'd string.
|
2012-03-29 03:00:31 +02:00
|
|
|
*/
|
|
|
|
static char *
|
2021-04-07 19:06:47 +02:00
|
|
|
generate_normalized_query(JumbleState *jstate, const char *query,
|
2020-09-08 10:08:46 +02:00
|
|
|
int query_loc, int *query_len_p)
|
2012-03-29 03:00:31 +02:00
|
|
|
{
|
|
|
|
char *norm_query;
|
|
|
|
int query_len = *query_len_p;
|
|
|
|
int i,
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
norm_query_buflen, /* Space allowed for norm_query */
|
2012-03-29 03:00:31 +02:00
|
|
|
len_to_wrt, /* Length (in bytes) to write */
|
|
|
|
quer_loc = 0, /* Source query byte location */
|
|
|
|
n_quer_loc = 0, /* Normalized query byte location */
|
|
|
|
last_off = 0, /* Offset from start for previous tok */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
last_tok_len = 0; /* Length (in bytes) of that tok */
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Get constants' lengths (core system only gives us locations). Note
|
|
|
|
* this also ensures the items are sorted by location.
|
|
|
|
*/
|
2017-01-14 22:17:30 +01:00
|
|
|
fill_in_constant_lengths(jstate, query, query_loc);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
2017-03-28 02:14:36 +02:00
|
|
|
/*
|
|
|
|
* Allow for $n symbols to be longer than the constants they replace.
|
|
|
|
* Constants must take at least one byte in text form, while a $n symbol
|
|
|
|
* certainly isn't more than 11 bytes, even if n reaches INT_MAX. We
|
|
|
|
* could refine that limit based on the max value of n for the current
|
|
|
|
* query, but it hardly seems worth any extra effort to do so.
|
|
|
|
*/
|
|
|
|
norm_query_buflen = query_len + jstate->clocations_count * 10;
|
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
/* Allocate result buffer */
|
2017-03-28 02:14:36 +02:00
|
|
|
norm_query = palloc(norm_query_buflen + 1);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
for (i = 0; i < jstate->clocations_count; i++)
|
|
|
|
{
|
|
|
|
int off, /* Offset from start for cur tok */
|
|
|
|
tok_len; /* Length (in bytes) of that tok */
|
|
|
|
|
|
|
|
off = jstate->clocations[i].location;
|
2017-01-14 22:17:30 +01:00
|
|
|
/* Adjust recorded location if we're dealing with partial string */
|
|
|
|
off -= query_loc;
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
tok_len = jstate->clocations[i].length;
|
|
|
|
|
|
|
|
if (tok_len < 0)
|
|
|
|
continue; /* ignore any duplicates */
|
|
|
|
|
2014-01-28 18:34:29 +01:00
|
|
|
/* Copy next chunk (what precedes the next constant) */
|
2012-03-29 03:00:31 +02:00
|
|
|
len_to_wrt = off - last_off;
|
|
|
|
len_to_wrt -= last_tok_len;
|
|
|
|
|
|
|
|
Assert(len_to_wrt >= 0);
|
|
|
|
memcpy(norm_query + n_quer_loc, query + quer_loc, len_to_wrt);
|
|
|
|
n_quer_loc += len_to_wrt;
|
|
|
|
|
2017-03-28 02:14:36 +02:00
|
|
|
/* And insert a param symbol in place of the constant token */
|
|
|
|
n_quer_loc += sprintf(norm_query + n_quer_loc, "$%d",
|
|
|
|
i + 1 + jstate->highest_extern_param_id);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
quer_loc = off + tok_len;
|
|
|
|
last_off = off;
|
|
|
|
last_tok_len = tok_len;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We've copied up until the last ignorable constant. Copy over the
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
* remaining bytes of the original query string.
|
2012-03-29 03:00:31 +02:00
|
|
|
*/
|
|
|
|
len_to_wrt = query_len - quer_loc;
|
|
|
|
|
|
|
|
Assert(len_to_wrt >= 0);
|
|
|
|
memcpy(norm_query + n_quer_loc, query + quer_loc, len_to_wrt);
|
|
|
|
n_quer_loc += len_to_wrt;
|
|
|
|
|
2017-03-28 02:14:36 +02:00
|
|
|
Assert(n_quer_loc <= norm_query_buflen);
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
norm_query[n_quer_loc] = '\0';
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Keep pg_stat_statements' query texts in a file, not in shared memory.
This change allows us to eliminate the previous limit on stored query
length, and it makes the shared-memory hash table very much smaller,
allowing more statements to be tracked. (The default value of
pg_stat_statements.max is therefore increased from 1000 to 5000.)
In typical scenarios, the hash table can be large enough to hold all the
statements commonly issued by an application, so that there is little
"churn" in the set of tracked statements, and thus little need to do I/O
to the file.
To further reduce the need for I/O to the query-texts file, add a way
to retrieve all the columns of the pg_stat_statements view except for
the query text column. This is probably not of much interest for human
use but it could be exploited by programs, which will prefer using the
queryid anyway.
Ordinarily, we'd need to bump the extension version number for the latter
change. But since we already advanced pg_stat_statements' version number
from 1.1 to 1.2 in the 9.4 development cycle, it seems all right to just
redefine what 1.2 means.
Peter Geoghegan, reviewed by Pavel Stehule
2014-01-27 21:37:54 +01:00
|
|
|
*query_len_p = n_quer_loc;
|
2012-03-29 03:00:31 +02:00
|
|
|
return norm_query;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Given a valid SQL string and an array of constant-location records,
|
|
|
|
* fill in the textual lengths of those constants.
|
|
|
|
*
|
|
|
|
* The constants may use any allowed constant syntax, such as float literals,
|
|
|
|
* bit-strings, single-quoted strings and dollar-quoted strings. This is
|
|
|
|
* accomplished by using the public API for the core scanner.
|
|
|
|
*
|
|
|
|
* It is the caller's job to ensure that the string is a valid SQL statement
|
|
|
|
* with constants at the indicated locations. Since in practice the string
|
|
|
|
* has already been parsed, and the locations that the caller provides will
|
|
|
|
* have originated from within the authoritative parser, this should not be
|
|
|
|
* a problem.
|
|
|
|
*
|
|
|
|
* Duplicate constant pointers are possible, and will have their lengths
|
2014-05-06 18:12:18 +02:00
|
|
|
* marked as '-1', so that they are later ignored. (Actually, we assume the
|
2012-03-29 03:00:31 +02:00
|
|
|
* lengths were initialized as -1 to start with, and don't change them here.)
|
|
|
|
*
|
2017-01-14 22:17:30 +01:00
|
|
|
* If query_loc > 0, then "query" has been advanced by that much compared to
|
|
|
|
* the original string start, so we need to translate the provided locations
|
|
|
|
* to compensate. (This lets us avoid re-scanning statements before the one
|
|
|
|
* of interest, so it's worth doing.)
|
|
|
|
*
|
2012-03-29 03:00:31 +02:00
|
|
|
* N.B. There is an assumption that a '-' character at a Const location begins
|
|
|
|
* a negative numeric constant. This precludes there ever being another
|
|
|
|
* reason for a constant to start with a '-'.
|
|
|
|
*/
|
|
|
|
static void
|
2021-04-07 19:06:47 +02:00
|
|
|
fill_in_constant_lengths(JumbleState *jstate, const char *query,
|
2017-01-14 22:17:30 +01:00
|
|
|
int query_loc)
|
2012-03-29 03:00:31 +02:00
|
|
|
{
|
2021-04-07 19:06:47 +02:00
|
|
|
LocationLen *locs;
|
2012-03-29 03:00:31 +02:00
|
|
|
core_yyscan_t yyscanner;
|
|
|
|
core_yy_extra_type yyextra;
|
|
|
|
core_YYSTYPE yylval;
|
|
|
|
YYLTYPE yylloc;
|
|
|
|
int last_loc = -1;
|
|
|
|
int i;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Sort the records by location so that we can process them in order while
|
|
|
|
* scanning the query text.
|
|
|
|
*/
|
|
|
|
if (jstate->clocations_count > 1)
|
|
|
|
qsort(jstate->clocations, jstate->clocations_count,
|
2021-04-07 19:06:47 +02:00
|
|
|
sizeof(LocationLen), comp_location);
|
2012-03-29 03:00:31 +02:00
|
|
|
locs = jstate->clocations;
|
|
|
|
|
|
|
|
/* initialize the flex scanner --- should match raw_parser() */
|
|
|
|
yyscanner = scanner_init(query,
|
|
|
|
&yyextra,
|
Replace the data structure used for keyword lookup.
Previously, ScanKeywordLookup was passed an array of string pointers.
This had some performance deficiencies: the strings themselves might
be scattered all over the place depending on the compiler (and some
quick checking shows that at least with gcc-on-Linux, they indeed
weren't reliably close together). That led to very cache-unfriendly
behavior as the binary search touched strings in many different pages.
Also, depending on the platform, the string pointers might need to
be adjusted at program start, so that they couldn't be simple constant
data. And the ScanKeyword struct had been designed with an eye to
32-bit machines originally; on 64-bit it requires 16 bytes per
keyword, making it even more cache-unfriendly.
Redesign so that the keyword strings themselves are allocated
consecutively (as part of one big char-string constant), thereby
eliminating the touch-lots-of-unrelated-pages syndrome. And get
rid of the ScanKeyword array in favor of three separate arrays:
uint16 offsets into the keyword array, uint16 token codes, and
uint8 keyword categories. That reduces the overhead per keyword
to 5 bytes instead of 16 (even less in programs that only need
one of the token codes and categories); moreover, the binary search
only touches the offsets array, further reducing its cache footprint.
This also lets us put the token codes somewhere else than the
keyword strings are, which avoids some unpleasant build dependencies.
While we're at it, wrap the data used by ScanKeywordLookup into
a struct that can be treated as an opaque type by most callers.
That doesn't change things much right now, but it will make it
less painful to switch to a hash-based lookup method, as is being
discussed in the mailing list thread.
Most of the change here is associated with adding a generator
script that can build the new data structure from the same
list-of-PG_KEYWORD header representation we used before.
The PG_KEYWORD lists that plpgsql and ecpg used to embed in
their scanner .c files have to be moved into headers, and the
Makefiles have to be taught to invoke the generator script.
This work is also necessary if we're to consider hash-based lookup,
since the generator script is what would be responsible for
constructing a hash table.
Aside from saving a few kilobytes in each program that includes
the keyword table, this seems to speed up raw parsing (flex+bison)
by a few percent. So it's worth doing even as it stands, though
we think we can gain even more with a follow-on patch to switch
to hash-based lookup.
John Naylor, with further hacking by me
Discussion: https://postgr.es/m/CAJVSVGXdFVU2sgym89XPL=Lv1zOS5=EHHQ8XWNzFL=mTXkKMLw@mail.gmail.com
2019-01-06 23:02:57 +01:00
|
|
|
&ScanKeywords,
|
|
|
|
ScanKeywordTokens);
|
2012-03-29 03:00:31 +02:00
|
|
|
|
Prevent duplicate escape-string warnings when using pg_stat_statements.
contrib/pg_stat_statements will sometimes run the core lexer a second time
on submitted statements. Formerly, if you had standard_conforming_strings
turned off, this led to sometimes getting two copies of any warnings
enabled by escape_string_warning. While this is probably no longer a big
deal in the field, it's a pain for regression testing.
To fix, change the lexer so it doesn't consult the escape_string_warning
GUC variable directly, but looks at a copy in the core_yy_extra_type state
struct. Then, pg_stat_statements can change that copy to disable warnings
while it's redoing the lexing.
It seemed like a good idea to make this happen for all three of the GUCs
consulted by the lexer, not just escape_string_warning. There's not an
immediate use-case for callers to adjust the other two AFAIK, but making
it possible is easy enough and seems like good future-proofing.
Arguably this is a bug fix, but there doesn't seem to be enough interest to
justify a back-patch. We'd not be able to back-patch exactly as-is anyway,
for fear of breaking ABI compatibility of the struct. (We could perhaps
back-patch the addition of only escape_string_warning by adding it at the
end of the struct, where there's currently alignment padding space.)
2015-01-23 00:10:47 +01:00
|
|
|
/* we don't want to re-emit any escape string warnings */
|
|
|
|
yyextra.escape_string_warning = false;
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
/* Search for each constant, in sequence */
|
|
|
|
for (i = 0; i < jstate->clocations_count; i++)
|
|
|
|
{
|
|
|
|
int loc = locs[i].location;
|
|
|
|
int tok;
|
|
|
|
|
2017-01-14 22:17:30 +01:00
|
|
|
/* Adjust recorded location if we're dealing with partial string */
|
|
|
|
loc -= query_loc;
|
|
|
|
|
2012-03-29 03:00:31 +02:00
|
|
|
Assert(loc >= 0);
|
|
|
|
|
|
|
|
if (loc <= last_loc)
|
|
|
|
continue; /* Duplicate constant, ignore */
|
|
|
|
|
|
|
|
/* Lex tokens until we find the desired constant */
|
|
|
|
for (;;)
|
|
|
|
{
|
|
|
|
tok = core_yylex(&yylval, &yylloc, yyscanner);
|
|
|
|
|
|
|
|
/* We should not hit end-of-string, but if we do, behave sanely */
|
|
|
|
if (tok == 0)
|
|
|
|
break; /* out of inner for-loop */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We should find the token position exactly, but if we somehow
|
|
|
|
* run past it, work with that.
|
|
|
|
*/
|
|
|
|
if (yylloc >= loc)
|
|
|
|
{
|
|
|
|
if (query[loc] == '-')
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* It's a negative value - this is the one and only case
|
|
|
|
* where we replace more than a single token.
|
|
|
|
*
|
|
|
|
* Do not compensate for the core system's special-case
|
|
|
|
* adjustment of location to that of the leading '-'
|
|
|
|
* operator in the event of a negative constant. It is
|
|
|
|
* also useful for our purposes to start from the minus
|
2014-05-06 18:12:18 +02:00
|
|
|
* symbol. In this way, queries like "select * from foo
|
2012-03-29 03:00:31 +02:00
|
|
|
* where bar = 1" and "select * from foo where bar = -2"
|
|
|
|
* will have identical normalized query strings.
|
|
|
|
*/
|
|
|
|
tok = core_yylex(&yylval, &yylloc, yyscanner);
|
|
|
|
if (tok == 0)
|
|
|
|
break; /* out of inner for-loop */
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* We now rely on the assumption that flex has placed a zero
|
|
|
|
* byte after the text of the current token in scanbuf.
|
|
|
|
*/
|
|
|
|
locs[i].length = strlen(yyextra.scanbuf + loc);
|
|
|
|
break; /* out of inner for-loop */
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/* If we hit end-of-string, give up, leaving remaining lengths -1 */
|
|
|
|
if (tok == 0)
|
|
|
|
break;
|
|
|
|
|
|
|
|
last_loc = loc;
|
|
|
|
}
|
|
|
|
|
|
|
|
scanner_finish(yyscanner);
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2021-04-07 19:06:47 +02:00
|
|
|
* comp_location: comparator for qsorting LocationLen structs by location
|
2012-03-29 03:00:31 +02:00
|
|
|
*/
|
|
|
|
static int
|
|
|
|
comp_location(const void *a, const void *b)
|
|
|
|
{
|
2021-04-07 19:06:47 +02:00
|
|
|
int l = ((const LocationLen *) a)->location;
|
|
|
|
int r = ((const LocationLen *) b)->location;
|
2012-03-29 03:00:31 +02:00
|
|
|
|
|
|
|
if (l < r)
|
|
|
|
return -1;
|
|
|
|
else if (l > r)
|
|
|
|
return +1;
|
|
|
|
else
|
|
|
|
return 0;
|
|
|
|
}
|