postgresql/src/backend
Andres Freund 5891c7a8ed pgstat: store statistics in shared memory.
Previously the statistics collector received statistics updates via UDP and
shared statistics data by writing them out to temporary files regularly. These
files can reach tens of megabytes and are written out up to twice a
second. This has repeatedly prevented us from adding additional useful
statistics.

Now statistics are stored in shared memory. Statistics for variable-numbered
objects are stored in a dshash hashtable (backed by dynamic shared
memory). Fixed-numbered stats are stored in plain shared memory.

The header for pgstat.c contains an overview of the architecture.

The stats collector is not needed anymore, remove it.

By utilizing the transactional statistics drop infrastructure introduced in a
prior commit statistics entries cannot "leak" anymore. Previously leaked
statistics were dropped by pgstat_vacuum_stat(), called from [auto-]vacuum. On
systems with many small relations pgstat_vacuum_stat() could be quite
expensive.

Now that replicas drop statistics entries for dropped objects, it is not
necessary anymore to reset stats when starting from a cleanly shut down
replica.

Subsequent commits will perform some further code cleanup, adapt docs and add
tests.

Bumps PGSTAT_FILE_FORMAT_ID.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Andres Freund <andres@anarazel.de>
Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Thomas Munro <thomas.munro@gmail.com>
Reviewed-By: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-By: "David G. Johnston" <david.g.johnston@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@2ndquadrant.com> (in a much earlier version)
Reviewed-By: Arthur Zakirov <a.zakirov@postgrespro.ru> (in a much earlier version)
Reviewed-By: Antonin Houska <ah@cybertec.at> (in a much earlier version)
Discussion: https://postgr.es/m/20220303021600.hs34ghqcw6zcokdh@alap3.anarazel.de
Discussion: https://postgr.es/m/20220308205351.2xcn6k4x5yivcxyd@alap3.anarazel.de
Discussion: https://postgr.es/m/20210319235115.y3wz7hpnnrshdyv6@alap3.anarazel.de
2022-04-06 21:29:46 -07:00
..
access pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
bootstrap pg_upgrade: Preserve relfilenodes and tablespace OIDs. 2022-01-17 13:40:27 -05:00
catalog Reorder subskiplsn in pg_subscription to avoid alignment issues. 2022-04-07 09:39:25 +05:30
commands pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
executor Refactor and cleanup runtime partition prune code a little 2022-04-05 11:46:48 +02:00
foreign Create routine able to set single-call SRFs for Materialize mode 2022-03-07 10:26:29 +09:00
jit SQL/JSON query functions 2022-03-29 16:57:13 -04:00
lib dshash: revise sequential scan support. 2022-04-04 14:32:52 -07:00
libpq Add system view pg_ident_file_mappings 2022-03-29 10:15:48 +09:00
main Fix collection of typos in the code and the documentation 2022-03-15 11:29:35 +09:00
nodes Allow asynchronous execution in more cases. 2022-04-06 15:45:00 +09:00
optimizer Allow asynchronous execution in more cases. 2022-04-06 15:45:00 +09:00
parser Allow granting SET and ALTER SYSTEM privileges on GUC parameters. 2022-04-06 13:24:33 -04:00
partitioning Refactor and cleanup runtime partition prune code a little 2022-04-05 11:46:48 +02:00
po Translation updates 2021-06-21 12:33:50 +02:00
port Ensure that the argument of shmdt(2) is declared "void *". 2022-02-15 17:17:28 -05:00
postmaster pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
regex Call pg_newlocale_from_collation() also with default collation 2022-01-20 09:50:18 +01:00
replication pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
rewrite Add support for MERGE SQL command 2022-03-28 16:47:48 +02:00
snowball Update copyright for 2022 2022-01-07 19:04:57 -05:00
statistics Fix various typos, grammar and code style in comments and docs 2022-01-25 09:40:04 +09:00
storage pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
tcop pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
tsearch Update copyright for 2022 2022-01-07 19:04:57 -05:00
utils pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
.gitignore Add .gitignore entries for AIX-specific intermediate build artifacts. 2015-07-08 20:44:22 -04:00
common.mk Remove PARTIAL_LINKING build mode. 2018-03-30 17:33:04 -07:00
Makefile Server-side gzip compression. 2022-01-24 15:13:18 -05:00
nls.mk Report progress of startup operations that take a long time. 2021-10-25 11:51:57 -04:00