2021-04-03 04:45:24 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
* wait_event.h
|
|
|
|
|
* Definitions related to wait event reporting
|
|
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Copyright (c) 2001-2024, PostgreSQL Global Development Group
|
2021-04-03 04:45:24 +02:00
|
|
|
*
|
|
|
|
|
* src/include/utils/wait_event.h
|
|
|
|
|
* ----------
|
|
|
|
|
*/
|
|
|
|
|
#ifndef WAIT_EVENT_H
|
|
|
|
|
#define WAIT_EVENT_H
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* ----------
|
|
|
|
|
* Wait Classes
|
|
|
|
|
* ----------
|
|
|
|
|
*/
|
|
|
|
|
#define PG_WAIT_LWLOCK 0x01000000U
|
|
|
|
|
#define PG_WAIT_LOCK 0x03000000U
|
2023-07-03 04:01:02 +02:00
|
|
|
#define PG_WAIT_BUFFERPIN 0x04000000U
|
2021-04-03 04:45:24 +02:00
|
|
|
#define PG_WAIT_ACTIVITY 0x05000000U
|
|
|
|
|
#define PG_WAIT_CLIENT 0x06000000U
|
|
|
|
|
#define PG_WAIT_EXTENSION 0x07000000U
|
|
|
|
|
#define PG_WAIT_IPC 0x08000000U
|
|
|
|
|
#define PG_WAIT_TIMEOUT 0x09000000U
|
|
|
|
|
#define PG_WAIT_IO 0x0A000000U
|
|
|
|
|
|
Generate automatically code and documentation related to wait events
The documentation and the code is generated automatically from a new
file called wait_event_names.txt, formatted in sections dedicated to
each wait event class (Timeout, Lock, IO, etc.) with three tab-separated
fields:
- C symbol in enums
- Format in the system views
- Description in the docs
Using this approach has several advantages, as we have proved to be
rather bad in maintaining this area of the tree across the years:
- The order of each item in the documentation and the code, which should
be alphabetical, has become incorrect multiple times, and the script
generating the code and documentation has a few rules to enforce that,
making the maintenance a no-brainer.
- Some wait events were added to the code, but not documented, so this
cannot be missed now.
- The order of the tables for each wait event class is enforced in the
documentation (the input .txt file does so as well for clarity, though
this is not mandatory).
- Less code, shaving 1.2k lines from the tree, with 1/3 of the savings
coming from the code, the rest from the documentation.
The wait event types "Lock" and "LWLock" still have their own code path
for their code, hence only the documentation is created for them. These
classes are listed with a special marker called WAIT_EVENT_DOCONLY in
the input file.
Adding a new wait event now requires only an update of
wait_event_names.txt, with "Lock" and "LWLock" treated as exceptions.
This commit has been tested with configure/Makefile, the CI and VPATH
build. clean, distclean and maintainer-clean were working fine.
Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9b98@gmail.com
2023-07-05 03:53:11 +02:00
|
|
|
/* enums for wait events */
|
|
|
|
|
#include "utils/wait_event_types.h"
|
2021-04-03 04:45:24 +02:00
|
|
|
|
|
|
|
|
extern const char *pgstat_get_wait_event(uint32 wait_event_info);
|
|
|
|
|
extern const char *pgstat_get_wait_event_type(uint32 wait_event_info);
|
|
|
|
|
static inline void pgstat_report_wait_start(uint32 wait_event_info);
|
|
|
|
|
static inline void pgstat_report_wait_end(void);
|
Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
2021-04-03 20:44:47 +02:00
|
|
|
extern void pgstat_set_wait_event_storage(uint32 *wait_event_info);
|
|
|
|
|
extern void pgstat_reset_wait_event_storage(void);
|
2021-04-03 04:45:24 +02:00
|
|
|
|
Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
2021-04-03 20:44:47 +02:00
|
|
|
extern PGDLLIMPORT uint32 *my_wait_event_info;
|
2021-04-03 04:45:24 +02:00
|
|
|
|
|
|
|
|
|
2023-07-31 10:09:24 +02:00
|
|
|
/* ----------
|
|
|
|
|
* Wait Events - Extension
|
|
|
|
|
*
|
|
|
|
|
* Use this category when the server process is waiting for some condition
|
|
|
|
|
* defined by an extension module.
|
|
|
|
|
*
|
Change custom wait events to use dynamic shared hash tables
Currently, the names of the custom wait event must be registered for
each backend, requiring all these to link to the shared memory area of
an extension, even if these are not loaded with
shared_preload_libraries.
This patch relaxes the constraints related to this infrastructure by
storing the wait events and their names in two dynamic hash tables in
shared memory. This has the advantage to simplify the registration of
custom wait events to a single routine call that returns an event ID
ready for consumption:
uint32 WaitEventExtensionNew(const char *wait_event_name);
The caller of this routine can then cache locally the ID returned, to be
used for pgstat_report_wait_start(), WaitLatch() or a similar routine.
The implementation uses two hash tables: one with a key based on the
event name to avoid duplicates and a second using the event ID as key
for event lookups, like on pg_stat_activity. These tables can hold a
minimum of 16 entries, and a maximum of 128 entries, which should be plenty
enough.
The code changes done in worker_spi show how things are simplified (most
of the code removed in this commit comes from there):
- worker_spi_init() is gone.
- No more shared memory hooks required (size requested and
initialization).
- The custom wait event ID is cached in the process that needs to set
it, with one single call to WaitEventExtensionNew() to retrieve it.
Per suggestion from Andres Freund.
Author: Masahiro Ikeda, with a few tweaks from me.
Discussion: https://postgr.es/m/20230801032349.aaiuvhtrcvvcwzcx@awork3.anarazel.de
2023-08-14 07:47:27 +02:00
|
|
|
* Extensions can define their own wait events in this category. They should
|
|
|
|
|
* call WaitEventExtensionNew() with a wait event string. If the wait event
|
|
|
|
|
* associated to a string is already allocated, it returns the wait event
|
|
|
|
|
* information to use. If not, it gets one wait event ID allocated from
|
|
|
|
|
* a shared counter, associates the string to the ID in the shared dynamic
|
|
|
|
|
* hash and returns the wait event information.
|
|
|
|
|
*
|
|
|
|
|
* The ID retrieved can be used with pgstat_report_wait_start() or equivalent.
|
2023-07-31 10:09:24 +02:00
|
|
|
*/
|
|
|
|
|
typedef enum
|
|
|
|
|
{
|
|
|
|
|
WAIT_EVENT_EXTENSION = PG_WAIT_EXTENSION,
|
|
|
|
|
WAIT_EVENT_EXTENSION_FIRST_USER_DEFINED,
|
|
|
|
|
} WaitEventExtension;
|
|
|
|
|
|
|
|
|
|
extern void WaitEventExtensionShmemInit(void);
|
|
|
|
|
extern Size WaitEventExtensionShmemSize(void);
|
|
|
|
|
|
Change custom wait events to use dynamic shared hash tables
Currently, the names of the custom wait event must be registered for
each backend, requiring all these to link to the shared memory area of
an extension, even if these are not loaded with
shared_preload_libraries.
This patch relaxes the constraints related to this infrastructure by
storing the wait events and their names in two dynamic hash tables in
shared memory. This has the advantage to simplify the registration of
custom wait events to a single routine call that returns an event ID
ready for consumption:
uint32 WaitEventExtensionNew(const char *wait_event_name);
The caller of this routine can then cache locally the ID returned, to be
used for pgstat_report_wait_start(), WaitLatch() or a similar routine.
The implementation uses two hash tables: one with a key based on the
event name to avoid duplicates and a second using the event ID as key
for event lookups, like on pg_stat_activity. These tables can hold a
minimum of 16 entries, and a maximum of 128 entries, which should be plenty
enough.
The code changes done in worker_spi show how things are simplified (most
of the code removed in this commit comes from there):
- worker_spi_init() is gone.
- No more shared memory hooks required (size requested and
initialization).
- The custom wait event ID is cached in the process that needs to set
it, with one single call to WaitEventExtensionNew() to retrieve it.
Per suggestion from Andres Freund.
Author: Masahiro Ikeda, with a few tweaks from me.
Discussion: https://postgr.es/m/20230801032349.aaiuvhtrcvvcwzcx@awork3.anarazel.de
2023-08-14 07:47:27 +02:00
|
|
|
extern uint32 WaitEventExtensionNew(const char *wait_event_name);
|
Add system view pg_wait_events
This new view, wrapped around a SRF, shows some information known about
wait events, as of:
- Name.
- Type (Activity, I/O, Extension, etc.).
- Description.
All the information retrieved comes from wait_event_names.txt, and the
description is the same as the documentation with filters applied to
remove any XML markups. This view is useful when joined with
pg_stat_activity to get the description of a wait event reported.
Custom wait events for extensions are included in the view.
Original idea by Yves Colin.
Author: Bertrand Drouvot
Reviewed-by: Kyotaro Horiguchi, Masahiro Ikeda, Tom Lane, Michael
Paquier
Discussion: https://postgr.es/m/0e2ae164-dc89-03c3-cf7f-de86378053ac@gmail.com
2023-08-20 08:35:02 +02:00
|
|
|
extern char **GetWaitEventExtensionNames(int *nwaitevents);
|
2023-07-31 10:09:24 +02:00
|
|
|
|
2021-04-03 04:45:24 +02:00
|
|
|
/* ----------
|
|
|
|
|
* pgstat_report_wait_start() -
|
|
|
|
|
*
|
|
|
|
|
* Called from places where server process needs to wait. This is called
|
|
|
|
|
* to report wait event information. The wait information is stored
|
|
|
|
|
* as 4-bytes where first byte represents the wait event class (type of
|
|
|
|
|
* wait, for different types of wait, refer WaitClass) and the next
|
|
|
|
|
* 3-bytes represent the actual wait event. Currently 2-bytes are used
|
|
|
|
|
* for wait event which is sufficient for current usage, 1-byte is
|
|
|
|
|
* reserved for future usage.
|
|
|
|
|
*
|
Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
2021-04-03 20:44:47 +02:00
|
|
|
* Historically we used to make this reporting conditional on
|
|
|
|
|
* pgstat_track_activities, but the check for that seems to add more cost
|
|
|
|
|
* than it saves.
|
|
|
|
|
*
|
|
|
|
|
* my_wait_event_info initially points to local memory, making it safe to
|
|
|
|
|
* call this before MyProc has been initialized.
|
2021-04-03 04:45:24 +02:00
|
|
|
* ----------
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
pgstat_report_wait_start(uint32 wait_event_info)
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* Since this is a four-byte field which is always read and written as
|
|
|
|
|
* four-bytes, updates are atomic.
|
|
|
|
|
*/
|
Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
2021-04-03 20:44:47 +02:00
|
|
|
*(volatile uint32 *) my_wait_event_info = wait_event_info;
|
2021-04-03 04:45:24 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* ----------
|
|
|
|
|
* pgstat_report_wait_end() -
|
|
|
|
|
*
|
|
|
|
|
* Called to report end of a wait.
|
|
|
|
|
* ----------
|
|
|
|
|
*/
|
|
|
|
|
static inline void
|
|
|
|
|
pgstat_report_wait_end(void)
|
|
|
|
|
{
|
Improve efficiency of wait event reporting, remove proc.h dependency.
pgstat_report_wait_start() and pgstat_report_wait_end() required two
conditional branches so far. One to check if MyProc is NULL, the other to
check if pgstat_track_activities is set. As wait events are used around
comparatively lightweight operations, and are inlined (reducing branch
predictor effectiveness), that's not great.
The dependency on MyProc has a second disadvantage: Low-level subsystems, like
storage/file/fd.c, report wait events, but architecturally it is preferable
for them to not depend on inter-process subsystems like proc.h (defining
PGPROC). After this change including pgstat.h (nor obviously its
sub-components like backend_status.h, wait_event.h, ...) does not pull in IPC
related headers anymore.
These goals, efficiency and abstraction, are achieved by having
pgstat_report_wait_start/end() not interact with MyProc, but instead a new
my_wait_event_info variable. At backend startup it points to a local variable,
removing the need to check for MyProc being NULL. During process
initialization my_wait_event_info is redirected to MyProc->wait_event_info. At
shutdown this is reversed. Because wait event reporting now does not need to
know about where the wait event is stored, it does not need to know about
PGPROC anymore.
The removal of the branch for checking pgstat_track_activities is simpler:
Don't check anymore. The cost due to the branch are often higher than the
store - and even if not, pgstat_track_activities is rarely disabled.
The main motivator to commit this work now is that removing the (indirect)
pgproc.h include from pgstat.h simplifies a patch to move statistics reporting
to shared memory (which still has a chance to get into 14).
Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20210402194458.2vu324hkk2djq6ce@alap3.anarazel.de
2021-04-03 20:44:47 +02:00
|
|
|
/* see pgstat_report_wait_start() */
|
|
|
|
|
*(volatile uint32 *) my_wait_event_info = 0;
|
2021-04-03 04:45:24 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
#endif /* WAIT_EVENT_H */
|
