postgresql/src/backend/utils
Thomas Munro 5dc0418fab Prefetch data referenced by the WAL, take II.
Introduce a new GUC recovery_prefetch.  When enabled, look ahead in the
WAL and try to initiate asynchronous reading of referenced data blocks
that are not yet cached in our buffer pool.  For now, this is done with
posix_fadvise(), which has several caveats.  Since not all OSes have
that system call, "try" is provided so that it can be enabled where
available.  Better mechanisms for asynchronous I/O are possible in later
work.

Set to "try" for now for test coverage.  Default setting to be finalized
before release.

The GUC wal_decode_buffer_size limits the distance we can look ahead in
bytes of decoded data.

The existing GUC maintenance_io_concurrency is used to limit the number
of concurrent I/Os allowed, based on pessimistic heuristics used to
infer that I/Os have begun and completed.  We'll also not look more than
maintenance_io_concurrency * 4 block references ahead.

Reviewed-by: Julien Rouhaud <rjuju123@gmail.com>
Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com>
Reviewed-by: Alvaro Herrera <alvherre@2ndquadrant.com> (earlier version)
Reviewed-by: Andres Freund <andres@anarazel.de> (earlier version)
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> (earlier version)
Tested-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> (earlier version)
Tested-by: Jakub Wartak <Jakub.Wartak@tomtom.com> (earlier version)
Tested-by: Dmitry Dolgov <9erthalion6@gmail.com> (earlier version)
Tested-by: Sait Talha Nisanci <Sait.Nisanci@microsoft.com> (earlier version)
Discussion: https://postgr.es/m/CA%2BhUKGJ4VJN8ttxScUFM8dOKX0BrBiboo5uz1cq%3DAovOddfHpA%40mail.gmail.com
2022-04-07 19:42:14 +12:00
..
activity pgstat: add pg_stat_have_stats() test helper. 2022-04-07 00:21:54 -07:00
adt Prefetch data referenced by the WAL, take II. 2022-04-07 19:42:14 +12:00
cache pgstat: store statistics in shared memory. 2022-04-06 21:29:46 -07:00
error pgstat: stats collector references in comments. 2022-04-06 13:56:06 -07:00
fmgr Refactor DLSUFFIX handling 2022-03-25 08:56:02 +01:00
hash Update copyright for 2022 2022-01-07 19:04:57 -05:00
init Custom WAL Resource Managers. 2022-04-06 23:06:46 -07:00
mb Update copyright for 2022 2022-01-07 19:04:57 -05:00
misc Prefetch data referenced by the WAL, take II. 2022-04-07 19:42:14 +12:00
mmgr Improve the generation memory allocator 2022-04-04 20:53:13 +12:00
resowner Update copyright for 2022 2022-01-07 19:04:57 -05:00
sort Use Generation memory contexts to store tuples in sorts 2022-04-04 22:52:35 +12:00
time Clean up assorted failures under clang's -fsanitize=undefined checks. 2022-03-03 18:13:24 -05:00
.gitignore Rearrange makefile rules for running Gen_fmgrtab.pl. 2018-05-03 17:54:18 -04:00
errcodes.txt Update copyright for 2022 2022-01-07 19:04:57 -05:00
Gen_dummy_probes.pl Update copyright for 2022 2022-01-07 19:04:57 -05:00
Gen_dummy_probes.pl.prolog Update copyright for 2022 2022-01-07 19:04:57 -05:00
Gen_dummy_probes.sed Update copyright for 2022 2022-01-07 19:04:57 -05:00
Gen_fmgrtab.pl Update copyright for 2022 2022-01-07 19:04:57 -05:00
generate-errcodes.pl Update copyright for 2022 2022-01-07 19:04:57 -05:00
Makefile Correct Makefile dependencies for catalog scripts 2022-02-14 09:07:09 +07:00
probes.d Update copyright for 2022 2022-01-07 19:04:57 -05:00
README.Gen_dummy_probes Tweak generation of Gen_dummy_probes.pl 2021-05-11 20:02:02 -04:00

# Generating dummy probes

If Postgres isn't configured with dtrace enabled, we need to generate
dummy probes for the entries in probes.d, that do nothing.

This is accomplished in Unix via the sed script `Gen_dummy_probes.sed`. We
used to use this in MSVC builds using the perl utility `psed`, which mimicked
sed. However, that utility disappeared from Windows perl distributions and so
we converted the sed script to a perl script to be used in MSVC builds.

We still keep the sed script as the authoritative source for generating
these dummy probes because except on Windows perl is not a hard requirement
when building from a tarball.

So, if you need to change the way dummy probes are generated, first change
the sed script, and when it's working generate the perl script. This can
be accomplished by using the perl utility s2p.

s2p is no longer part of the perl core, so it might not be on your system,
but it is available on CPAN and also in many package systems. e.g.
on Fedora it can be installed using `cpan App::s2p` or
`dnf install perl-App-s2p`.

The Makefile contains a recipe for regenerating Gen_dummy_probes.pl, so all
you need to do is once you have s2p installed is `make Gen_dummy_probes.pl`
Note that in a VPATH build this will generate the file in the vpath tree,
not the source tree.