Commit Graph

12552 Commits

Author SHA1 Message Date
Dean Rasheed d0460a31de Add missing support for the latest SPI status codes.
SPI_result_code_string() was missing support for SPI_OK_TD_REGISTER,
and in v15 and later, it was missing support for SPI_OK_MERGE, as was
pltcl_process_SPI_result().

The last of those would trigger an error if a MERGE was executed from
PL/Tcl. The others seem fairly innocuous, but worth fixing.

Back-patch to all supported branches. Before v15, this is just adding
SPI_OK_TD_REGISTER to SPI_result_code_string(), which is unlikely to
be seen by anyone, but seems worth doing for completeness.

Reviewed by Tom Lane.

Discussion:
  https://postgr.es/m/CAEZATCUg8V%2BK%2BGcafOPqymxk84Y_prXgfe64PDoopjLFH6Z0Aw%40mail.gmail.com
  https://postgr.es/m/CAEZATCUMe%2B_KedPMM9AxKqm%3DSZogSxjUcrMe%2BsakusZh3BFcQw%40mail.gmail.com
2023-02-22 13:23:09 +00:00
Daniel Gustafsson e00bc6c922 doc: Add default value of createrole_self_grant
Document that the default value for createrole_self_grant is an empty
string which in turn disable the feature.

Author: Shi Yu <shiy.fnst@fujitsu.com>
Discussion: https://postgr.es/m/OSZPR01MB63105D0D96A9A72A7FCD4FFEFDA09@OSZPR01MB6310.jpnprd01.prod.outlook.com
2023-02-22 11:05:20 +01:00
Peter Eisentraut 2ddab010c2 Implement ANY_VALUE aggregate
SQL:2023 defines an ANY_VALUE aggregate whose purpose is to emit an
implementation-dependent (i.e. non-deterministic) value from the
aggregated rows.

Author: Vik Fearing <vik@postgresfriends.org>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5cff866c-10a8-d2df-32cb-e9072e6b04a2@postgresfriends.org
2023-02-22 09:33:07 +01:00
Michael Paquier 1a943d03d6 doc: Add missing values for meson options -Dcassert and -Db_coverage
The -Dcassert and -Db_coverage that can be specified to a meson command
require values after an equal sign but the documentation did not
properly reflect that.  All the other options specify the argument
values they expect, so close the gap.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQRXd1z+AoQ4tC5tqPk1_NQJohf6xwdEL=z9KgxHau2maQ@mail.gmail.com
2023-02-22 10:55:14 +09:00
Michael Paquier 35739b87dc Redesign archive modules
A new callback named startup_cb, called shortly after a module is
loaded, is added.  This makes possible the initialization of any
additional state data required by a module.  This initial state data can
be saved in a ArchiveModuleState, that is now passed down to all the
callbacks that can be defined in a module.  With this design, it is
possible to have a per-module state, aimed at opening the door to the
support of more than one archive module.

The initialization of the callbacks is changed so as
_PG_archive_module_init() does not anymore give in input a
ArchiveModuleCallbacks that a module has to fill in with callback
definitions.  Instead, a module now needs to return a const
ArchiveModuleCallbacks.

All the structure and callback definitions of archive modules are moved
into their own header, named archive_module.h, from pgarch.h.
Command-based archiving follows the same line, with a new set of files
named shell_archive.{c,h}.

There are a few more items that are under discussion to improve the
design of archive modules, like the fact that basic_archive calls
sigsetjmp() by itself to define its own error handling flow.  These will
be adjusted later, the changes done here cover already a good portion
of what has been discussed.

Any modules created for v15 will need to be adjusted to this new
design.

Author: Nathan Bossart
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20230130194810.6fztfgbn32e7qarj@awork3.anarazel.de
2023-02-17 14:26:42 +09:00
Amit Kapila fce003cfde Add a new wait state and use it when sending data in the apply worker.
d9d7fe68d3 made use of an existing wait event when sending data from the
apply worker, but we should have invented a new wait event since this is a
new place to wait.

This patch corrects the mistake by using a new wait event
"LogicalApplySendData".

Author: Hou Zhijie
Reviewed-by: Peter Smith
Discussion: https://postgr.es/m/CA+TgmobWzbr9H3yN3dLVckviEZKemPwd+XyCFKEgyZQZhgP66Q@mail.gmail.com
2023-02-16 07:46:31 +05:30
David Rowley 5352ca22e0 Rename force_parallel_mode to debug_parallel_query
force_parallel_mode is meant to be used to allow us to exercise the
parallel query infrastructure to ensure that it's working as we expect.
It seems some users think this GUC is for forcing the query planner into
picking a parallel plan regardless of the costs.  A quick look at the
documentation would have made them realize that they were wrong, but the
GUC is likely too conveniently named which, evidently, seems to often
result in users expecting that it forces the planner into usefully
parallelizing queries.

Here we rename the GUC to something which casual users are less likely to
mistakenly think is what they need to make their query run more quickly.

For now, the old name can still be used.  We'll revisit if the old name
mapping can be removed once the buildfarm configs are all updated.

Reviewed-by: John Naylor
Discussion: https://postgr.es/m/CAApHDvrsOi92_uA7PEaHZMH-S4Xv+MGhQWA+GrP8b1kjpS1HjQ@mail.gmail.com
2023-02-15 21:21:59 +13:00
Michael Paquier 8e0e0698f1 doc: Fix spelling of option -Dtap_tests for meson
This option was documented as -Dtap-tests, but the option requires an
underscore in its name.

Author: Katsuragi Yuta
Discussion: https://postgr.es/m/cff4558e170be78d8fd64ed45edbe1cb@oss.nttdata.com
2023-02-15 15:01:24 +09:00
Jeff Davis a8a44828a2 Correct docs for the default locale_provider of a new database.
If the locale provider is not specified, it defaults to be the same as
the template from which it was created. Previously, the documentation
said the default was libc.

Also adjust wording of CREATE DATABASE and CREATE COLLATION docs to be
definite that there are exactly two possible collation providers.

Discussion: https://postgr.es/m/6befdaada61c046b67f3b269f7fa6f069a35803e.camel%40j-davis.com
Reviewed-by: Nathan Bossart
2023-02-13 17:16:13 -08:00
Andres Freund a9c70b46db Add pg_stat_io view, providing more detailed IO statistics
Builds on 28e626bde0 and f30d62c2fc. See the former for motivation.

Rows of the view show IO operations for a particular backend type, IO target
object, IO context combination (e.g. a client backend's operations on
permanent relations in shared buffers) and each column in the view is the
total number of IO Operations done (e.g. writes). So a cell in the view would
be, for example, the number of blocks of relation data written from shared
buffers by client backends since the last stats reset.

In anticipation of tracking WAL IO and non-block-oriented IO (such as
temporary file IO), the "op_bytes" column specifies the unit of the "reads",
"writes", and "extends" columns for a given row.

Rows for combinations of IO operation, backend type, target object and context
that never occur, are ommitted entirely. For example, checkpointer will never
operate on temporary relations.

Similarly, if an IO operation never occurs for such a combination, the IO
operation's cell will be null, to distinguish from 0 observed IO
operations. For example, bgwriter should not perform reads.

Note that some of the cells in the view are redundant with fields in
pg_stat_bgwriter (e.g. buffers_backend). For now, these have been kept for
backwards compatibility.

Bumps catversion.

Author: Melanie Plageman <melanieplageman@gmail.com>
Author: Samay Sharma <smilingsamay@gmail.com>
Reviewed-by: Maciek Sakrejda <m.sakrejda@gmail.com>
Reviewed-by: Lukas Fittl <lukas@fittl.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20200124195226.lth52iydq2n2uilq@alap3.anarazel.de
2023-02-11 09:52:15 -08:00
Michael Paquier ef7002dbe0 Fix various typos in code and tests
Most of these are recent, and the documentation portions are new as of
v16 so there is no need for a backpatch.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20230208155644.GM1653@telsasoft.com
2023-02-09 14:43:53 +09:00
Andres Freund 28e626bde0 pgstat: Infrastructure for more detailed IO statistics
This commit adds the infrastructure for more detailed IO statistics. The calls
to actually count IOs, a system view to access the new statistics,
documentation and tests will be added in subsequent commits, to make review
easier.

While we already had some IO statistics, e.g. in pg_stat_bgwriter and
pg_stat_database, they did not provide sufficient detail to understand what
the main sources of IO are, or whether configuration changes could avoid
IO. E.g., pg_stat_bgwriter.buffers_backend does contain the number of buffers
written out by a backend, but as that includes extending relations (always
done by backends) and writes triggered by the use of buffer access strategies,
it cannot easily be used to tune background writer or checkpointer. Similarly,
pg_stat_database.blks_read cannot easily be used to tune shared_buffers /
compute a cache hit ratio, as the use of buffer access strategies will often
prevent a large fraction of the read blocks to end up in shared_buffers.

The new IO statistics count IO operations (evict, extend, fsync, read, reuse,
and write), and are aggregated for each combination of backend type (backend,
autovacuum worker, bgwriter, etc), target object of the IO (relations, temp
relations) and context of the IO (normal, vacuum, bulkread, bulkwrite).

What is tracked in this series of patches, is sufficient to perform the
aforementioned analyses. Further details, e.g. tracking the number of buffer
hits, would make that even easier, but was left out for now, to keep the scope
of the already large patchset manageable.

Bumps PGSTAT_FILE_FORMAT_ID.

Author: Melanie Plageman <melanieplageman@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20200124195226.lth52iydq2n2uilq@alap3.anarazel.de
2023-02-08 20:53:42 -08:00
Tom Lane 969509c3f2 Stop recommending auto-download of DTD files, and indeed disable it.
It appears no longer possible to build the SGML docs without a local
installation of the DocBook DTD, because sourceforge.net now only
permits HTTPS access, and no common version of xsltproc supports that.
Hence, remove the bits of our documentation suggesting that that's
possible or useful.

In fact, we might as well add the --nonet option to the build recipes
automatically, for a bit of extra security.

Also fix our documentation-tool-installation recipes for macOS to
ensure that xmllint and xsltproc are pulled in from MacPorts or
Homebrew.  The previous recipes assumed you could use the
Apple-supplied versions of these tools; which still works, except that
you'd need to set an environment variable to ensure that they would
find DTD files provided by those package managers.  Simpler and easier
to just recommend pulling in the additional packages.

In HEAD, also document how to build docs using Meson, and adjust
"ninja docs" to just build the HTML docs, for consistency with the
default behavior of doc/src/sgml/Makefile.

In a fit of neatnik-ism, I also made the ordering of the package
lists match the order in which the tools are described at the head
of the appendix.

Aleksander Alekseev, Peter Eisentraut, Tom Lane

Discussion: https://postgr.es/m/CAJ7c6TO8Aro2nxg=EQsVGiSDe-TstP4EsSvDHd7DSRsP40PgGA@mail.gmail.com
2023-02-08 17:15:25 -05:00
Peter Eisentraut aa69541046 Remove useless casts to (void *) in arguments of some system functions
The affected functions are: bsearch, memcmp, memcpy, memset, memmove,
qsort, repalloc

Reviewed-by: Corey Huinker <corey.huinker@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/fd9adf5d-b1aa-e82f-e4c7-263c30145807%40enterprisedb.com
2023-02-07 06:57:59 +01:00
Peter Eisentraut 7ba09efe24 doc: Fix SQL keywords lists
Some keywords were listed as both reserved and nonreserved.  Fix as
appropriate.
2023-02-06 17:54:10 +01:00
Peter Eisentraut 0ae4e49fa6 doc: Fix description of SQL standard parts
The splitting into parts actually started earlier than the text had
claimed, but that is ancient history anyway by now and does not need
to be mentioned.  Update that and tweak the text a bit.
2023-02-06 14:01:16 +01:00
Michael Paquier d07c2948bf Add support for progress reporting to pg_verifybackup
This adds a new option to pg_verifybackup called -P/--progress, showing
every second some information about the progress of the checksum
verification based on the data of a backup manifest.

Similarly to what is done for pg_rewind and pg_basebackup, the
information printed in the progress report consists of the current
amount of data computed and the total amount of data that will be
computed.  Note that files found with an incorrect size do not have
their checksum verified, hence their size is not appended to the total
amount of data estimated during the first scan of the manifest data
(such incorrect sizes could be overly high, for one, falsifying the
progress report).

Author: Masahiko Sawada
Discussion: https://postgr.es/m/CAD21AoC5+JOgMd4o3z_oxw0f8JDSsCYY7zSbhe-O9x7f33rw_A@mail.gmail.com
2023-02-06 14:40:31 +09:00
Dean Rasheed faff8f8e47 Allow underscores in integer and numeric constants.
This allows underscores to be used in integer and numeric literals,
and their corresponding type input functions, for visual grouping.
For example:

    1_500_000_000
    3.14159_26535_89793
    0xffff_ffff
    0b_1001_0001

A single underscore is allowed between any 2 digits, or immediately
after the base prefix indicator of non-decimal integers, per SQL:202x
draft.

Peter Eisentraut and Dean Rasheed

Discussion: https://postgr.es/m/84aae844-dc55-a4be-86d9-4f0fa405cc97%40enterprisedb.com
2023-02-04 09:48:51 +00:00
Peter Eisentraut 2010d8b75a doc: Fix XML formatting that psql cannot handle
Breaking <phrase> over two lines is not handled by psql's
create_help.pl.  (It creates faulty \help output.)

Undo the formatting change introduced by
9bdad1b515 to fix this for now.
2023-02-03 09:04:35 +01:00
Thomas Munro 117d2604c2 Doc: Abstract AF_UNIX sockets don't work on Windows.
An early release of AF_UNIX in Windows apparently supported Linux-style
"abstract" Unix sockets, but they do not seem to work in current Windows
versions and there is no mention of any of this in the Winsock
documentation.  Remove the mention of Windows from the documentation.

Back-patch to 14, where commit c9f0624b landed.

Discussion: https://postgr.es/m/CA%2BhUKGKrYbSZhrk4NGfoQGT_3LQS5pC5KNE1g0tvE_pPBZ7uew%40mail.gmail.com
2023-02-02 18:25:10 +13:00
Amit Kapila 9f2213a7c5 Allow the logical_replication_mode to be used on the subscriber.
Extend the existing developer option 'logical_replication_mode' to help
test the parallel apply of large transactions on the subscriber.

When set to 'buffered', the leader sends changes to parallel apply workers
via a shared memory queue. When set to 'immediate', the leader serializes
all changes to files and notifies the parallel apply workers to read and
apply them at the end of the transaction.

This helps in adding tests to cover the serialization code path in
parallel streaming mode.

Author: Hou Zhijie
Reviewed-by: Peter Smith, Kuroda Hayato, Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-02-02 08:15:18 +05:30
Tom Lane 856ca51a23 Doc: clarify use of NULL to drop comments and security labels.
This was only mentioned in the description of the text/label, which
are marked as being in quotes in the synopsis, which can cause
confusion (as witnessed on IRC).

Also separate the literal and NULL cases in the parameter list, per
suggestion from Tom Lane.

Also add an example of dropping a security label.

Dagfinn Ilmari Mannsåker, with some tweaks by me

Discussion: https://postgr.es/m/87sffqk4zp.fsf@wibble.ilmari.org
2023-01-31 14:32:24 -05:00
Tom Lane 2489d76c49 Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees.  This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.

To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle.  PlaceHolderVars
receive similar decoration for the same reason.

In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos.  This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.

This change affects FDWs that want to plan foreign joins.  They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.

Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup.  (The README additions
and comments mention some stuff that will appear in the follow-up.)

Patch by me; thanks to Richard Guo for review.

Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 13:16:20 -05:00
Tom Lane ec7e053a98 Doc: clarify behavior of boolean options in replication commands.
defGetBoolean() allows the "value" part of "option = value"
syntax to be omitted, in which case it's taken as "true".
This is acknowledged in our syntax summaries for relevant commands,
but we don't seem to have documented the actual behavior anywhere.
Do so for CREATE/ALTER PUBLICATION/SUBSCRIPTION.  Use generic
boilerplate text for this, with the idea that we can copy-and-paste
it into other relevant reference pages, whenever someone gets
around to that.

Peter Smith, edited a bit by me

Discussion: https://postgr.es/m/CAHut+PvwjZfdGt2R8HTXgSZft=jZKymrS8KUg31pS7zqaaWKKw@mail.gmail.com
2023-01-30 11:59:37 -05:00
Amit Kapila 1e8b61735c Rename GUC logical_decoding_mode to logical_replication_mode.
Rename the developer option 'logical_decoding_mode' to the more flexible
name 'logical_replication_mode' because doing so will make it easier to
extend this option in the future to help test other areas of logical
replication.

Currently, it is used on the publisher side to allow streaming or
serializing each change in logical decoding. In the upcoming patch, we are
planning to use it on the subscriber. On the subscriber, it will allow
serializing the changes to file and notifies the parallel apply workers to
read and apply them at the end of the transaction.

We discussed exposing this parameter as a subscription option but
it did not seem advisable since it is primarily used for testing/debugging
and there is no other such parameter. We also discussed having separate
GUCs for publisher and subscriber but for current testing/debugging
requirements, one GUC is sufficient.

Author: Hou Zhijie
Reviewed-by: Peter Smith, Kuroda Hayato, Sawada Masahiko, Amit Kapila
Discussion: https://postgr.es/m/CAD21AoAy2c=Mx=FTCs+EwUsf2kQL5MmU3N18X84k0EmCXntK4g@mail.gmail.com
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-30 08:02:08 +05:30
Peter Eisentraut 17e72ec45d doc: Adjust a few more references to "postmaster"
Reported-by: Karl O. Pinc <kop@karlpinc.com>
Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-27 08:42:08 +01:00
Tom Lane e35bb9f158 Doc: use less-awkward phrasing.
Improve wording in note about tools required to build
from the source repository.

Laurenz Albe, per gripe from Riivo Kolka

Discussion: https://postgr.es/m/167463493588.2667301.13267758265445155872@wrigleys.postgresql.org
2023-01-26 11:34:17 -05:00
Peter Eisentraut 37e2673350 Don't install postmaster symlink anymore
This has long been deprecated.  Some of the build systems didn't even
install it.

Also remove man page.

Reviewed-by: Karl O. Pinc <kop@karlpinc.com>
Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-26 11:33:01 +01:00
Peter Geoghegan 6c6b497266 Revert "Add eager and lazy freezing strategies to VACUUM."
This reverts commit 4d41799261.  Broad
concerns about regressions caused by eager freezing strategy have been
raised.  Whether or not these concerns can be worked through in any time
frame is far from certain.

Discussion: https://postgr.es/m/20230126004347.gepcmyenk2csxrri@awork3.anarazel.de
2023-01-25 22:22:27 -08:00
Jeff Davis 8b5f36bb6c Clarify documentation for CLUSTER on partitioned tables.
Author: Nathan Bossart
Discussion: https://postgr.es/m/20230114224000.GA2505377@nathanxps13
2023-01-25 20:23:32 -08:00
Michael Paquier 9aeff092c0 Revert "Rename contrib module basic_archive to basic_wal_module"
This reverts commit 0ad3c60, as per feedback from Tom Lane, Robert Haas
and Andres Freund.  The new name used for the module had little
support.

This moves back to basic_archive as module name, and we will likely use
that as template for recovery modules, as well.

Discussion: https://postgr.es/m/CA+TgmoYG5uGOp7DGFT5gzC1kKFWGjkLSj_wOQxGhfMcvVEiKGA@mail.gmail.com
2023-01-26 09:13:39 +09:00
Peter Geoghegan 1a8e72bff7 Doc: update VACUUM VERBOSE freezing tip.
VACUUM VERBOSE/autovacuuming logging have reported on the number of
pages frozen by VACUUM since commit d977ffd9 added that capability.
This information is directly related to relfrozenxid advancement, so
update an older tip from the documentation about how relfrozenxid is
reported on by the same instrumentation code.  Now the tip directly
mentions newly frozen pages, too.
2023-01-25 14:31:41 -08:00
Peter Geoghegan 4d41799261 Add eager and lazy freezing strategies to VACUUM.
Eager freezing strategy avoids large build-ups of all-visible pages.  It
makes VACUUM trigger page-level freezing whenever doing so will enable
the page to become all-frozen in the visibility map.  This is useful for
tables that experience continual growth, particularly strict append-only
tables such as pgbench's history table.  Eager freezing significantly
improves performance stability by spreading out the cost of freezing
over time, rather than doing most freezing during aggressive VACUUMs.
It complements the insert autovacuum mechanism added by commit b07642db.

VACUUM determines its freezing strategy based on the value of the new
vacuum_freeze_strategy_threshold GUC (or reloption) with logged tables.
Tables that exceed the size threshold use the eager freezing strategy.
Unlogged tables and temp tables always use eager freezing strategy,
since the added cost is negligible there.  Non-permanent relations won't
incur any extra overhead in WAL written (for the obvious reason), nor in
pages dirtied (since any extra freezing will only take place on pages
whose PD_ALL_VISIBLE bit needed to be set either way).

VACUUM uses lazy freezing strategy for logged tables that fall under the
GUC size threshold.  Page-level freezing triggers based on the criteria
established in commit 1de58df4, which added basic page-level freezing.

Eager freezing is strictly more aggressive than lazy freezing.  Settings
like vacuum_freeze_min_age still get applied in just the same way in
every VACUUM, independent of the strategy in use.  The only mechanical
difference between eager and lazy freezing strategies is that only the
former applies its own additional criteria to trigger freezing pages.
Note that even lazy freezing strategy will trigger freezing whenever a
page happens to have required that an FPI be written during pruning,
provided that the page will thereby become all-frozen in the visibility
map afterwards (due to the FPI optimization from commit 1de58df4).

The vacuum_freeze_strategy_threshold default setting is 4GB.  This is a
relatively low setting that prioritizes performance stability.  It will
be reviewed at the end of the Postgres 16 beta period.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Jeff Davis <pgsql@j-davis.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzkFok_6EAHuK39GaW4FjEFQsY=3J0AAd6FXk93u-Xq3Fg@mail.gmail.com
2023-01-25 14:15:38 -08:00
Michael Paquier d7c4830abb doc: Fix network_ops -> inet_ops in SpGiST operator class list
network_ops is an opclass family of SpGiST, and the opclass able to
work on the inet type is named inet_ops.

Oversight in 7a1cd52, that reworked the design of the table listing all
the operators available.

Reported-by: Laurence Parry
Reviewed-by: Tom Lane, David G. Johnston
Discussion: https://postgr.es/m/167458110639.2667300.14741268666497110766@wrigleys.postgresql.org
Backpatch-through: 14
2023-01-25 20:00:36 +09:00
Michael Paquier 0ad3c60caf Rename contrib module basic_archive to basic_wal_module
This rename is in preparation for the introduction of recovery modules,
where basic_wal_module will be used as a base template for the set of
callbacks introduced.  The former name did not really reflect all that.

Author: Nathan Bossart
Discussion: https://postgr.es/m/20221227192449.GA3672473@nathanxps13
2023-01-25 14:36:51 +09:00
Robert Haas f1358ca52d Adjust interaction of CREATEROLE with role properties.
Previously, a CREATEROLE user without SUPERUSER could not alter
REPLICATION users in any way, and could not set the BYPASSRLS
attribute. However, they could manipulate the CREATEDB property
even if they themselves did not possess it.

With this change, a CREATEROLE user without SUPERUSER can set or
clear the REPLICATION, BYPASSRLS, or CREATEDB property on a new
role or a role that they have rights to manage if and only if
that property is set for their own role.

This implements the standard idea that you can't give permissions
you don't have (but you can give the ones you do have). We might
in the future want to provide more powerful ways to constrain
what a CREATEROLE user can do - for example, to limit whether
CONNECTION LIMIT can be set or the values to which it can be set -
but that is left as future work.

Patch by me, reviewed by Nathan Bossart, Tushar Ahuja, and Neha
Sharma.

Discussion: http://postgr.es/m/CA+TgmobX=LHg_J5aT=0pi9gJy=JdtrUVGAu0zhr-i5v5nNbJDg@mail.gmail.com
2023-01-24 10:57:09 -05:00
Michael Paquier c31cf1c03d pg_walinspect: Add pg_get_wal_fpi_info()
This function is able to extract the full page images from a range of
records, specified as of input arguments start_lsn and end_lsn.  Like
the other functions of this module, an error is returned if using LSNs
that do not reflect real system values.  All the FPIs stored in a single
record are extracted.

The module's version is bumped to 1.1.

Author: Bharath Rupireddy
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/CALj2ACVCcvzd7WiWvD=6_7NBvVB_r6G0EGSxL4F8vosAi6Se4g@mail.gmail.com
2023-01-23 13:55:18 +09:00
David Rowley 16fd03e956 Allow parallel aggregate on string_agg and array_agg
This adds combine, serial and deserial functions for the array_agg() and
string_agg() aggregate functions, thus allowing these aggregates to
partake in partial aggregations.  This allows both parallel aggregation to
take place when these aggregates are present and also allows additional
partition-wise aggregation plan shapes to include plans that require
additional aggregation once the partially aggregated results from the
partitions have been combined.

Author: David Rowley
Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane
Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 17:35:01 +13:00
Tom Lane 5a3a95385b Track logrep apply workers' last start times to avoid useless waits.
Enforce wal_retrieve_retry_interval on a per-subscription basis,
rather than globally, and arrange to skip that delay in case of
an intentional worker exit.  This probably makes little difference
in the field, where apply workers wouldn't be restarted often;
but it has a significant impact on the runtime of our logical
replication regression tests (even though those tests use
artificially-small wal_retrieve_retry_interval settings already).

Nathan Bossart, with mostly-cosmetic editorialization by me

Discussion: https://postgr.es/m/20221122004119.GA132961@nathanxps13
2023-01-22 14:08:46 -05:00
Robert Haas 6e2775e4d4 Add new GUC reserved_connections.
This provides a way to reserve connection slots for non-superusers.
The slots reserved via the new GUC are available only to users who
have the new predefined role pg_use_reserved_connections.
superuser_reserved_connections remains as a final reserve in case
reserved_connections has been exhausted.

Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me.

Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20 15:39:13 -05:00
Robert Haas 6c1d5ba486 Update docs and error message for superuser_reserved_connections.
Commit ea92368cd1 made max_wal_senders
a separate pool of backends from max_connections, but the documentation
and error message for superuser_reserved_connections weren't updated
at the time, and as a result are somewhat misleading. Update.

This is arguably a back-patchable bug fix, but because it seems quite
minor, no back-patch.

Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me.

Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20 15:23:04 -05:00
Alvaro Herrera e86c8b728f
Describe each contrib module in its SGML section title
The original titles only had the module name, which is not very useful
when scanning the list.  By adding a very brief description to each
title, the table of contents becomes friendlier.

Also amend the introduction in the "additional modules" appendix, using
the word "Extension" more extensively.  Nowadays, almost all contrib
modules are extensions, so this is also helpful.

Author: Karl O. Pinc <kop@karlpinc.com>
Reviewed-by: Brar Piening <brar@gmx.de>
Discussion: https://postgr.es/m/20230102180015.372995a9@slate.karlpinc.com
2023-01-20 20:01:59 +01:00
Amit Kapila 7670add9f4 Improve the description of Output Plugin Callbacks.
We were inconsistently specifying the required and optional marking for
plugin callbacks. Additionally, this patch improves the description for
stream_prepare callback.

Author: Wang wei
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OS3PR01MB627553DAFD39ECDADD08DC909EFC9@OS3PR01MB6275.jpnprd01.prod.outlook.com
2023-01-20 08:12:19 +05:30
Michael Paquier efb6f4a4f9 Support the same patterns for pg-user in pg_ident.conf as in pg_hba.conf
While pg_hba.conf has support for non-literal username matches, and
this commit extends the capabilities that are supported for the
PostgreSQL user listed in an ident entry part of pg_ident.conf, with
support for:
1. The "all" keyword, where all the requested users are allowed.
2. Membership checks using the + prefix.
3. Using a regex to match against multiple roles.

1. is a feature that has been requested by Jelte Fennema, 2. something
that has been mentioned independently by Andrew Dunstan, and 3. is
something I came up with while discussing how to extend the first one,
whose implementation is facilitated by 8fea868.

This allows matching certain system users against many different
postgres users with a single line in pg_ident.conf.  Without this, one
would need one line for each of the postgres users that a system user
can log in as, which can be cumbersome to maintain.

Tests are added to the TAP test of peer authentication to provide
coverage for all that.

Note that this introduces a set of backward-incompatible changes to be
able to detect the new patterns, for the following cases:
- A role named "all".
- A role prefixed with '+' characters, which is something that would not
have worked in HBA entries anyway.
- A role prefixed by a slash character, similarly to 8fea868.
Any of these can be still be handled by using quotes in the Postgres
role defined in an ident entry.

A huge advantage of this change is that the code applies the same checks
for the Postgres roles in HBA and ident entries, via the common routine
check_role().

**This compatibility change should be mentioned in the release notes.**

Author: Jelte Fennema
Discussion: https://postgr.es/m/DBBPR83MB0507FEC2E8965012990A80D0F7FC9@DBBPR83MB0507.EURPRD83.prod.outlook.com
2023-01-20 11:21:55 +09:00
Michael Paquier 4f74f5641d doc: Fix some issues in logical replication section
wal_retrieve_retry_interval was mentioned under an incorrect name, and
wal_sender_timeout was not listed as affecting WAL senders in logical
replication mode.

Author: Takamichi Osumi
Discussion: https://postgr.es/m/TYCPR01MB8373D65E6B0A769ED12EADCBEDC79@TYCPR01MB8373.jpnprd01.prod.outlook.com
2023-01-19 14:00:23 +09:00
Amit Kapila d540a02a72 Display the leader apply worker's PID for parallel apply workers.
Add leader_pid to pg_stat_subscription. leader_pid is the process ID of
the leader apply worker if this process is a parallel apply worker. If
this field is NULL, it indicates that the process is a leader apply
worker or a synchronization worker. The new column makes it easier to
distinguish parallel apply workers from other kinds of workers and helps
to identify the leader for the parallel workers corresponding to a
particular subscription.

Additionally, update the leader_pid column in pg_stat_activity as well to
display the PID of the leader apply worker for parallel apply workers.

Author: Hou Zhijie
Reviewed-by: Peter Smith, Sawada Masahiko, Amit Kapila, Shveta Mallik
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-18 09:03:12 +05:30
Tom Lane 09d465c397 Doc: fix a few oddly-spelled SGML ID attributes.
Avoid use of "_" in SGML IDs.  Awhile back that was actually
disallowed by the toolchain, as a consequence of which our convention
has been to use "-" instead.  Fix a couple of stragglers that are
particularly inconsistent with that convention and with related IDs.

This is just neatnik-ism, so no need for back-patch.

Discussion: https://postgr.es/m/769446.1673478332@sss.pgh.pa.us
2023-01-17 17:13:20 -05:00
Michael Paquier 0b717432ff Track behavior of \1 in pg_ident.conf when quoted
Entries of pg-user in pg_ident.conf that are quoted and include '\1'
allow a replacement from a subexpression in a system user regexp.  This
commit adds a test to track this behavior and a note in the
documentation, as it could be affected by the use of an AuthToken for
the pg-user in the IdentLines parsed.

This subject has come up in the discussion aimed at extending the
support of pg-user in ident entries for more patterns.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQRNow4MwkBjgPxywXdJU_K3a9+Pm78JB7De3yQwwkTDew@mail.gmail.com
2023-01-17 13:41:09 +09:00
Robert Haas 6fa66ec88f Assorted improvements to SECURITY DEFINER functions documentation.
Add a cross-reference from the part of the page that introdues SECURITY
INVOKER and SECURITY DEFINER to the part of the page that talks about
writing SECURITY DEFINER functions safely, so that users are less likely
to miss it.

Remove discussion of the pre-8.3 behavior on the theory that it's
probably not very relevant any more, that release having gone out of
support nearly a decade ago.

Add a mention of the new createrole_self_grant GUC, which in
certain cases might need to be set to a safe value to avoid
unexpected consequences.

Possibly this section needs major surgery rather than just these
small tweaks, but hopefully this is at least a small step
forward.

Discussion: http://postgr.es/m/CA+Tgmoauqd1cHQjsNEoxL5O-kEO4iC9dAPyCudSvmNqPJGmy9g@mail.gmail.com
2023-01-16 10:49:59 -05:00
Robert Haas 3cdf7502f8 More documentation update for GRANT ... WITH SET OPTION.
Update the reference pages for various ALTER commands that
mentioned that you must be a member of role that will be the
new owner to instead say that you must be able to SET ROLE
to the new owner. Update ddl.sgml's generate statement on this
topic along similar lines.

Likewise, update CREATE SCHEMA and CREATE DATABASE, which
have options to specify who will own the new objects, to say
that you must be able to SET ROLE to the role that will own
them.

Finally, update the documentation for the GRANT statement
itself with some general principles about how the SET option
works and how it can be used.

Patch by me, reviewed (but not fully endorsed) by Noah Misch.

Discussion: http://postgr.es/m/CA+TgmoZk6VB3DQ83+DO5P_HP=M9PQAh1yj-KgeV30uKefVaWDg@mail.gmail.com
2023-01-16 10:35:29 -05:00