Commit Graph

16682 Commits

Author SHA1 Message Date
Bruce Momjian
320a27aab6 doc: remove mention of backslash doubling in strings
Reported-by: Laurenz Albe

Discussion: https://postgr.es/m/0b03f91a875fb44182f5bed9e1d404ed6d138066.camel@cybertec.at

Author: Laurenz Albe

Backpatch-through: 11
2023-09-08 17:25:15 -04:00
Daniel Gustafsson
5a3423ad8e Add JIT deform_counter
generation_counter includes time spent on both JIT:ing expressions
and tuple deforming which are configured independently via options
jit_expressions and jit_tuple_deforming.  As they are  combined in
the same counter it's not apparent what fraction of time the tuple
deforming takes.

This adds deform_counter dedicated to tuple deforming, which allows
seeing more directly the influence jit_tuple_deforming is having on
the query. The counter is exposed in EXPLAIN and pg_stat_statements
bumpin pg_stat_statements to 1.11.

Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/20220612091253.eegstkufdsu4kfls@erthalion.local
2023-09-08 15:05:12 +02:00
Daniel Gustafsson
6fe3cefde4 doc: Extend documentation of PG_TEST_EXTRA
Extend the PG_TEST_EXTRA documentation to mention resource intensive
tests as well. The previous wording only mentioned special software
and security in the main paragraph, with resource usage listed on one
of the tests in the list.

Backpatch to v15 where f47ed79cc8 added wal_consistenct_checking as
a PG_TEST_EXTRA target.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/CAN55FZ0OthTuBdiNkaX2BvxuHdK4Y1MVEb8_uEuD1yHMPmT9Og@mail.gmail.com
Backpatch-through: 15
2023-09-08 11:34:43 +02:00
Bruce Momjian
8438a3ffef doc: change "cross product" to "Cartesian product"
Reported-by: Erik Wienhold

Discussion: https://postgr.es/m/2053109544.160895.1684955437950@office.mailbox.org

Author: Erik Wienhold

Backpatch-through: master
2023-09-07 15:48:18 -04:00
Nathan Bossart
8c16ad3b43 Allow using syncfs() in frontend utilities.
This commit allows specifying a --sync-method in several frontend
utilities that must synchronize many files to disk (initdb,
pg_basebackup, pg_checksums, pg_dump, pg_rewind, and pg_upgrade).
On Linux, users can specify "syncfs" to synchronize the relevant
file systems instead of calling fsync() for every single file.  In
many cases, using syncfs() is much faster.

As with recovery_init_sync_method, this new option comes with some
caveats.  The descriptions of these caveats have been moved to a
new appendix section in the documentation.

Co-authored-by: Justin Pryzby
Reviewed-by: Michael Paquier, Thomas Munro, Robert Haas, Justin Pryzby
Discussion: https://postgr.es/m/20210930004340.GM831%40telsasoft.com
2023-09-06 16:27:16 -07:00
Bruce Momjian
059e4ee921 doc: mention that to_char() values are rounded
Reported-by: barsikdacat@gmail.com

Diagnosed-by: Laurenz Albe

Discussion: https://postgr.es/m/168991536429.626.9957835774751337210@wrigleys.postgresql.org

Author: Laurenz Albe

Backpatch-through: 11
2023-09-06 16:52:24 -04:00
Bruce Momjian
a899d07b24 doc: mention libpq regression tests
Reported-by: Ryo Matsumura

Discussion: https://postgr.es/m/TYCPR01MB11316B3FB56EE54D70BF0CEF6E8E4A@TYCPR01MB11316.jpnprd01.prod.outlook.com

Backpatch-through: 11
2023-09-05 13:05:28 -04:00
Thomas Munro
f691f5b80a Remove the "snapshot too old" feature.
Remove the old_snapshot_threshold setting and mechanism for producing
the error "snapshot too old", originally added by commit 848ef42b.
Unfortunately it had a number of known problems in terms of correctness
and performance, mostly reported by Andres in the course of his work on
snapshot scalability.  We agreed to remove it, after a long period
without an active plan to fix it.

This is certainly a desirable feature, and someone might propose a new
or improved implementation in the future.

Reported-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CACG%3DezYV%2BEvO135fLRdVn-ZusfVsTY6cH1OZqWtezuEYH6ciQA%40mail.gmail.com
Discussion: https://postgr.es/m/20200401064008.qob7bfnnbu4w5cw4%40alap3.anarazel.de
Discussion: https://postgr.es/m/CA%2BTgmoY%3Daqf0zjTD%2B3dUWYkgMiNDegDLFjo%2B6ze%3DWtpik%2B3XqA%40mail.gmail.com
2023-09-05 19:53:43 +12:00
Etsuro Fujita
7ef5f5fb32 Doc: Improve documentation for creating custom scan paths.
This improves the documentation related to parallel query a little bit.

Reviewed by Richard Guo.

Discussion: https://postgr.es/m/CAPmGK159eJShAR4ek4Db8oHD4%2BZ18zwJkisEWkqRN-80BdDGFQ%40mail.gmail.com
2023-08-30 17:45:00 +09:00
Tatsuo Ishii
3c662643c4 Allow pgbench to exit immediately when any client is aborted.
Previously when client was aborted due to some error during
benchmarking, other clients continued their run until certain number
of transactions specified -t was reached or the time specified by -T
was expired. At the end, the results are printed with caution: "Run
was aborted; the above results are incomplete" shows.

New option "--exit-on-abort" allows pgbench to exit immediately in
this case so that users could quickly fix the cause of the failure and
try again another round of benchmarking.

Author: Yugo Nagata
Reviewed-by: Fabien COELHO, Tatsuo Ishii
Discussion: https://postgr.es/m/flat/20230804130325.df32e60879c38c92bca64207%40sraoss.co.jp
2023-08-30 10:03:31 +09:00
Peter Eisentraut
63956bed7b Rename logical_replication_mode to debug_logical_replication_streaming
The logical_replication_mode GUC is intended for testing and debugging
purposes, but its current name may be misleading and encourage users to make
unnecessary changes.

To avoid confusion, renaming the GUC to a less misleading name
debug_logical_replication_streaming that casual users are less likely to mistakenly
assume needs to be modified in a regular logical replication setup.

Author: Hou Zhijie <houzj.fnst@cn.fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/d672d774-c44b-6fec-f993-793e744f169a%40eisentraut.org
2023-08-29 15:19:56 +02:00
Daniel Gustafsson
f347ec76e2 Allow \watch queries to stop on minimum rows returned
When running a repeat query with \watch in psql, it can be
helpful to be able to stop the watch process when the query
no longer returns the expected amount of rows.  An example
would be to watch for the presence of a certain event in
pg_stat_activity and stopping when the event is no longer
present, or to watch an index creation and stop when the
index is created.

This adds a min_rows=MIN parameter to \watch which can be
set to a non-negative integer, and the watch query will
stop executing when it returns less than MIN rows.

Author: Greg Sabino Mullane <htamfids@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/CAKAnmmKStATuddYxP71L+p0DHtp9Rvjze3XRoy0Dyw67VQ45UA@mail.gmail.com
2023-08-29 11:30:11 +02:00
Alvaro Herrera
b0e96f3119
Catalog not-null constraints
We now create contype='n' pg_constraint rows for not-null constraints.

We propagate these constraints to other tables during operations such as
adding inheritance relationships, creating and attaching partitions and
creating tables LIKE other tables.  We also spawn not-null constraints
for inheritance child tables when their parents have primary keys.
These related constraints mostly follow the well-known rules of
conislocal and coninhcount that we have for CHECK constraints, with some
adaptations: for example, as opposed to CHECK constraints, we don't
match not-null ones by name when descending a hierarchy to alter it,
instead matching by column name that they apply to.  This means we don't
require the constraint names to be identical across a hierarchy.

For now, we omit them for system catalogs.  Maybe this is worth
reconsidering.  We don't support NOT VALID nor DEFERRABLE clauses
either; these can be added as separate features later (this patch is
already large and complicated enough.)

psql shows these constraints in \d+.

pg_dump requires some ad-hoc hacks, particularly when dumping a primary
key.  We now create one "throwaway" not-null constraint for each column
in the PK together with the CREATE TABLE command, and once the PK is
created, all those throwaway constraints are removed.  This avoids
having to check each tuple for nullness when the dump restores the
primary key creation.

pg_upgrading from an older release requires a somewhat brittle procedure
to create a constraint state that matches what would be created if the
database were being created fresh in Postgres 17.  I have tested all the
scenarios I could think of, and it works correctly as far as I can tell,
but I could have neglected weird cases.

This patch has been very long in the making.  The first patch was
written by Bernd Helmle in 2010 to add a new pg_constraint.contype value
('n'), which I (Álvaro) then hijacked in 2011 and 2012, until that one
was killed by the realization that we ought to use contype='c' instead:
manufactured CHECK constraints.  However, later SQL standard
development, as well as nonobvious emergent properties of that design
(mostly, failure to distinguish them from "normal" CHECK constraints as
well as the performance implication of having to test the CHECK
expression) led us to reconsider this choice, so now the current
implementation uses contype='n' again.  During Postgres 16 this had
already been introduced by commit e056c557ae, but there were some
problems mainly with the pg_upgrade procedure that couldn't be fixed in
reasonable time, so it was reverted.

In 2016 Vitaly Burovoy also worked on this feature[1] but found no
consensus for his proposed approach, which was claimed to be closer to
the letter of the standard, requiring an additional pg_attribute column
to track the OID of the not-null constraint for that column.
[1] https://postgr.es/m/CAKOSWNkN6HSyatuys8xZxzRCR-KL1OkHS5-b9qd9bf1Rad3PLA@mail.gmail.com

Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Author: Bernd Helmle <mailings@oopsware.de>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Dean Rasheed <dean.a.rasheed@gmail.com>
2023-08-25 13:31:24 +02:00
Nathan Bossart
260a1f18da Add to_bin() and to_oct().
This commit introduces functions for converting numbers to their
equivalent binary and octal representations.  Also, the base
conversion code for these functions and to_hex() has been moved to
a common helper function.

Co-authored-by: Eric Radman
Reviewed-by: Ian Barwick, Dag Lem, Vignesh C, Tom Lane, Peter Eisentraut, Kirk Wolak, Vik Fearing, John Naylor, Dean Rasheed
Discussion: https://postgr.es/m/Y6IyTQQ/TsD5wnsH%40vm3.eradman.com
2023-08-23 07:49:03 -07:00
Daniel Gustafsson
ee99330a0b doc: Replace list of drivers and PLs with wiki link
The list of external language drivers and procedural languages was
never complete or exhaustive, and rather than attempting to manage
it the content has migrated to the wiki.  This replaces the tables
altogether with links to the wiki as we regularly get requests for
adding various projects,  which we reject without any clear policy
for why or how the content should be managed.

The threads linked to below are the most recent discussions about
this, the archives contain many more.

Backpatch to all supported branches since the list on the wiki
applies to all branches.

Author: Jonathan Katz <jkatz@postgresql.org>
Discussion: https://postgr.es/m/169165415312.635.10247434927885764880@wrigleys.postgresql.org
Discussion: https://postgr.es/m/169177958824.635.11087800083040275266@wrigleys.postgresql.org
Backpatch-through: v11
2023-08-23 14:13:07 +02:00
Peter Eisentraut
17ec2c5dfa doc: Add more ICU rules examples
In particular, add an example EBCDIC collation.

Author: Daniel Verite <daniel@manitou-mail.org>
Discussion: https://www.postgresql.org/message-id/flat/35cc1684-e516-4a01-a256-351632d47066@manitou-mail.org
2023-08-23 11:23:42 +02:00
Peter Eisentraut
ed057fb687 doc: Improve ICU external link
It previously pointed to the collation API documentation, which our
users don't need, but the containing chapter seems useful.
2023-08-23 08:25:56 +02:00
Peter Eisentraut
ed9330cff5 Improve vertical spacing of documentation markup 2023-08-23 08:12:50 +02:00
Michael Paquier
1e68e43d3f Add system view pg_wait_events
This new view, wrapped around a SRF, shows some information known about
wait events, as of:
- Name.
- Type (Activity, I/O, Extension, etc.).
- Description.

All the information retrieved comes from wait_event_names.txt, and the
description is the same as the documentation with filters applied to
remove any XML markups.  This view is useful when joined with
pg_stat_activity to get the description of a wait event reported.

Custom wait events for extensions are included in the view.

Original idea by Yves Colin.

Author: Bertrand Drouvot
Reviewed-by: Kyotaro Horiguchi, Masahiro Ikeda, Tom Lane, Michael
Paquier
Discussion: https://postgr.es/m/0e2ae164-dc89-03c3-cf7f-de86378053ac@gmail.com
2023-08-20 15:35:02 +09:00
Peter Eisentraut
78806a9509 Remove incorrect field from information schema
The source code comment already said that the presence of the field
element_types.domain_default might be a bug in the standard, since it
never made sense there.  Indeed, the field is gone in newer versions
of the standard.  So just remove it.
2023-08-16 13:46:26 +02:00
Etsuro Fujita
9e9931d2bf Re-allow FDWs and custom scan providers to replace joins with pseudoconstant quals.
This was disabled in commit 6f80a8d9c due to the lack of support for
handling of pseudoconstant quals assigned to replaced joins in
createplan.c.  To re-allow it, this patch adds the support by 1)
modifying the ForeignPath and CustomPath structs so that if they
represent foreign and custom scans replacing a join with a scan, they
store the list of RestrictInfo nodes to apply to the join, as in
JoinPaths, and by 2) modifying create_scan_plan() in createplan.c so
that it uses that list in that case, instead of the baserestrictinfo
list, to get pseudoconstant quals assigned to the join, as mentioned in
the commit message for that commit.

Important item for the release notes: this is non-backwards-compatible
since it modifies the ForeignPath and CustomPath structs, as mentioned
above, and changes the argument lists for FDW helper functions
create_foreignscan_path(), create_foreign_join_path(), and
create_foreign_upper_path().

Richard Guo, with some additional changes by me, reviewed by Nishant
Sharma, Suraj Kharage, and Richard Guo.

Discussion: https://postgr.es/m/CADrsxdbcN1vejBaf8a%2BQhrZY5PXL-04mCd4GDu6qm6FigDZd6Q%40mail.gmail.com
2023-08-15 16:45:00 +09:00
Michael Paquier
af720b4c50 Change custom wait events to use dynamic shared hash tables
Currently, the names of the custom wait event must be registered for
each backend, requiring all these to link to the shared memory area of
an extension, even if these are not loaded with
shared_preload_libraries.

This patch relaxes the constraints related to this infrastructure by
storing the wait events and their names in two dynamic hash tables in
shared memory.  This has the advantage to simplify the registration of
custom wait events to a single routine call that returns an event ID
ready for consumption:
uint32 WaitEventExtensionNew(const char *wait_event_name);

The caller of this routine can then cache locally the ID returned, to be
used for pgstat_report_wait_start(), WaitLatch() or a similar routine.

The implementation uses two hash tables: one with a key based on the
event name to avoid duplicates and a second using the event ID as key
for event lookups, like on pg_stat_activity.  These tables can hold a
minimum of 16 entries, and a maximum of 128 entries, which should be plenty
enough.

The code changes done in worker_spi show how things are simplified (most
of the code removed in this commit comes from there):
- worker_spi_init() is gone.
- No more shared memory hooks required (size requested and
initialization).
- The custom wait event ID is cached in the process that needs to set
it, with one single call to WaitEventExtensionNew() to retrieve it.

Per suggestion from Andres Freund.

Author: Masahiro Ikeda, with a few tweaks from me.
Discussion: https://postgr.es/m/20230801032349.aaiuvhtrcvvcwzcx@awork3.anarazel.de
2023-08-14 14:47:27 +09:00
Etsuro Fujita
74a2dfee22 Doc: update documentation for creating custom scan paths.
Commit f49842d1e added a new callback for custom scan paths, but missed
updating the documentation.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/CAPmGK15ODkN%2B%3DhkBCufj1HBW0x5OTb65Xuy7ryXchMdiCMpx_g%40mail.gmail.com
2023-08-03 17:45:00 +09:00
Michael Paquier
c9af054653 Support custom wait events for wait event type "Extension"
Two backend routines are added to allow extension to allocate and define
custom wait events, all of these being allocated in the type
"Extension":
* WaitEventExtensionNew(), that allocates a wait event ID computed from
a counter in shared memory.
* WaitEventExtensionRegisterName(), to associate a custom string to the
wait event ID allocated.

Note that this includes an example of how to use this new facility in
worker_spi with tests in TAP for various scenarios, and some
documentation about how to use them.

Any code in the tree that currently uses WAIT_EVENT_EXTENSION could
switch to this new facility to define custom wait events.  This is left
as work for future patches.

Author: Masahiro Ikeda
Reviewed-by: Andres Freund, Michael Paquier, Tristan Partin, Bharath
Rupireddy
Discussion: https://postgr.es/m/b9f5411acda0cf15c8fbb767702ff43e@oss.nttdata.com
2023-07-31 17:09:24 +09:00
Nathan Bossart
6d982e3b3b Harmonize password reuse in vacuumdb, clusterdb, and reindexdb.
Commits 83dec5a712 and ff402ae11b taught vacuumdb to reuse
passwords instead of prompting repeatedly.  However, the docs still
warn about repeated prompts, and this improvement was not applied
to clusterdb and reindexdb.  This commit allows clusterdb and
reindexdb to reuse passwords just like vacuumdb does, and it
expunges the aforementioned warnings from the docs.

Reviewed-by: Gurjeet Singh, Zhang Mingli
Discussion: https://postgr.es/m/20230628045741.GA1813397%40nathanxps13
2023-07-28 10:07:44 -07:00
Amit Langote
e055b6be7e doc: add missing <returnvalue> and whitespace
Missed in commit 03734a7fed.

Author: Shinoda, Noriyoshi <noriyoshi.shinoda@hpe.com>
Discussion: https://postgr.es/m/DM4PR84MB1734E58BB4DC0E1B6E2990EBEE01A%40DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
2023-07-28 16:05:44 +09:00
Amit Langote
03734a7fed Add more SQL/JSON constructor functions
This Patch introduces three SQL standard JSON functions:

JSON()
JSON_SCALAR()
JSON_SERIALIZE()

JSON() produces json values from text, bytea, json or jsonb values,
and has facilitites for handling duplicate keys.

JSON_SCALAR() produces a json value from any scalar sql value,
including json and jsonb.

JSON_SERIALIZE() produces text or bytea from input which containis
or represents json or jsonb;

For the most part these functions don't add any significant new
capabilities, but they will be of use to users wanting standard
compliant JSON handling.

Catversion bumped as this changes ruleutils.c.

Author: Nikita Glukhov <n.gluhov@postgrespro.ru>
Author: Teodor Sigaev <teodor@sigaev.ru>
Author: Oleg Bartunov <obartunov@gmail.com>
Author: Alexander Korotkov <aekorotkov@gmail.com>
Author: Andrew Dunstan <andrew@dunslane.net>
Author: Amit Langote <amitlangote09@gmail.com>

Reviewers have included (in no particular order) Andres Freund, Alexander
Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu,
Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby, Álvaro Herrera,
Peter Eisentraut

Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
Discussion: https://postgr.es/m/20220616233130.rparivafipt6doj3@alap3.anarazel.de
Discussion: https://postgr.es/m/abd9b83b-aa66-f230-3d6d-734817f0995d%40postgresql.org
Discussion: https://postgr.es/m/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
2023-07-26 17:08:33 +09:00
Michael Paquier
e35cc3b3f2 pgbench: Use COPY for client-side data generation
This commit switches the client-side data generation from INSERT queries
to COPY for the two tables pgbench_branches and pgbench_tellers.
pgbench_accounts was already using COPY.

COPY is a better interface for bulk loading or high latency connections
(this point can be countered with the option for server-side data
generation, still client-side is the default), and measurements have
proved that using it for these two other tables can lead to improvements
during initialization.  I did not notice slowdowns at large scale
numbers on a local setup, either, most of the work happening for the
accounts table.

Previously COPY was only used for the pgbench_accounts table because the
amount of data was much larger than the two other tables.  The code is
refactored so as all three tables use the same code path to execute the
COPY queries, with a callback to build data rows.

Author: Tristan Partin
Discussion: https://postgr.es/m/CSTU5P82ONZ1.19XFUGHMXHBRY@c3po
2023-07-24 13:48:22 +09:00
Daniel Gustafsson
29a0ccbce9 Revert "Add notBefore and notAfter to SSL cert info display"
Due to an oversight in reviewing, this used functionality not
compatible with old versions of OpenSSL.

This reverts commit 75ec5e7bec.
2023-07-20 17:18:12 +02:00
Daniel Gustafsson
75ec5e7bec Add notBefore and notAfter to SSL cert info display
This adds the X509 attributes notBefore and notAfter to sslinfo
as well as pg_stat_ssl to allow verifying and identifying the
validity period of the current client certificate.

Author: Cary Huang <cary.huang@highgo.ca>
Discussion: https://postgr.es/m/182b8565486.10af1a86f158715.2387262617218380588@highgo.ca
2023-07-20 17:07:32 +02:00
Nathan Bossart
ab29a7a9c6 Doc: move unparenthesized syntaxes for a few commands.
Move documentation of the unparenthesized syntaxes for VACUUM,
ANALYZE, EXPLAIN, and CLUSTER to the "Compatibility" section of
their documentation to improve readability of the preferred,
parenthesized syntaxes.

Author: Melanie Plageman
Discussion: https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:59 -07:00
Nathan Bossart
cdaedfc96d Support parenthesized syntax for CLUSTER without a table name.
b5913f6120 added a parenthesized syntax for CLUSTER, but it
requires specifying a table name.  This is unlike commands such as
VACUUM and ANALYZE, which do not require specifying a table in the
parenthesized syntax.  This change resolves this inconsistency.
This is preparatory work for a follow-up commit that will move the
unparenthesized syntax to the "Compatibility" section of the
CLUSTER documentation.

Reviewed-by: Melanie Plageman, Michael Paquier
Discussion: https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:52 -07:00
Nathan Bossart
018b61f81b Rearrange CLUSTER rules in gram.y.
This change moves the unparenthesized syntax for CLUSTER to the end
of the ClusterStmt rules in preparation for a follow-up commit that
will move this syntax to the "Compatibility" section of the CLUSTER
documentation.  The documentation for the CLUSTER syntaxes has also
been consolidated.

Suggested-by: Melanie Plageman
Discussion https://postgr.es/m/CAAKRu_bc5uHieG1976kGqJKxyWtyQt9yvktjsVX%2Bi7NOigDjOA%40mail.gmail.com
2023-07-19 15:26:43 -07:00
Tom Lane
d65ddaca93 Add psql \drg command to display role grants.
With the addition of INHERIT and SET options for role grants,
the historical display of role memberships in \du/\dg is woefully
inadequate.  Besides those options, there are pre-existing
shortcomings that you can't see the ADMIN option nor the grantor.

To fix this, remove the "Member of" column from \du/\dg altogether
(making that output usefully narrower), and invent a new meta-command
"\drg" that is specifically for displaying role memberships.  It
shows one row for each role granted to the selected role(s), with
the grant options and grantor.

We would not normally back-patch such a feature addition post
feature freeze, but in this case the change is mainly driven by
v16 changes in the server, so it seems appropriate to include it
in v16.

Pavel Luzanov, with bikeshedding and review from a lot of people,
but particularly David Johnston

Discussion: https://postgr.es/m/b9be2d0e-a9bc-0a30-492f-a4f68e4f7740@postgrespro.ru
2023-07-19 12:46:30 -04:00
Tom Lane
15c68cd84a Doc: improve description of IN and row-constructor comparisons.
IN and NOT IN work fine on records and arrays, so just say that
they accept "expressions" not "scalar expressions".  I think that
that phrasing was meant to say that they don't work on set-returning
expressions, but that's not the common meaning of "scalar".

Revise the description of row-constructor comparisons to make it
perhaps a bit less confusing.  (This partially reverts some
dubious wording changes made by commit f56651519.)

Per gripe from Ilya Nenashev.  Back-patch to supported branches.
In HEAD and v16, also drop a NOTE about pre-8.2 behavior, which
is hopefully no longer of interest to anybody.

Discussion: https://postgr.es/m/168968062460.632.14303906825812821399@wrigleys.postgresql.org
2023-07-19 11:00:34 -04:00
Michael Paquier
3f8c98d0b6 pg_archivecleanup: Add --clean-backup-history
By default, pg_archivecleanup does not remove backup history files.
These are just few bytes useful for debugging purposes, still keeping
them around can bloat an archive path history files mixed with the WAL
segments if the path has a long history.

This patch adds a new option to control if backup history files are
removed, depending on the oldest segment name to keep around.

While on it, the TAP tests are refactored so as these are now able to
handle lists of files.  Each file has a flag to track if it should still
exist or not depending on the oldest segment defined with the command
run.

Author: Atsushi Torikoshi
Reviewed-by: Kyotaro Horiguchi, Fujii Masao, Michael Paquier
Discussion: https://postgr.es/m/d660ef741ce3d82f3b4283f1cafd576c@oss.nttdata.com
2023-07-19 13:41:22 +09:00
Amit Kapila
961cf5c905 Doc: Update the logical replication restriction w.r.t Replica Identity Full.
Author: Kuroda Hayato
Reviewed-by: Peter Smith, Onder Kalaci, Sergei Kornilov, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58662174ED62648E0D611194F530A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-07-19 08:11:44 +05:30
Tom Lane
137b131d6f Doc: fix out-of-date example of SPI usage.
The "count" argument of SPI_exec() only limits execution when
the query is actually returning rows.  This was not the case
before PG 9.0, so this example was correct when written; but
we missed updating it in commit 2ddc600f8.  Extend the example
to show the behavior both with and without RETURNING.

While here, improve the commentary and markup for the rest
of the example.

David G. Johnston and Tom Lane, per report from Curt Kolovson.
Back-patch to all supported branches.

Discussion: https://postgr.es/m/CANhYJV6HWtgz_qjx_APfK0PAgLUzY-2vjLuj7i_o=TZF1LAQew@mail.gmail.com
2023-07-18 11:59:39 -04:00
Nathan Bossart
884eee5bfb Remove db_user_namespace.
This feature was intended to be a temporary measure to support
per-database user names.  A better one hasn't materialized in the
~21 years since it was added, and nobody claims to be using it, so
let's just remove it.

Reviewed-by: Michael Paquier, Magnus Hagander
Discussion: https://postgr.es/m/20230630200509.GA2830328%40nathanxps13
Discussion: https://postgr.es/m/20230630215608.GD2941194%40nathanxps13
2023-07-17 11:44:59 -07:00
Nathan Bossart
a0363ab7aa Fix privilege check for SET SESSION AUTHORIZATION.
Presently, the privilege check for SET SESSION AUTHORIZATION checks
whether the original authenticated role was a superuser at
connection start time.  Even if the role loses the superuser
attribute, its existing sessions are permitted to change session
authorization to any role.

This commit modifies this privilege check to verify the original
authenticated role currently has superuser.  In the event that the
authenticated role loses superuser within a session authorization
change, the authorization change will remain in effect, which means
the user can still take advantage of the target role's privileges.
However, [RE]SET SESSION AUTHORIZATION will only permit switching
to the original authenticated role.

Author: Joseph Koshakow
Discussion: https://postgr.es/m/CAAvxfHc-HHzONQ2oXdvhFF9ayRnidPwK%2BfVBhRzaBWYYLVQL-g%40mail.gmail.com
2023-07-13 21:13:45 -07:00
Amit Kapila
edca342434 Allow the use of a hash index on the subscriber during replication.
Commit 89e46da5e5 allowed using BTREE indexes that are neither
PRIMARY KEY nor REPLICA IDENTITY on the subscriber during apply of
update/delete. This patch extends that functionality to also allow HASH
indexes.

We explored supporting other index access methods as well but they don't
have a fixed strategy for equality operation which is required by the
current infrastructure in logical replication to scan the indexes.

Author: Kuroda Hayato
Reviewed-by: Peter Smith, Onder Kalaci, Amit Kapila
Discussion: https://postgr.es/m/TYAPR01MB58669D7414E59664E17A5827F522A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-07-14 08:21:54 +05:30
Thomas Munro
d0c28601ef Remove wal_sync_method=fsync_writethrough on Windows.
The "fsync" level already flushes drive write caches on Windows (as does
"fdatasync"), so it only confuses matters to have an apparently higher
level that isn't actually different at all.

That leaves "fsync_writethrough" only for macOS, where it actually does
something different.

Reviewed-by: Magnus Hagander <magnus@hagander.net>
Discussion: https://postgr.es/m/CA%2BhUKGJ2CG2SouPv2mca2WCTOJxYumvBARRcKPraFMB6GSEMcA%40mail.gmail.com
2023-07-14 12:30:13 +12:00
Andres Freund
c66a7d75e6 Handle DROP DATABASE getting interrupted
Until now, when DROP DATABASE got interrupted in the wrong moment, the removal
of the pg_database row would also roll back, even though some irreversible
steps have already been taken. E.g. DropDatabaseBuffers() might have thrown
out dirty buffers, or files could have been unlinked. But we continued to
allow connections to such a corrupted database.

To fix this, mark databases invalid with an in-place update, just before
starting to perform irreversible steps. As we can't add a new column in the
back branches, we use pg_database.datconnlimit = -2 for this purpose.

An invalid database cannot be connected to anymore, but can still be
dropped.

Unfortunately we can't easily add output to psql's \l to indicate that some
database is invalid, it doesn't fit in any of the existing columns.

Add tests verifying that a interrupted DROP DATABASE is handled correctly in
the backend and in various tools.

Reported-by: Evgeny Morozov <postgresql3@realityexists.net>
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20230509004637.cgvmfwrbht7xm7p6@awork3.anarazel.de
Discussion: https://postgr.es/m/20230314174521.74jl6ffqsee5mtug@awork3.anarazel.de
Backpatch: 11-, bug present in all supported versions
2023-07-13 13:03:28 -07:00
Masahiko Sawada
fd48a86c62 Doc: clarify the conditions of usable indexes for REPLICA IDENTITY FULL tables.
Commit 89e46da5e allowed REPLICA IDENTITY FULL tables to use an index
on the subscriber during apply of update/delete. This commit clarifies
in the documentation that the leftmost field of candidate indexes must
be a column (not an expression) that references the published relation
column.

The source code comments are also updated accordingly.

Reviewed-by: Peter Smith, Amit Kapila
Discussion: https://postgr.es/m/CAD21AoDJjffEvUFKXT27Q5U8-UU9JHv4rrJ9Ke8Zkc5UPWHLvA@mail.gmail.com
Backpatch-through: 16
2023-07-13 15:03:17 +09:00
Daniel Gustafsson
e68dd9e907 doc: Clarify the errhint example
The errhint example wasn't using the mandated style for errhint
which was needlessly confusing.  Fix by rewriting it using the
mandated style.

Reported-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Jonathan S. Katz <jkatz@postgresql.org>
Discussion: https://postgr.es/m/F02F4B4F-7B7E-406A-B25A-FF2AA300AB9A@yesql.se
2023-07-12 19:38:51 +02:00
Peter Eisentraut
8c852ba9a4 Allow some exclusion constraints on partitions
Previously we only allowed unique B-tree constraints on partitions
(and only if the constraint included all the partition keys).  But we
could allow exclusion constraints with the same restriction.  We also
require that those columns be compared for equality, not something
like &&.

Author: Paul A. Jungwirth <pj@illuminatedcomputing.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://www.postgresql.org/message-id/flat/ec8b1d9b-502e-d1f8-e909-1bf9dffe6fa5@illuminatedcomputing.com
2023-07-12 09:25:17 +02:00
Thomas Munro
ce0b0fa3e7 Doc: Adjust libpq docs about thread safety.
Describe the situation now that --disable-thread-safety is gone.

Author: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
2023-07-12 08:57:55 +12:00
Thomas Munro
68a4b58eca Remove --disable-thread-safety and related code.
All supported computers have either POSIX or Windows threads, and we no
longer have any automated testing of --disable-thread-safety.  We define
a vestigial ENABLE_THREAD_SAFETY macro to 1 in ecpg_config.h in case it
is useful, but we no longer test it anywhere in PostgreSQL code, and
associated dead code paths are removed.

The Meson and perl-based Windows build scripts never had an equivalent
build option.

Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com
2023-07-12 08:20:43 +12:00
Masahiko Sawada
46ebdfe164 Report index vacuum progress.
This commit adds two columns: indexes_total and indexes_processed, to
pg_stat_progress_vacuum system view to show the index vacuum
progress. These numbers are reported in the "vacuuming indexes" and
"cleaning up indexes" phases.

This uses the new parallel message type for progress reporting added
by be06506e7.

Bump catversion because this changes the definition of
pg_stat_progress_vacuum.

Author: Sami Imseih
Reviewed by: Masahiko Sawada, Michael Paquier, Nathan Bossart, Andres Freund
Discussion: https://www.postgresql.org/message-id/flat/5478DFCD-2333-401A-B2F0-0D186AB09228@amazon.com
2023-07-11 12:34:01 +09:00
Peter Eisentraut
3c963d33ea doc: Use proper markup for emphasis 2023-07-09 10:02:15 +02:00