Commit Graph

6952 Commits

Author SHA1 Message Date
Peter Eisentraut 721856ff24 Remove distprep
A PostgreSQL release tarball contains a number of prebuilt files, in
particular files produced by bison, flex, perl, and well as html and
man documentation.  We have done this consistent with established
practice at the time to not require these tools for building from a
tarball.  Some of these tools were hard to get, or get the right
version of, from time to time, and shipping the prebuilt output was a
convenience to users.

Now this has at least two problems:

One, we have to make the build system(s) work in two modes: Building
from a git checkout and building from a tarball.  This is pretty
complicated, but it works so far for autoconf/make.  It does not
currently work for meson; you can currently only build with meson from
a git checkout.  Making meson builds work from a tarball seems very
difficult or impossible.  One particular problem is that since meson
requires a separate build directory, we cannot make the build update
files like gram.h in the source tree.  So if you were to build from a
tarball and update gram.y, you will have a gram.h in the source tree
and one in the build tree, but the way things work is that the
compiler will always use the one in the source tree.  So you cannot,
for example, make any gram.y changes when building from a tarball.
This seems impossible to fix in a non-horrible way.

Second, there is increased interest nowadays in precisely tracking the
origin of software.  We can reasonably track contributions into the
git tree, and users can reasonably track the path from a tarball to
packages and downloads and installs.  But what happens between the git
tree and the tarball is obscure and in some cases non-reproducible.

The solution for both of these issues is to get rid of the step that
adds prebuilt files to the tarball.  The tarball now only contains
what is in the git tree (*).  Getting the additional build
dependencies is no longer a problem nowadays, and the complications to
keep these dual build modes working are significant.  And of course we
want to get the meson build system working universally.

This commit removes the make distprep target altogether.  The make
dist target continues to do its job, it just doesn't call distprep
anymore.

(*) - The tarball also contains the INSTALL file that is built at make
dist time, but not by distprep.  This is unchanged for now.

The make maintainer-clean target, whose job it is to remove the
prebuilt files in addition to what make distclean does, is now just an
alias to make distprep.  (In practice, it is probably obsolete given
that git clean is available.)

The following programs are now hard build requirements in configure
(they were already required by meson.build):

- bison
- flex
- perl

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
2023-11-06 15:18:04 +01:00
Peter Eisentraut 7b5275eec3 pg_resetwal: Add more tests and test coverage
pg_resetwal had poor test coverage.  There are some TAP tests, but
they all run with -n, so they don't actually test the full
functionality.  (There is a non-dry-run call of pg_resetwal in the
recovery test suite, but that is incidental.)

This adds a bunch of more tests to test all the different options and
scenarios.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-11-06 09:16:00 +01:00
Bruce Momjian 42d3125ada doc: \copy can get data values \. and end-of-input confused
Reported-by: Svante Richter

Discussion: https://postgr.es/m/fcd57e4-8f23-4c3e-a5db-2571d09208e2@beta.fastmail.com

Backpatch-through: 11
2023-11-03 13:57:59 -04:00
Michael Paquier 7021d3b176 Prevent startup of logical replication launcher during pg_upgrade
The logical replication launcher may start apply workers during an
upgrade.  This could be the cause of corruptions on a new cluster if
these are able to apply changes before the physical files are copied
over to the new cluster.

The chance of being able to do so is small as pg_upgrade uses its own
port and unix domain directory (the latter is customizable with
--socketdir), but just preventing the launcher to start is safer at the
end, because we are then sure that no changes will be applied.  Like
29d0a77fa6 for max_slot_wal_keep_size, this is only set when a cluster
uses v17 or newer.

Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm2g9ZKf=y8X6z6MsLCuh8WwU-=Q6pLj35NFi2M5BZNS_A@mail.gmail.com
2023-11-02 14:34:51 +09:00
Bruce Momjian 3fdc1d2693 C comment: mention why no setting lasterrno in dir_existsfile()
Reported-by: Wei Sun

Discussion: https://postgr.es/m/tencent_1276C08F98579CC19D8A4488C848A8411806@qq.com

Backpatch-through: master
2023-10-31 11:59:14 -04:00
Bruce Momjian 56b30e266e pgindent run to fix commits de64268561 and 5ae2087202
Reported-by: Michael Paquier

Discussion: https://postgr.es/m/ZT9YH7-TTx27V3yW@paquier.xyz

Backpatch-through: master
2023-10-30 14:52:35 -04:00
Tom Lane 5ba4cc3090 Teach pg_dump about the new pg_subscription.subrunasowner option.
Among numerous other oversights, commit 482675987 neglected to fix
pg_dump to dump this new subscription option.  Since the new default
is "false" while the previous behavior corresponds to "true", this
would cause legacy subscriptions to silently change behavior during
dump/reload or pg_upgrade.  That seems like a bad idea.  Even if it
was intended, failing to preserve the option once set in a new
installation is certainly not OK.

While here, reorder associated stanzas in pg_dump to match the
field order in pg_subscription, in hopes of reducing the impression
that all this code was written with the aid of a dartboard.

Back-patch to v16 where this new field was added.

Philip Warner (cosmetic tweaks by me)

Discussion: https://postgr.es/m/20231027042539.01A3A220F0A@thebes.rime.com.au
2023-10-29 12:56:35 -04:00
Alexander Korotkov 5ae2087202 Teach contrib/amcheck to check the unique constraint violation
Add the 'checkunique' argument to bt_index_check() and bt_index_parent_check().
When the flag is specified the procedures will check the unique constraint
violation for unique indexes.  Only one heap entry for all equal keys in
the index should be visible (including posting list entries).  Report an error
otherwise.

pg_amcheck called with the --checkunique option will do the same check for all
the indexes it checks.

Author: Anastasia Lubennikova <lubennikovaav@gmail.com>
Author: Pavel Borisov <pashkin.elfe@gmail.com>
Author: Maxim Orlov <orlovmg@gmail.com>
Reviewed-by: Mark Dilger <mark.dilger@enterprisedb.com>
Reviewed-by: Zhihong Yu <zyu@yugabyte.com>
Reviewed-by: Peter Geoghegan <pg@bowt.ie>
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://postgr.es/m/CALT9ZEHRn5xAM5boga0qnrCmPV52bScEK2QnQ1HmUZDD301JEg%40mail.gmail.com
2023-10-28 00:21:23 +03:00
Amit Kapila 8949b978ff Commit b195e6d482 forgot to update meson.build.
Author: Hayato Kuroda
Discussion: http://postgr.es/m/TYCPR01MB58701BA358D80073ED1BD528F5DCA@TYCPR01MB5870.jpnprd01.prod.outlook.com
2023-10-27 13:19:49 +05:30
Amit Kapila b195e6d482 Use shorter file names in the upgrade logical slots test.
The longer file names exceeded the Windows path limit on buildfarm animal
fairywren.

Diagnosed-by: Hou Zhijie
Author: Hayato Kuroda
Reviewed-by: Bharath Rupireddy
Discussion: http://postgr.es/m/OS0PR01MB57160DF709ACD02248DB830C94DDA@OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-10-27 11:09:45 +05:30
Peter Eisentraut 611806cd72 Add trailing commas to enum definitions
Since C99, there can be a trailing comma after the last value in an
enum definition.  A lot of new code has been introducing this style on
the fly.  Some new patches are now taking an inconsistent approach to
this.  Some add the last comma on the fly if they add a new last
value, some are trying to preserve the existing style in each place,
some are even dropping the last comma if there was one.  We could
nudge this all in a consistent direction if we just add the trailing
commas everywhere once.

I omitted a few places where there was a fixed "last" value that will
always stay last.  I also skipped the header files of libpq and ecpg,
in case people want to use those with older compilers.  There were
also a small number of cases where the enum type wasn't used anywhere
(but the enum values were), which ended up confusing pgindent a bit,
so I left those alone.

Discussion: https://www.postgresql.org/message-id/flat/386f8c45-c8ac-4681-8add-e3b0852c1620%40eisentraut.org
2023-10-26 09:20:54 +02:00
Amit Kapila 29d0a77fa6 Migrate logical slots to the new node during an upgrade.
While reading information from the old cluster, a list of logical
slots is fetched. At the later part of upgrading, pg_upgrade revisits the
list and restores slots by executing pg_create_logical_replication_slot()
on the new cluster. Migration of logical replication slots is only
supported when the old cluster is version 17.0 or later.

If the old node has invalid slots or slots with unconsumed WAL records,
the pg_upgrade fails. These checks are needed to prevent data loss.

The significant advantage of this commit is that it makes it easy to
continue logical replication even after upgrading the publisher node.
Previously, pg_upgrade allowed copying publications to a new node. With
this patch, adjusting the connection string to the new publisher will
cause the apply worker on the subscriber to connect to the new publisher
automatically. This enables seamless continuation of logical replication,
even after an upgrade.

Author: Hayato Kuroda, Hou Zhijie
Reviewed-by: Peter Smith, Bharath Rupireddy, Dilip Kumar, Vignesh C, Shlok Kyal
Discussion: http://postgr.es/m/TYAPR01MB58664C81887B3AF2EB6B16E3F5939@TYAPR01MB5866.jpnprd01.prod.outlook.com
Discussion: http://postgr.es/m/CAA4eK1+t7xYcfa0rEQw839=b2MzsfvYDPz3xbD+ZqOdP3zpKYg@mail.gmail.com
2023-10-26 07:06:55 +05:30
Nathan Bossart fdeb6e6a74 Remove dead code in pg_ctl.c.
Missed in 39969e2a1e.

Author: David Steele
Discussion: https://postgr.es/m/0c742f0c-d663-419d-b5a7-4fe867f5566c%40pgmasters.net
2023-10-25 16:26:59 -05:00
Michael Paquier 3f9b1f26ca pg_upgrade: Fix test name in 002_pg_upgrade.pl
Author: Hou Zhijie
Discussion: https://postgr.es/m/TYAPR01MB5724A40D47E71F4717357EC694D5A@TYAPR01MB5724.jpnprd01.prod.outlook.com
Backpatch-through: 15
2023-10-18 18:23:25 +09:00
Peter Eisentraut 41a284411e Clean up some pg_dump tests
1) Remove useless entries from "unlike" lists.  Runs that are not
   listed in "like" don't need to be excluded in "unlike".

2) Ensure there is always a "like" list, even if it is empty.  This
   makes the test more self-documenting.

3) Use predefined lists such as %full_runs where appropriate, instead
   of listing all runs separately.

Also add code that checks 1 and 2 automatically and dies with an error
for violations.

Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/1f8cb371-e84e-434e-0367-6b716fb16fa1@eisentraut.org
2023-10-18 08:08:17 +02:00
Alexander Korotkov e83d1b0c40 Add support event triggers on authenticated login
This commit introduces trigger on login event, allowing to fire some actions
right on the user connection.  This can be useful for logging or connection
check purposes as well as for some personalization of environment.  Usage
details are described in the documentation included, but shortly usage is
the same as for other triggers: create function returning event_trigger and
then create event trigger on login event.

In order to prevent the connection time overhead when there are no triggers
the commit introduces pg_database.dathasloginevt flag, which indicates database
has active login triggers.  This flag is set by CREATE/ALTER EVENT TRIGGER
command, and unset at connection time when no active triggers found.

Author: Konstantin Knizhnik, Mikhail Gribkov
Discussion: https://postgr.es/m/0d46d29f-4558-3af9-9c85-7774e14a7709%40postgrespro.ru
Reviewed-by: Pavel Stehule, Takayuki Tsunakawa, Greg Nancarrow, Ivan Panchenko
Reviewed-by: Daniel Gustafsson, Teodor Sigaev, Robert Haas, Andres Freund
Reviewed-by: Tom Lane, Andrey Sokolov, Zhihong Yu, Sergey Shinderuk
Reviewed-by: Gregory Stark, Nikita Malakhov, Ted Yu
2023-10-16 03:18:22 +03:00
Noah Misch 90ebcc32d9 Don't spuriously report FD_SETSIZE exhaustion on Windows.
Starting on 2023-08-03, this intermittently terminated a "pgbench -C"
test in CI.  It could affect a high-client-count "pgbench" without "-C".
While parallel reindexdb and vacuumdb reach the same problematic check,
sufficient client count and/or connection turnover is less plausible for
them.  Given the lack of examples from the buildfarm or from manual
builds, reproducing this must entail rare operating system
configurations.  Also correct the associated error message, which was
wrong for non-Windows.  Back-patch to v12, where the pgbench check first
appeared.  While v11 vacuumdb has the problematic check, reaching it
with typical vacuumdb usage is implausible.

Reviewed by Thomas Munro.

Discussion: https://postgr.es/m/CA+hUKG+JwvTNdcyJTriy9BbtzF1veSRQ=9M_ZKFn9_LqE7Kp7Q@mail.gmail.com
2023-10-14 15:54:46 -07:00
Michael Paquier d16eb83aba psql: Add completion support for AT [ LOCAL | TIME ZONE ]
AT TIME ZONE is completed with a list of supported timezones, something
not needed by AT LOCAL.

Author: Dagfinn Ilmari Mannsåker
Reviewed-by: Jim Jones
Discussion: https://postgr.es/m/87jzyzsvgv.fsf@wibble.ilmari.org
2023-10-13 14:19:07 +09:00
Michael Paquier c7c801ef3b Improve some wording in pg_upgrade/IMPLEMENTATION
Author: Gurjeet Singh
Discussion: https://postgr.es/m/CABwTF4VFKtKrb78fBnMXwHvOu4a+-7y86siBSEety2knti2eGA@mail.gmail.com
2023-10-11 13:54:33 +09:00
Peter Eisentraut e3679bc1c3 pg_resetwal: Corrections around -c option
The present pg_resetwal code hardcodes the minimum value for -c as 2,
which is FrozenTransactionId, but it's not clear why that is allowed.
After some research, it was probably a mistake in the original patch.
Change it to FirstNormalTransactionId, which matches other xid-related
options in pg_resetwal.

Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/d09f0e91-8757-642b-1a92-da9a52f5589a%40eisentraut.org
2023-10-10 08:58:50 +02:00
Tom Lane b6c7cfac88 Restore proper linkage of pg_char_to_encoding() and friends.
Back in the 8.3 era we discovered that it was problematic if
libpq.so had encoding ID assignments different from the backend,
which is possible because on some platforms libpq.so might be
of a different major version from the calling programs.
psql should use libpq's assignments, but initdb has to use the
backend's, else it will put wrong values into pg_database.
The solution devised in commit 8468146b0 relied on giving initdb
its own copy of encnames.c rather than relying on the functions
exported by libpq.  Later, that metamorphosed into ensuring that
libpgcommon got linked before libpq -- which made things OK for
initdb but broke psql.  We didn't notice for lack of any changes
in enum pg_enc since then.  Commit 06843df4a reversed that, fixing
the latent bug in psql but adding one in initdb.  The meson build
infrastructure is also not being sufficiently careful about link
order, and trying to make it so would be equally fragile.

Hence, let's use a new scheme based on giving the libpq-exported
symbols different real names than the same functions exported from
libpgcommon.a or libpgcommon_srv.a.  (We could distinguish those
two cases as well, but there seems no need to.)  libpq gets the
official names to avoid an ABI break for libpq clients, while the
other cases use #define's to make the real names "xxx_private"
rather than "xxx".  By controlling where the #define's are
applied, we can force any particular client program to use one
set or the other of the encnames.c functions.

We cannot back-patch this, since it'd be an ABI break for backend
loadable modules, but there seems little need to.  We're just
trying to ensure that the world is safe for hypothetical future
additions to enum pg_enc.

In passing this should fix "duplicate symbol" linker warnings
that we've been seeing on AIX buildfarm members since commit
06843df4a.  It's not very clear why that linker is complaining
now, when there were strictly *more* duplicates visible before,
but in any case this should remove the reason for complaint.

Patch by me; thanks to Andres Freund for review.

Discussion: https://postgr.es/m/2385119.1696354473@sss.pgh.pa.us
2023-10-07 12:08:10 -04:00
Peter Eisentraut b4336515b0 Remove environment-variable-based defaults in psql --help
This seemed inconsistent with the --help output of other tools.
Depending on the values, it can cause ugly formatting.  Also, we're
not getting the defaults from libpq, we're just emulating the methods
libpq uses to derive these values, so they might not be 100% correct.

Author: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Discussion: https://www.postgresql.org/message-id/flat/50ca8ff35a8dd8f9ec89963b503571a7@oss.nttdata.com
2023-10-06 11:54:36 +02:00
Alvaro Herrera 1c99cde2f3
Improve JsonLexContext's freeability
Previously, the JSON code didn't have to worry too much about freeing
JsonLexContext, because it was never too long-lived.  With new features
being added for SQL/JSON this is no longer the case.  Add a routine
that knows how to free this struct and apply that to a few places, to
prevent this from becoming problematic.

At the same time, we change the API of makeJsonLexContextCstringLen to
make it receive a pointer to JsonLexContext for callers that want it to
be stack-allocated; it can also be passed as NULL to get the original
behavior of a palloc'ed one.

This also causes an ABI break due to the addition of flags to
JsonLexContext, so we can't easily backpatch it.  AFAICS that's not much
of a problem; apparently some leaks might exist in JSON usage of
text-search, for example via json_to_tsvector, but I haven't seen any
complaints about that.

Per Coverity complaint about datum_to_jsonb_internal().

Discussion: https://postgr.es/m/20230808174110.oq3iymllsv6amkih@alvherre.pgsql
2023-10-05 10:59:08 +02:00
Peter Eisentraut 8666cf65ea Modernize const handling with readline
The comment

    /* On some platforms, readline is declared as readline(char *) */

is obsolete.  The casting away of const can be removed.

The const in the readline() prototype was added in GNU readline 4.2,
released in 2001.  BSD libedit has also had const in the prototype
since at least 2001.

(The commit that introduced this comment (187e865174) talked about
FreeBSD 4.8, which didn't have readline compatibility in libedit yet,
so it must have been talking about GNU readline in the base system.
This checks out, but already FreeBSD 5 had an updated GNU readline
with const.)

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/862fc1d4-9a0c-d2b6-5451-ee3dc750bcab%40eisentraut.org
2023-10-05 08:45:50 +02:00
Michael Paquier 6c77bb42ab Replace use of stat()[7] by -s switch in TAP tests to retrieve file size
The list form of stat() is an inelegant API as it relies on the position
of the file size in the list returned in result.  Like in any other
places of the tree, replace that with a -s switch instead.

Another suggestion from Dagfinn is File::Stat, which we've been already
using for some other fields.  It really comes down to a matter of taste
to choose that over -s, and the latter is more used in the tree.

Author: Bertrand Drouvot
Reviewed-by: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/b2020df7-d0fc-4ea5-b2a9-7efc6d36b2ac@gmail.com
2023-10-03 08:27:34 +09:00
Tom Lane 06c0c36884 Fix omission of column-level privileges in selective pg_restore.
In a selective restore, ACLs for a table should be dumped if the
table is selected to be dumped.  However, if the table has both
table-level and column-level ACLs, only the table-level ACL was
restored.  This happened because _tocEntryRequired assumed that
an ACL could have only one dependency (the one on its table),
and punted if there was more than one.  But since commit ea9125304,
column-level ACLs also depend on the table-level ACL if any, to
ensure correct ordering in parallel restores.  To fix, adjust the
logic in _tocEntryRequired to ignore dependencies on ACLs.

I extended a test case in 002_pg_dump.pl so that it purports to
test for this; but in fact the test passes even without the fix.
That's because this bug only manifests during a selective restore,
while the scenarios 002_pg_dump.pl tests include only selective dumps.
Perhaps somebody would like to extend the script so that it can test
scenarios including selective restore, but I'm not touching that.

Euler Taveira and Tom Lane, per report from Kong Man.
Back-patch to all supported branches.

Discussion: https://postgr.es/m/DM4PR11MB73976902DBBA10B1D652F9498B06A@DM4PR11MB7397.namprd11.prod.outlook.com
2023-10-02 13:27:58 -04:00
Heikki Linnakangas be8d4cb13c Add regression tests for psql \g piped into a program
Author: Daniel Vérité
Reviewed-by: Peter Eisentraut
Discussion: https://www.postgresql.org/message-id/33ce8350-8cd1-45ff-a5fe-f9be7bc70649%40manitou-mail.org
2023-10-02 11:46:25 +03:00
Michael Paquier 2940f1c837 psql: Set variables from query result on failure when printing tuples
SetResultVariables() was not getting called when "printing" a result
that failed (see around PrintQueryResult), which would cause some
variables to not be set, like ROW_COUNT, SQLSTATE or ERROR.  This can be
confusing as a previous result would be retained.

This state could be reached when failing to process tuples in a few
commands, like \gset when it returns no tuples, or \crosstabview.  A
test is added, based on \gset.

This is arguably a bug fix, but no backpatch is done as there is a risk
of breaking scripts that rely on the previous behavior, even if they do
so accidentally.

Reported-by: amutu
Author: Japin Li
Reviewed-by: Tom Lane, Michael Paquier
Discussion: https://postgr.es/m/18134-87126d90cb4dd049@postgresql.org
2023-10-02 11:05:05 +09:00
Peter Eisentraut 5daa50f962 Revert "pg_resetwal: Improve error with wrong/missing data directory"
This reverts commit 1d863c2504.

This broke specifying the data directory as a relative path.

Reported-by: Hayato Kuroda (Fujitsu) <kuroda.hayato@fujitsu.com>
Discussion: https://www.postgresql.org/message-id/flat/TYAPR01MB58664AD301F511B1EA5B72B4F5C0A%40TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-09-29 10:59:46 +02:00
Peter Eisentraut 5f1b00e64a doc: Improve documentation about pg_resetwal -f option
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-09-28 12:08:54 +02:00
Peter Eisentraut a11d8e10f2 pg_resetwal: Use frontend logging API
This now causes error messages related to the lack of the -f option to
appear on standard error rather than standard output.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-09-28 11:58:36 +02:00
Peter Eisentraut b5da1b3a93 pg_resetwal: Regroup --help output
Put the options to modify the control values into a separate group.
This matches the outline of the man page.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-09-28 11:49:20 +02:00
Peter Eisentraut 1d863c2504 pg_resetwal: Improve error with wrong/missing data directory
Run chdir() before permission check to get a less confusing error
message if the specified data directory does not exist.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-09-28 11:40:00 +02:00
Peter Eisentraut 7273945caf pg_resetwal: Update an obsolete comment
The comment claimed that pg_resetwal updates the pg_control file if it
is of an old version.  This has apparently never been true.  Also, in
c3c09be34b, another comment was added elsewhere that this currently
does not happen.  So this comment is wrong and redundant and can be
removed.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-09-28 11:27:22 +02:00
Peter Eisentraut 639e1aa81f pgbench: Improve help output of -I option
Add a description of the step letters to the --help output.

Author: Gurjeet Singh <gurjeet@singh.im>
Reviewed-by: Tristen Raab <tristen.raab@highgo.ca>
Discussion: https://www.postgresql.org/message-id/flat/CABwTF4Xbc=K4tFj5Znc8jx0GCufQa577GCDsWD7=71qDnUEOyQ@mail.gmail.com
2023-09-26 22:09:07 +01:00
Peter Eisentraut 64b787656d Add some const qualifiers
There was a mismatch between the const qualifiers for
excludeDirContents in src/backend/backup/basebackup.c and
src/bin/pg_rewind/filemap.c, which led to a quick search for similar
cases.  We should make excludeDirContents match, but the rest of the
changes seem like a good idea as well.

Author: David Steele <david@pgmasters.net>
Discussion: https://www.postgresql.org/message-id/flat/669a035c-d23d-2f38-7ff0-0cb93e01d610@pgmasters.net
2023-09-26 11:28:57 +01:00
Andres Freund 849d367ff9 pg_dump: tests: Correct test condition for invalid databases
For some reason I used not_like = { pg_dumpall_dbprivs => 1, } in the test
condition of one of the tests added in in c66a7d75e6. That doesn't make sense
for two reasons: 1) not_like isn't a valid test condition 2) the database
should not be dumped in any of the tests.  Due to 1), the test achieved its
goal, but clearly the formulation is confusing.  Instead use like => {}, with
a comment explaining why.

Reported-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/3ddf79f2-8b7b-a093-11d2-5c739bc64f86@eisentraut.org
Backpatch: 11-, like c66a7d75e6
2023-09-25 12:07:48 -07:00
Daniel Gustafsson aa9de547b7 vacuumdb: Reword --help message for clarity
The --help output stated that schemas were specified using PATTERN
when they in fact aren't pattern matched but are required to be
exact matches. This changes to SCHEMA to make that clear.

Backpatch through v16 where this was introduced.

Author: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp>
Discussion: https://postgr.es/m/CAMyC8qp9mXPQd5D6s6CJxvmignsbTqGZwDDB6VYJOn1A8WG38w@mail.gmail.com
Backpatch-through: 16
2023-09-25 16:03:32 +02:00
Daniel Gustafsson fb56a18117 vacuumdb: Fix excluding multiple schemas with -N
When specifying multiple schemas to exclude with -N parameters, none
of the schemas are actually excluded (a single -N worked as expected).
This fixes the catalog query to handle multiple exclusions and adds a
test for this case.

Backpatch to v16 where this was introduced.

Author: Nathan Bossart <nathandbossart@gmail.com>
Author: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp>
Reported-by: Kuwamura Masaki <kuwamura@db.is.i.nagoya-u.ac.jp>
Discussion: https://postgr.es/m/CAMyC8qp9mXPQd5D6s6CJxvmignsbTqGZwDDB6VYJOn1A8WG38w@mail.gmail.com
Backpatch-through: 16
2023-09-25 16:03:17 +02:00
Alvaro Herrera 2e3dc8c148
pg_upgrade: check for types removed in pg12
Commit cda6a8d01d removed a few datatypes, but didn't update
pg_upgrade --check to throw error if these types are used.  So the users
find that pg_upgrade --check tells them that everything is fine, only to
fail when the real upgrade is attempted.

Reviewed-by: Tristan Partin <tristan@neon.tech>
Reviewed-by: Suraj Kharage <suraj.kharage@enterprisedb.com>
Discussion: https://postgr.es/m/202309201654.ng4ksea25mti@alvherre.pgsql
2023-09-25 14:27:33 +02:00
Daniel Gustafsson cca97ce6a6 Allow dbname in pg_basebackup/pg_receivewal connstring
As physical replication work at the cluster level and not database
level, any dbname in the connection string is ignored. Proxies and
middleware used in connecting to the cluster might however need to
know the dbname in order to make the correct routing decision for
the connection.

With this the startup packet will include the dbname parameter.

Author: Jelte Fennema-Nio <me@jeltef.nl>
Reviewed-by: Tristen Raab <tristen.raab@highgo.ca>
Reviewed-by: Jim Jones <jim.jones@uni-muenster.de>
Discussion: https://postgr.es/m/CAGECzQTw-dZkVT_RELRzfWRzY714-VaTjoBATYfZq93R8C-auA@mail.gmail.com
2023-09-21 13:53:07 +02:00
David Rowley 5cfba1ad69 Fix vacuumdb to pass buffer-usage-limit with analyze-only mode
ae78cae3b added the --buffer-usage-limit to vacuumdb to allow it to
include the BUFFER_USAGE_LIMIT option in the VACUUM command.
Unfortunately, that commit forgot to adjust the code so the option was
added to the ANALYZE command when the -Z command line argument was
specified.

There were no issues with the -z command as that option just adds
ANALYZE to the VACUUM command.

In passing adjust the code to escape the --buffer-usage-limit option
before passing it to the server.  It seems nothing beyond a confusing
error message could become this lack of escaping as VACUUM cannot be
specified in a multi-command string.

Reported-by: Ryoga Yoshida
Author: Ryoga Yoshida, David Rowley
Discussion: https://postgr.es/m/08930c0b541700a5264e5fbf3a685f5a%40oss.nttdata.com
Backpatch-through: 16, where ae78cae3b was introduced.
2023-09-21 17:47:20 +12:00
Nathan Bossart 559bc17321 Remove open-coded binary heap in pg_dump_sort.c.
Thanks to commit 5af0263afd, binaryheap is available to frontend
code.  This commit replaces the open-coded heap implementation in
pg_dump_sort.c with a binaryheap, saving a few lines of code.

Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us
2023-09-19 19:18:34 -07:00
Michael Paquier cb943054f3 psql: Reset query buffer of \e, \ef and \ev on error
If any of these commands fail during editing or pre-processing, the
command stored in the query buffer would remain around without being
executed immediately as PSQL_CMD_ERROR is returned as status.  The next
command provided by the user would run it, likely causing failures as
this could include silently some of the contents generated automatically
for views or functions.

The problems would be different depending on the psql meta-command used:
- For \ev and \ef, some errors can happen in a predictable way while
doing an object lookup or while creating an object command.  A failure
while editing is equally problematic, but the class of failures
happening in the code path of do_edit() are unlikely.  The query reset
is kept in exec_command_ef_ev() as a query may be unchanged.
- For \e, error can happen while editing.

In both cases, the query buffer is reset on error for an incorrect file
number provided, whose value check is done before filling up the query
buffer.

This is a slight change of behavior compared to the past for some of the
predictable error patterns for \ev and \ef, so for now I have made the
choice to not backpatch this commit (argument particularly available for
v11 that's going to be EOL'd soon).  Perhaps this could be revisited
later depending on the feedback of this new behavior.

Author: Ryoga Yoshida, Michael Paquier
Reviewed-by: Aleksander Alekseev, Kyotaro Horiguchi
Discussion: https://postgr.es/m/01419622d84ef093fd4fe585520bf03c@oss.nttdata.com
2023-09-20 09:26:15 +09:00
Nathan Bossart 9bfd44bbde Convert pg_restore's ready_list to a priority queue.
Presently, parallel restores spend a lot of time sorting this list
so that we pick the largest items first.  With many tables, this
sorting can become a significant bottleneck.  There are a couple of
reports from the field about this, and it is easy to reproduce.

This commit improves the performance of parallel pg_restore with
many tables by converting its ready_list to a priority queue, i.e.,
a binary heap.  We will first try to run the highest priority item,
but if it cannot be chosen due to the lock heuristic, we'll do a
sequential scan through the heap nodes until we find one that is
runnable.  This means that we might end up picking an item with a
much lower priority.  However, we expect that we will typically be
able to pick one of the first few items, which should usually have
a relatively high priority.

Suggested-by: Tom Lane
Tested-by: Pierre Ducroquet
Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/3612876.1689443232%40sss.pgh.pa.us
2023-09-19 14:31:29 -07:00
Heikki Linnakangas 1f998863b0 Fix psql tab-completion for identifiers containing dollars.
Dollar ($) is a legit character for identifiers, except as the first
character, since commit 1bd22f55cf in version 7.4. Update the
tab-completion code accordingly.

Author: Mikhail Gribkov
Reviewed-by: Vik Fearing
Discussion: https://www.postgresql.org/message-id/CAMEv5_sTAvPvhye%2Bu4jkWDe5UGDiQ1ZkQomnKCboM08zDzOe%3Dg%40mail.gmail.com
2023-09-19 19:26:29 +03:00
Alvaro Herrera d726897c57
Fix psql's \? output for \watch
It was reported as misaligned by Kyotaro, but it also needed to be
turned into a single translatable phrase (like the one for \g is), as
reported by Yugo.

This is a new issue (commit f347ec76e2), so no backpatch is needed.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Author: Yugo NAGATA <nagata@sraoss.co.jp>
Discussion: https://postgr.es/m/20230907.142956.2038600444404289870.horikyota.ntt@gmail.com
2023-09-18 16:19:25 +02:00
Daniel Gustafsson b0ec61c9c2 Quote filenames in error messages
The majority of all filenames are quoted in user facing error and
log messages, but a few were still printed without quotes.  While
these filenames do not risk causing any ambiguity as their format
is strict, quote them anyways to be consistent across all logs.

Also concatenate a message to keep it one line to make it easier
to grep for in the code.

Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/080EEABE-6645-4A46-AB20-6285ADAC44FE@yesql.se
2023-09-14 11:17:33 +02:00
Michael Paquier 522a31ac87 Switch psql's TAP test for query cancellation to use IPC::Run::signal()
Previously, the test relied on a trick with a shell to retrieve the PID
of the psql session to be stopped with SIGINT, that was skipped on
Windows.  This commit changes the test to use IPC::Run::signal()
instead, which still does not work on Windows, but for a different
reason: SIGINT would stop the test before finishing.

This should allow the test to run on non-Windows platforms where PPID is
not supported (like NetBSD), spreading it a bit more across the
buildfarm.  And the logic of the test is simpler.

It is the first time in the tree that IPC::Run::signal() is used, so, as
a matter of safety (or just call that as me having cold feet), no
backpatch is done, at least for now.

Author: Yugo NAGATA
Reviewed-by: Fabien Coelho
Discussion: https://postgr.es/m/20230810125935.22c2922ea5250ba79358965b@sraoss.co.jp
2023-09-13 10:10:04 +09:00
Michael Paquier c53e288dba Skip psql's TAP test for query cancellation entirely on Windows
This changes 020_cancel.pl so as the test is entirely skipped on
Windows.  This test was already doing nothing under WIN32, except
initializing and starting a node without using it so this shaves a few
test cycles.

Author: Yugo NAGATA
Reviewed-by: Fabien Coelho
Discussion: https://postgr.es/m/20230810125935.22c2922ea5250ba79358965b@sraoss.co.jp
Backpatch-through: 15
2023-09-13 09:53:48 +09:00