Commit Graph

6563 Commits

Author SHA1 Message Date
Thomas Munro cf112c1220 Remove dead pread and pwrite replacement code.
pread() and pwrite() are in SUSv2, and all targeted Unix systems have
them.

Previously, we defined pg_pread and pg_pwrite to emulate these function
with lseek() on old Unixen.  The names with a pg_ prefix were a reminder
of a portability hazard: they might change the current file position.
That hazard is gone, so we can drop the prefixes.

Since the remaining replacement code is Windows-only, move it into
src/port/win32p{read,write}.c, and move the declarations into
src/include/port/win32_port.h.

No need for vestigial HAVE_PREAD, HAVE_PWRITE macros as they were only
used for declarations in port.h which have now moved into win32_port.h.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-08-05 09:49:21 +12:00
Thomas Munro 2b1f580ee2 Remove configure probes for symlink/readlink, and dead code.
symlink() and readlink() are in SUSv2 and all targeted Unix systems have
them.  We have partial emulation on Windows.  Code that raised runtime
errors on systems without it has been dead for years, so we can remove
that and also references to such systems in the documentation.

Define HAVE_READLINK and HAVE_SYMLINK macros on Unix.  Our Windows
replacement functions based on junction points can't be used for
relative paths or for non-directories, so the macros can be used to
check for full symlink support.  The places that deal with tablespaces
can just use symlink functions without checking the macros.  (If they
did check the macros, they'd need to provide an #else branch with a
runtime or compile time error, and it'd be dead code.)

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-08-05 09:22:56 +12:00
Thomas Munro bdb657edd6 Remove configure probe and related tests for getrlimit.
getrlimit() is in SUSv2 and all targeted systems have it.

Windows doesn't have it.  We could just use #ifndef WIN32, but for a
little more explanation about why we're making things conditional, let's
retain the HAVE_GETRLIMIT macro.  It's defined in port.h for Unix systems.

On systems that have it, it's not necessary to test for RLIMIT_CORE,
RLIMIT_STACK or RLIMIT_NOFILE macros, since SUSv2 requires those and all
targeted systems have them.  Also remove references to a pre-historic
alternative spelling of RLIMIT_NOFILE, and coding that seemed to believe
that Cygwin didn't have it.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-08-05 09:18:34 +12:00
Robert Haas 87e22f675f Revert recent changes to 002_pg_upgrade.pl.
The test is proving to be unreliable in the buildfarm, and we neither
agree on how best to fix it nor have time to do so before the upcoming
release. So for now, put things back to the way they were before commit
d498e052b4.

Discussion: http://postgr.es/m/3628089.1659640252@sss.pgh.pa.us
2022-08-04 15:26:07 -04:00
Michael Paquier 8b1ec7d295 Fix error reporting after ioctl() call with pg_upgrade --clone
errno was not reported correctly after attempting to clone a file,
leading to incorrect error reports.  While scanning through the code, I
have not noticed any similar mistakes.

Error introduced in 3a769d8.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20220731134135.GY15006@telsasoft.com
Backpatch-through: 12
2022-08-01 16:38:23 +09:00
Andrew Dunstan 7781f4e3e7 Add --schema and --exclude-schema options to vacuumdb.
These two new options can be used to either process all tables in
specific schemas or to skip processing all tables in specific
schemas.  This change also refactors the handling of invalid
combinations of command-line options to a new helper function.

Author: Gilles Darold
Reviewed-by: Justin Pryzby, Nathan Bossart and Michael Paquier.
Discussion: https://postgr.es/m/929fbf3c-24b8-d454-811f-1d5898ab3e91%40migops.com
2022-07-31 16:46:13 -04:00
Robert Haas 212bdc0cbc Revise test case added in 4374699639.
Instead of using command_ok() to run psql, use safe_psql(). wrasse
isn't happy, and it be because of failure to pass -X to the psql
invocation, which safe_psql() will do automatically.

Since safe_psql() returns standard output instead of writing it to
a file, this requires some changes to the incantation for running
'diff'.

Test against the 'regression' database rather than 'postgres' so
we test more than just one table. That also means we need to record
the horizons later, after the test does "VACUUM FULL pg_largeobject".

Add an ORDER BY clause to the horizon query for stability.

Patch by me, reviewed by Tom Lane.

Discussion: http://postgr.es/m/CA+TgmoaGBbpzgu3=du1f9zDUbkfycO0y=_uWrLFy=KKEqXWeLQ@mail.gmail.com
2022-07-29 23:24:39 -04:00
Robert Haas 4374699639 Fix brown paper bag bug in bbe08b8869.
We must issue the TRUNCATE command first and update relfrozenxid
and relminmxid afterward; otherwise, TRUNCATE overwrites the
previously-set values.

Add a test case like I should have done the first time.

Per buildfarm report from TestUpgradeXversion.pm, by way of Tom
Lane.
2022-07-29 16:31:57 -04:00
Robert Haas 80d6907219 Fix mistake in bbe08b8869.
The earlier commit used pg_class.relfilenode where it should have
used pg_class.oid. This could lead to emitting an UPDATE statement
into the dump that would update nothing (or the wrong thing) when
executed in the new cluster, resulting in relfrozenxid and
relminmxid being improperly carried forward for pg_largeobject.

Noticed by Dilip Kumar.

Discussion: http://postgr.es/m/CAFiTN-ty1Gzs6stk2vt9BJiq0m0hzf=aPnh3a-4Z3Tk5GzoENw@mail.gmail.com
2022-07-29 11:20:07 -04:00
Robert Haas bbe08b8869 Use TRUNCATE to preserve relfilenode for pg_largeobject + index.
Commit 9a974cbcba arranged to preserve
the relfilenode of user tables across pg_upgrade, but failed to notice
that pg_upgrade treats pg_largeobject as a user table and thus it needs
the same treatment. Otherwise, large objects will appear to vanish
after a  pg_upgrade.

Commit d498e052b4 fixed this problem
by teaching pg_dump to UPDATE pg_class.relfilenode for pg_largeobject
and its index. However, because an UPDATE on the catalog rows doesn't
change anything on disk, this can leave stray files behind in the new
cluster. They will normally be empty, but it's a little bit untidy.

Hence, this commit arranges to do the same thing using DDL. Specifically,
it makes TRUNCATE work for the pg_largeobject catalog when in
binary-upgrade mode, and it then uses that command in binary-upgrade
dumps as a way of setting pg_class.relfilenode for pg_largeobject and
its index. That way, the old files are removed from the new cluster.

Discussion: http://postgr.es/m/CA+TgmoYYMXGUJO5GZk1-MByJGu_bB8CbOL6GJQC8=Bzt6x6vDg@mail.gmail.com
2022-07-28 16:03:42 -04:00
Michael Paquier 0a5f06b84d Fix a few issues with REINDEX grammar
This addresses a couple of bugs in the REINDEX grammar, introduced by
83011ce:
- A name was never specified for DATABASE/SYSTEM, even if the query
included one.  This caused such REINDEX queries to always work with any
object name, but we should complain if the object name specified does
not match the name of the database we are connected to.  A test is added
for this case in the main regression test suite, provided by Álvaro.
- REINDEX SYSTEM CONCURRENTLY [name] was getting rejected in the
parser.  Concurrent rebuilds are not supported for catalogs but the
error provided at execution time is more helpful for the user, and
allowing this flavor results in a simplification of the parsing logic.
- REINDEX DATABASE CONCURRENTLY was rebuilding the index in a
non-concurrent way, as the option was not being appended correctly in
the list of DefElems in ReindexStmt (REINDEX (CONCURRENTLY) DATABASE was
working fine.  A test is added in the TAP tests of reindexdb for this
case, where we already have a REINDEX DATABASE CONCURRENTLY query
running on a small-ish instance.  This relies on the work done in
2cbc3c1 for SYSTEM, but here we check if the OIDs of the index relations
match or not after the concurrent rebuild.  Note that in order to get
this part to work, I had to tweak the tests so as the index OID and
names are saved separately.  This change not affect the reliability or
of the coverage of the existing tests.

While on it, I have implemented a tweak in the grammar to reduce the
parsing by one branch, simplifying things even more.

Author: Michael Paquier, Álvaro Herrera
Discussion: https://postgr.es/m/YttqI6O64wDxGn0K@paquier.xyz
2022-07-26 10:16:26 +09:00
Andrew Dunstan a45388d6e0 Add xheader_width pset option to psql
The setting controls tha maximum length of the header line in expanded
format output. Possible settings are full, column, page, or an integer.
the default is full, the current behaviour, and in this case the header
line is the length of the widest line of output. column causes the
header to be truncated to the width of the first column, page causes it
to be truncated to the width of the terminal page, and an integer causes
it to be truncated to that value. If the full value is less than the
page or integer value no truncation occurs. If given without an argument
this option prints its current setting.

Platon Pronko, somewhat modified by me.

Discussion: https://postgr.es/m/f03d38a3-db96-a56e-d1bc-dbbc80bbde4d@gmail.com
2022-07-25 14:25:02 -04:00
Thomas Munro fb12becdfb Remove dead handling for pre-POSIX sigwait().
sigwait() is in SUSv2 and all targeted Unix systems have it.  An earlier
pre-standard function prototype existed on some older systems, but we
no longer need a workaround for that.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Greg Stark <stark@mit.edu>
Reviewed-by: Robert Haas <robertmhaas@gmail.com>
Discussion: https://postgr.es/m/CA+hUKGJ3LHeP9w5Fgzdr4G8AnEtJ=z=p6hGDEm4qYGEUX5B6fQ@mail.gmail.com
2022-07-24 09:32:34 +12:00
Tom Lane 90474c16a7 Fix minor memory leaks in psql's tab completion.
Tang Haiying and Tom Lane

Discussion: https://postgr.es/m/OS0PR01MB6113EA19F05E217C823B4CCAFB909@OS0PR01MB6113.jpnprd01.prod.outlook.com
2022-07-22 10:53:26 -04:00
Thomas Munro 5344723755 Remove unnecessary Windows-specific basebackup code.
Commit c6f2f016 added an explicit check for a Windows "junction point".
That turned out to be needed only because get_dirent_type() was busted
on Windows.  It's been fixed by commit 9d3444dc, so remove it.

Add a TAP-test to demonstrate that in-place tablespaces are copied by
pg_basebackup.  This exercises the codepath that would fail before
c6f2f016 on Windows, and shows that it still doesn't fail now that we're
using get_dirent_type() on both Windows and Unix.

Back-patch to 15, where in-place tablespaces arrived and caused this
problem (ie directories where previously only symlinks were expected).

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGLzLK4PUPx0_AwXEWXOYAejU%3D7XpxnYE55Y%2Be7hB2N3FA%40mail.gmail.com
2022-07-22 17:41:47 +12:00
Thomas Munro a1b56090eb Remove O_FSYNC and associated macros.
O_FSYNC was a pre-POSIX way of spelling O_SYNC, supported since commit
9d645fd84c for non-conforming operating systems of the time.  It's not
needed on any modern system.  We can just use standard O_SYNC directly
if it exists (= all targeted systems except Windows), and get rid of our
OPEN_SYNC_FLAG macro.

Similarly for standard O_DSYNC, we can just use that directly if it
exists (= all targeted systems except DragonFlyBSD), and get rid of our
OPEN_DATASYNC_FLAG macro.

We still avoid choosing open_datasync as a default value for
wal_sync_method if O_DSYNC has the same value as O_SYNC (= only
OpenBSD), so there is no change in default behavior.

Discussion: https://postgr.es/m/CA%2BhUKGJE7y92NY7FG2ftUbZUaqohBU65_Ys_7xF5mUHo4wirTQ%40mail.gmail.com
2022-07-22 12:41:17 +12:00
Amit Kapila 366283961a Allow users to skip logical replication of data having origin.
This patch adds a new SUBSCRIPTION parameter "origin". It specifies
whether the subscription will request the publisher to only send changes
that don't have an origin or send changes regardless of origin. Setting it
to "none" means that the subscription will request the publisher to only
send changes that have no origin associated. Setting it to "any" means
that the publisher sends changes regardless of their origin. The default
is "any".
Usage:
CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=postgres port=9999'
PUBLICATION pub1 WITH (origin = none);

This can be used to avoid loops (infinite replication of the same data)
among replication nodes.

This feature allows filtering only the replication data originating from
WAL but for initial sync (initial copy of table data) we don't have such a
facility as we can only distinguish the data based on origin from WAL. As
a follow-up patch, we are planning to forbid the initial sync if the
origin is specified as none and we notice that the publication tables were
also replicated from other publishers to avoid duplicate data or loops.

We forbid to allow creating origin with names 'none' and 'any' to avoid
confusion with the same name options.

Author: Vignesh C, Amit Kapila
Reviewed-By: Peter Smith, Amit Kapila, Dilip Kumar, Shi yu, Ashutosh Bapat, Hayato Kuroda
Discussion: https://postgr.es/m/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
2022-07-21 08:47:38 +05:30
Michael Paquier 2b99ce10c2 Tweak a bit the new TAP tests of REINDEX DATABASE/SYSTEM
This renames the relation storing the relfilenode state into something
more generic as it also stores data for non-toast relations.  A
restriction on the number of digits used for the OID number when
filtering toast relation names is removed, while on it, as there is no
need for it.

Reported-by: Justin Pryzby
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/20220719022652.GE12702@telsasoft.com
2022-07-21 11:00:48 +09:00
Michael Paquier 171ab95f64 Fix various memory leaks in psql's describe commands \d*
Most of these have been introduced in d2d3547 with the new pattern
validation logic, and would leak memory worth an amount of one
PQExpBuffer each time (as of 256 bytes at minimum, possibly more).

Most of the patch has been written by Tang Haiying, with a few tweaks
coming from Álvaro Herrera.

Reported-by: Tang Haiying
Author: Tang Haiying, Álvaro Herrera
Reviewed-by: Mark Dilger, Andres Freund, Álvaro Herrera, Tom Lane, Japin
Li, Michael Paquier, Junwang Zhao
Backpatch-through: 15
2022-07-21 10:41:33 +09:00
Michael Paquier 2cbc3c17a5 Rework logic and simplify syntax of REINDEX DATABASE/SYSTEM
Per discussion, this commit includes a couple of changes to these two
flavors of REINDEX:
* The grammar is changed to make the name of the object optional, hence
one can rebuild all the indexes of the wanted area by specifying only
"REINDEX DATABASE;" or "REINDEX SYSTEM;".  Previously, the object name
was mandatory and had to match the name of the database on which the
command is issued.
* REINDEX DATABASE is changed to ignore catalogs, making this task only
possible with REINDEX SYSTEM.  This is a historical change, but there
was no way to work only on the indexes of a database without touching
the catalogs.  We have discussed more approaches here, like the addition
of an option to skip the catalogs without changing the original
behavior, but concluded that what we have here is for the best.

This builds on top of the TAP tests introduced in 5fb5b6c, showing the
change in behavior for REINDEX SYSTEM.  reindexdb is updated so as we do
not issue an extra REINDEX SYSTEM when working on a database in the
non-concurrent case, something that was confusing when --concurrently
got introduced, so this simplifies the code.

Author: Simon Riggs
Reviewed-by: Ashutosh Bapat, Bernd Helmle, Álvaro Herrera, Cary Huang,
Michael Paquier
Discussion: https://postgr.es/m/CANbhV-H=NH6Om4-X6cRjDWfH_Mu1usqwkuYVp-hwdB_PSHWRfg@mail.gmail.com
2022-07-19 11:45:06 +09:00
Michael Paquier 5fb5b6c4c1 Add more tests for REINDEX DATABASE/SYSTEM with relfilenode changes
Adding such commands in the main regression test suite is not a good
approach performance-wise as it impacts all the objects in the
regression database, so this additional coverage is added in the TAP
tests of reindexdb where we already run a few REINDEX commands with
SYSTEM and DATABASE so there is no runtime difference for the test.
This additional coverage checks which relations are rewritten with
relfilenode changes, as of:
- a toast index in user table.
- a toast index in catalog table.
- a catalog index.
- a user index.

This test suite is something I have implemented for a separate patch
that reworks a bit the way we handle these two REINDEX behaviors, but it
has enough value in itself to be in a separate commit.  This also makes
easier to follow what actually changes once the REINDEX logic is
reworked (currently, DABATASE rewrites both catalog and user tables, and
SYSTEM works only on catalogs).

Discussion: https://postgr.es/m/YtOqA7ldcJQADEE8@paquier.xyz
2022-07-19 10:51:27 +09:00
Andres Freund 7c3c2cb9ae psql: Output dir and dependency generation for sql_help
This is in preparation for building postgres with meson / ninja.

When building with meson, commands are run at the root of the build tree. Add
an option to put build output into the appropriate place. This can be utilized
by src/tools/msvc/ for a minor simplification, which also provides some
coverage for the new option.

To deal with dependencies to the variable set of input files to this script,
add an option to generate a dependency file (which meson / ninja can consume).

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/5e216522-ba3c-f0e6-7f97-5276d0270029@enterprisedb.com
2022-07-18 12:24:12 -07:00
Peter Eisentraut 973137de08 pg_upgrade: Adjust quoting style in message to match guidelines 2022-07-18 14:53:00 +02:00
Peter Eisentraut 9fd45870c1 Replace many MemSet calls with struct initialization
This replaces all MemSet() calls with struct initialization where that
is easily and obviously possible.  (For example, some cases have to
worry about padding bits, so I left those.)

(The same could be done with appropriate memset() calls, but this
patch is part of an effort to phase out MemSet(), so it doesn't touch
memset() calls.)

Reviewed-by: Ranier Vilela <ranier.vf@gmail.com>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/9847b13c-b785-f4e2-75c3-12ec77a3b05c@enterprisedb.com
2022-07-16 08:50:49 +02:00
John Naylor 82785effc0 Correct some uses of e.g. and i.e. in message strings and documentation
E.g. means "for example" and i.e. means "that is". Fix a couple uses
that don't match the intended meaning.

Kyotaro Horiguchi

Reviewed by Junwang Zhao and Aleksander Alekseev, with one addition by me
Discussion: https://www.postgresql.org/message-id/flat/20220713.180943.589079824955875739.horikyota.ntt%40gmail.com
2022-07-14 09:38:06 +07:00
Michael Paquier 4ca9985957 Fix output of createuser --help with --valid-until
The argument required by --valid-until, a timestamp string, was missing
in the description of --help.

Author: Shinoda, Noriyoshi
Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/DM4PR84MB1734A6CE3839A68B59BEA599EE899@DM4PR84MB1734.NAMPRD84.PROD.OUTLOOK.COM
2022-07-14 08:32:18 +09:00
Tom Lane a5f9f1b885 Mop up pg_upgrade's nls.mk for commit b0a55e432.
We'll do this the hard way for today.

Discussion: https://postgr.es/m/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
2022-07-13 14:33:24 -04:00
Tom Lane 54ba2f0623 Revert "Use wildcards instead of manually-maintained file lists in */nls.mk."
This reverts commit 617d691412.
While I still think the basic idea is attractive, we need to sort
out what happens with built .c files, and there also seem to be
VPATH issues.
2022-07-13 14:29:10 -04:00
Tom Lane 617d691412 Use wildcards instead of manually-maintained file lists in */nls.mk.
The backend already used a mechanically-generated list of *.c files,
but everywhere else we had a manually-written-out list of files in
which to seek translatable messages.  Commit b0a55e432 contains the
latest in a long line of failures to update those lists.  Rather than
manually fix its oversight, let's change to using "$(wildcard *.c)"
in all these nls.mk files.

Many of these files also have manual references to some *.c files
in other directories, most often src/common/.  Perhaps we should try
to improve that situation too; but it's a bit less clear how, so for
now just fix the local file references.

Kyotaro Horiguchi and Tom Lane

Discussion: https://postgr.es/m/20220713.160853.453362706160476128.horikyota.ntt@gmail.com
2022-07-13 12:56:42 -04:00
Peter Eisentraut 88dad06b47 NLS: Put list of available languages into LINGUAS files
This moves the list of available languages from nls.mk into a separate
file called po/LINGUAS.  Advantages:

- It keeps the parts notionally managed by programmers (nls.mk)
  separate from the parts notionally managed by translators (LINGUAS).

- It's the standard practice recommended by the Gettext manual
  nowadays.

- The Meson build system also supports this layout (and of course
  doesn't know anything about our custom nls.mk), so this would enable
  sharing the list of languages between the two build systems.

(The MSVC build system currently finds all po files by globbing, so it
is not affected by this change.)

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/557a9f5c-e871-edc7-2f58-a4140fb65b7b@enterprisedb.com
2022-07-13 08:19:17 +02:00
Michael Paquier 08951a7c93 createuser: Add support for more clause types through new options
The following options are added to createuser:
* --valid-until to generate a VALID UNTIL clause for the role created.
* --bypassrls/--no-bypassrls for BYPASSRLS/NOBYPASSRLS.
* -m/--member to make the new role a member of an existing role, with an
extra ROLE clause generated.  The clause generated overlaps with
-g/--role, but per discussion this was the most popular choice as option
name.
* -a/--admin for the addition of an ADMIN clause.

These option names are chosen to be completely new, so as they do not
impact anybody relying on the existing option set.  Tests are added for
the new options and extended a bit, while on it, to cover more patterns
where quotes are added to various elements of the query generated.

Author: Shinya Kato
Reviewed-by: Nathan Bossart, Daniel Gustafsson, Robert Haas, Kyotaro
Horiguchi, David G. Johnston, Przemysław Sztoch
Discussion: https://postgr.es/m/69a9851035cf0f0477bcc5d742b031a3@oss.nttdata.com
2022-07-13 12:21:20 +09:00
Michael Paquier 50e4c280f0 createuser: Cleanup and fix internal option ordering
This utility supports 23 options that are not really ordered in the
code, making the addition of new things more complicated than necessary.
This cleanup is in preparation for a patch to add even more options.

Discussion: https://postgr.es/m/69a9851035cf0f0477bcc5d742b031a3@oss.nttdata.com
2022-07-13 11:29:02 +09:00
Tom Lane 920072339f Improve error reporting from validate_exec().
validate_exec() didn't guarantee to set errno to something appropriate
after a failure, leading to callers not being able to print an on-point
message.  Improve that.

Noted by Kyotaro Horiguchi, though this isn't exactly his proposal.

Discussion: https://postgr.es/m/20220615.131403.1791191615590453058.horikyota.ntt@gmail.com
2022-07-12 15:37:39 -04:00
Tom Lane 7652353d87 Remove trailing newlines in pg_upgrade's message strings.
pg_upgrade does not use common/logging.c, which is unfortunate
but changing it to do so seems like more work than is justified.
However, we really need to make it work more like common/logging.c
in one respect: the latter expects supplied message strings to not
end with a newline, instead adding one internally.  As it stands,
pg_upgrade's logging facilities expect a caller-supplied newline
in some cases and not others, which is already an invitation to bugs,
but the inconsistency with our other frontend code makes it worse.
There are already several places with missing or extra newlines,
and it's inevitable that there won't be more if we let this stand.

Hence, run around and get rid of all trailing newlines in message
strings, and add an Assert that there's not one, similar to the
existing Assert in common/logging.c.  Adjust the logging functions
to supply a newline at the right places.

(Some of these strings also have a *leading* newline, which would
be a good thing to get rid of too; but this patch doesn't attempt
that.)

There are some consequent minor changes in output.  The ones that
aren't outright bug fixes are generally removal of extra blank
lines that the original coding intentionally inserted.  It didn't
seem worth being bug-compatible with that.

Patch by me, reviewed by Kyotaro Horiguchi and Peter Eisentraut

Discussion: https://postgr.es/m/113191.1655233060@sss.pgh.pa.us
2022-07-12 15:17:44 -04:00
Peter Eisentraut 2cd2569c72 Convert macros to static inline functions (bufpage.h)
Remove PageIsValid() and PageSizeIsValid(), which weren't used and
seem unnecessary.

Some code using these formerly-macros needs some adjustments because
it was previously playing loose with the Page vs. PageHeader types,
which is no longer possible with the functions instead of macros.

Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-07-11 07:21:52 +02:00
Thomas Munro 21267b6477 Fix \watch's interaction with libedit on ^C.
When you hit ^C, the terminal driver in Unix-like systems echoes "^C" as
well as sending an interrupt signal (depending on stty settings).  At
least libedit (but maybe also libreadline) is then confused about the
current cursor location, and corrupts the display if you try to scroll
back.  Fix, by moving to a new line before the next prompt is displayed.

Back-patch to all supported released.

Author: Pavel Stehule <pavel.stehule@gmail.com>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/3278793.1626198638%40sss.pgh.pa.us
2022-07-10 16:52:38 +12:00
Robert Haas d498e052b4 Preserve relfilenode of pg_largeobject and its index across pg_upgrade.
Commit 9a974cbcba did this for user
tables, but pg_upgrade treats pg_largeobject as a user table, and so
needs the same treatment. Without this fix, if you rewrite the
pg_largeobject table and then perform an upgrade with pg_upgrade, the
table will apparently be empty on the new cluster, while all of your
objects will end up with an orphaned file.

With this fix, instead of the old cluster's pg_largeobject files ending
up orphaned, the original files fro the new cluster do. That's mostly
harmless because we expect the table to be empty, but we might want
to arrange to remove the as part of the upgrade. Since we're still
debating the best way of doing that, I (rhaas) have decided to postpone
dealing with that problem and get the basic fix committed.

Justin Pryzby, reviewed by Shruthi Gowda and by me.

Discussion: http://postgr.es/m/20220701231413.GI13040@telsasoft.com
2022-07-08 10:20:27 -04:00
Michael Paquier 495ed0ef2d Make Windows 10 the minimal runtime requirement for WIN32
This commit bumps the runtime value of _WIN32_WINNT to be 0x0A00 for any
builds on Windows.  Hence, this makes Windows 10 the minimal requirement
when running PostgreSQL under WIN32, be it for builds of Cygwin, MinGW
or Visual Studio.

The previous minimal runtime version was either Windows Vista when
building with at least Visual Studio 2015 or Windows XP for the rest.

Windows 10 is the most modern version supported by Microsoft, and per
discussion, as we don't have buildfarm members that run older versions
anymore, this is the minimal supported version that suits better for our
needs.  This will actually make easier the development of some patches,
two being async I/O and large page handling by avoiding a lot of
compatibility gotchas, on platforms that have most likely few users
anyway.

It is possible to remove MIN_WINNT in win32.h and the macros
IsWindowsXXXOrGreater() that were used in the code at runtime to check
which version of Windows was getting used.  The change in pg_locale.c
comes from Juan.  Note that all my tests passed, and that the CI is
green.  The buildfarm will quickly tell if this needs more adjustments.

Author: Michael Paquier, Juan José Santamaría Flecha
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/Yo7tHKD8VCkeNi71@paquier.xyz
2022-07-07 13:25:45 +09:00
Amit Kapila d547f7cf5e Re-order disable_on_error in tab-complete.
By convention, the tab-complete subscription parameters are listed in the
COMPLETE_WITH lists in alphabetical order, but when the "disable_on_error"
parameter was introduced this was not done.

This patch just tidies that up.

Reported-by: Peter Smith
Author: Peter Smith
Reviewed-by: Euler Taveira, Takamichi Osumi
Backpatch-through: 15, where it was introduced
Discussion: https://postgr.es/m/CAHut+PucvKZgg_eJzUW--iL6DXHg1Jwj6F09tQziE3kUF67uLg@mail.gmail.com
2022-07-07 09:00:27 +05:30
Michael Paquier 1409eade9f Clean up some includes and comments in TAP test scripts
A few tests included File::Path::rmtree without using it, and a comment
related to the segment size for replication slot limits was wrong.

Author: Pavel Borisov, Bharath Rupireddy
Reviewed-by: Maxim Orlov
Discussion: https://postgr.es/m/CALj2ACU4-aNLX=DrUM8F7QDwynJKzYRiqOj_33NhnGbhDs5-kQ@mail.gmail.com
2022-07-07 10:13:01 +09:00
Robert Haas b0a55e4329 Change internal RelFileNode references to RelFileNumber or RelFileLocator.
We have been using the term RelFileNode to refer to either (1) the
integer that is used to name the sequence of files for a certain relation
within the directory set aside for that tablespace/database combination;
or (2) that value plus the OIDs of the tablespace and database; or
occasionally (3) the whole series of files created for a relation
based on those values. Using the same name for more than one thing is
confusing.

Replace RelFileNode with RelFileNumber when we're talking about just the
single number, i.e. (1) from above, and with RelFileLocator when we're
talking about all the things that are needed to locate a relation's files
on disk, i.e. (2) from above. In the places where we refer to (3) as
a relfilenode, instead refer to "relation storage".

Since there is a ton of SQL code in the world that knows about
pg_class.relfilenode, don't change the name of that column, or of other
SQL-facing things that derive their name from it.

On the other hand, do adjust closely-related internal terminology. For
example, the structure member names dbNode and spcNode appear to be
derived from the fact that the structure itself was called RelFileNode,
so change those to dbOid and spcOid. Likewise, various variables with
names like rnode and relnode get renamed appropriately, according to
how they're being used in context.

Hopefully, this is clearer than before. It is also preparation for
future patches that intend to widen the relfilenumber fields from its
current width of 32 bits. Variables that store a relfilenumber are now
declared as type RelFileNumber rather than type Oid; right now, these
are the same, but that can now more easily be changed.

Dilip Kumar, per an idea from me. Reviewed also by Andres Freund.
I fixed some whitespace issues, changed a couple of words in a
comment, and made one other minor correction.

Discussion: http://postgr.es/m/CA+TgmoamOtXbVAQf9hWFzonUo6bhhjS6toZQd7HZ-pmojtAmag@mail.gmail.com
Discussion: http://postgr.es/m/CA+Tgmobp7+7kmi4gkq7Y+4AM9fTvL+O1oQ4-5gFTT+6Ng-dQ=g@mail.gmail.com
Discussion: http://postgr.es/m/CAFiTN-vTe79M8uDH1yprOU64MNFE+R3ODRuA+JWf27JbhY4hJw@mail.gmail.com
2022-07-06 11:39:09 -04:00
Tom Lane 08385ed261 Tighten pg_upgrade's new check for non-upgradable anyarray usages.
We only need to reject cases when the aggregate or operator is
itself declared with a polymorphic type.  Per buildfarm.

Discussion: https://postgr.es/m/3383880.QJadu78ljV@vejsadalnx
2022-07-05 18:23:33 -04:00
Andres Freund 3f8148c256 Revert 019_replslot_limit.pl related debugging aids.
This reverts most of 91c0570a79, f28bf667f6, fe0972ee5e, afdeff1052. The
only thing left is the retry loop in 019_replslot_limit.pl that avoids
spurious failures by retrying a couple times.

We haven't seen any hard evidence that this is caused by anything but slow
process shutdown. We did not find any cases where walsenders did not vanish
after waiting for longer. Therefore there's no reason for this debugging code
to remain.

Discussion: https://postgr.es/m/20220530190155.47wr3x2prdwyciah@alap3.anarazel.de
Backpatch: 15-
2022-07-05 11:01:10 -07:00
Tom Lane 09878cdd48 Fix pg_upgrade to detect non-upgradable anyarray usages.
When we changed some built-in functions to use anycompatiblearray
instead of anyarray, we created a dump/restore hazard for user-defined
operators and aggregates relying on those functions: the user objects
have to be modified to change their signatures similarly.  This causes
pg_upgrade to fail partway through if the source installation contains
such objects.  We generally try to have pg_upgrade detect such hazards
and fail before it does anything exciting, so add logic to detect
this case too.

Back-patch to v14 where the change was made.

Justin Pryzby, reviewed by Andrey Borodin

Discussion: https://postgr.es/m/3383880.QJadu78ljV@vejsadalnx
2022-07-05 13:06:31 -04:00
Tom Lane 255625df1d Use a short socket directory path in pg_upgrade testing.
Several buildfarm members are failing the pg_upgrade test in
REL_15_STABLE, though the identical test is fine in HEAD.
On thorntail it's possible to see that the problem is an
overlength socket path name, and I bet the same is true
on the others.

The normally-started postmasters used in the test are already
set up with short socket directory paths, but we neglected to
tell pg_upgrade itself to do likewise when starting child
postmasters, and indeed it seems to be explicitly selecting
the test directory instead.

Back-patch to v15 where the current test script was introduced.
(The previous script might have the same issue, because I don't
see any use of -s/--socketdir in it either; but we've had no
complaints, so leave it alone for now.)

Discussion: https://postgr.es/m/1410025.1656890531@sss.pgh.pa.us
2022-07-03 21:38:32 -04:00
Tom Lane 22a67fdd5d Simplify tab completion of extension versions, redux.
After commit 662dbe2c8, psql tab completion didn't conveniently
support the case of "ALTER EXTENSION foo UPDATE".  It'd always
add "TO", which is fine if you want to specify a target version
but not if you don't ... and surely the latter is the much more
common case.

To fix, remove "TO" from the initially offered completion; you now
need to press TAB one additional time to get that.  We won't try to
duplicate the old behavior of attempting initial completion on the
target version along with TO.  It's too squirrelly to get the quoting
right, and this is such an infrequent usage that it doesn't seem worth
expending a lot of effort and special code on.

Noted by Noah Misch.  Back-patch to v15.

Discussion: https://postgr.es/m/20220703083217.GB2476530@rfd.leadboat.com
2022-07-03 15:27:27 -04:00
Peter Eisentraut 5faef9d582 Remove redundant null pointer checks before PQclear and PQconninfoFree
These functions already had the free()-like behavior of handling null
pointers as a no-op.  But it wasn't documented, so add it explicitly
to the documentation, too.

Discussion: https://www.postgresql.org/message-id/flat/dac5d2d0-98f5-94d9-8e69-46da2413593d%40enterprisedb.com
2022-07-03 20:11:05 +02:00
Peter Eisentraut 02c408e21a Remove redundant null pointer checks before free()
Per applicable standards, free() with a null pointer is a no-op.
Systems that don't observe that are ancient and no longer relevant.
Some PostgreSQL code already required this behavior, so this change
does not introduce any new requirements, just makes the code more
consistent.

Discussion: https://www.postgresql.org/message-id/flat/dac5d2d0-98f5-94d9-8e69-46da2413593d%40enterprisedb.com
2022-07-03 11:47:15 +02:00
Peter Eisentraut 098c703d30 Remove redundant null pointer checks before pg_free()
These are especially useless because the whole point of pg_free() was
to do that very check before calling free().

pg_free() could be removed altogether, but I'm keeping it here to keep
the API consistent.

Discussion: https://www.postgresql.org/message-id/flat/dac5d2d0-98f5-94d9-8e69-46da2413593d%40enterprisedb.com
2022-07-03 09:05:34 +02:00
Thomas Munro 94ebf8117c Default to dynamic_shared_memory_type=sysv on Solaris.
POSIX shm_open() can sleep for a long time and fail spuriously because
of contention on an internal lock file on Solaris (and presumably
illumos).  Commit 389869af fixed the main problem with this, namely that
we could crash, but it's now clear that "posix" is not a good default.

Therefore, choose "sysv" at initdb time on Solaris and illumos.  Other
choices are still available by editing the postgresql.conf file.

Back-patch only to 15, because contention is much less likely further
back, and it doesn't seem like a good idea to change this in released
branches.  This should clear up the failures on build farm animal
margay.

Discussion: https://postgr.es/m/CA%2BhUKGKqKrCV5xKWfh9rnm%3Do%3DDwZLTLtnsj_XpUi9g5%3DV%2B9oyg%40mail.gmail.com
2022-07-02 16:23:39 +12:00