Commit Graph

40005 Commits

Author SHA1 Message Date
Peter Eisentraut 357cfefb09 Use C library functions instead of Abs() for int64
Instead of Abs() for int64, use the C standard functions labs() or
llabs() as appropriate.  Define a small wrapper around them that
matches our definition of int64.  (labs() is C90, llabs() is C99.)

Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/4beb42b5-216b-bce8-d452-d924d5794c63%40enterprisedb.com
2022-10-10 09:01:17 +02:00
Andres Freund 06dbd619bf pgstat: Prevent stats reset from corrupting slotname by removing slotname
Previously PgStat_StatReplSlotEntry contained the slotname, which was mainly
used when writing out the stats during shutdown, to identify the slot in the
serialized data (at runtime the index in ReplicationSlotCtl->replication_slots
is used, but that can change during a restart). Unfortunately the slotname was
overwritten when the slot's stats were reset.

That turned out to only cause "real" problems if the slot was active during
the reset, triggering an assertion failure at the next
pgstat_report_replslot(). In other paths the stats were re-initialized during
pgstat_acquire_replslot().

Fix this by removing slotname from PgStat_StatReplSlotEntry. Instead we can
get the slot's name from the slot itself. Besides fixing a bug, this also is
architecturally cleaner (a name is not really statistics). This is safe
because stats, for a slot removed while shut down, will not be restored at
startup.

In 15 the slotname is not removed, but renamed, to avoid changing the stats
format. In master, bump PGSTAT_FILE_FORMAT_ID.

This commit does not contain a test for the fix. I think this can only be
tested by a tap test starting pg_recvlogical in the background and checking
pg_recvlogical's output. That type of test is notoriously hard to be reliable,
so committing it shortly before the release is wrapped seems like a bad idea.

Reported-by: Jaime Casanova <jcasanov@systemguards.com.ec>
Author: Andres Freund <andres@anarazel.de>
Reviewed-by: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/YxfagaTXUNa9ggLb@ahch-to
Backpatch: 15-, where the bug was introduced in 5891c7a8ed
2022-10-08 09:43:29 -07:00
Peter Eisentraut e4c61bedcb Use fabsf() instead of Abs() or fabs() where appropriate
This function is new in C99.

Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/4beb42b5-216b-bce8-d452-d924d5794c63%40enterprisedb.com
2022-10-08 13:43:26 +02:00
Andres Freund 2473cb9ff3 autoconf: Rely on ar supporting index creation
This way we don't need RANLIB anymore, making it a bit simpler for the meson
build to generate Makefile.global for PGXS compatibility.

FreeBSD, NetBSD, OpenBSD, the only platforms where we didn't use AROPT=crs,
all have supported the 's' option for a long time.

On macOS we ran ranlib after installing a static library. This was added a
long time ago, in 58ad65ec2d. I cannot reproduce an issue in more recent
macOS versions. This is removed now.

Based on discussion with Tom, I left the 'touch' at the end of static
libraries generation, added in 826eff57c4, in place. While it looks like
current versions of Apple's ar/ranlib don't need it, it was needed not too
long ago.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20221005200710.luvw5evhwf6clig6@awork3.anarazel.de
2022-10-07 11:53:39 -07:00
Alvaro Herrera 614a406b4f
Fix self-referencing foreign keys with partitioned tables
There are a number of bugs in this area.  Two of them are fixed here,
namely:
1. get_relation_idx_constraint_oid does not restrict the type of
   constraint that's returned, so with sufficient bad luck it can
   return the OID of a foreign key constraint.  This has the effect that
   a primary key in a partition can end up as a child of a foreign key,
   which makes no sense (it needs to be the child of the equivalent
   primary key.)
   Change the API contract so that only index-backed constraints are
   returned, mimicking get_constraint_index().

2. Both CloneFkReferenced and CloneFkReferencing clone a
   self-referencing foreign key, so the partition ends up with
   a duplicate foreign key.  Change the former function to ignore such
   constraints.

Add some tests to verify that things are better now.  (However, these
new tests show some additional misbehavior that will be fixed later --
namely that there's a constraint marked NOT VALID.)

Backpatch to 12, where these constraints are possible at all.

Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Discussion: https://postgr.es/m/20220603154232.1715b14c@karst
2022-10-07 19:37:48 +02:00
Peter Eisentraut 3edc71ec04 Convert macros to static inline functions (rel.h)
Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-10-07 16:16:50 +02:00
Peter Eisentraut f14aad5169 Remove unnecessary uses of Abs()
Use C standard abs() or fabs() instead.

Reviewed-by: Zhang Mingli <zmlpostgres@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/4beb42b5-216b-bce8-d452-d924d5794c63%40enterprisedb.com
2022-10-07 13:29:33 +02:00
Tom Lane 80ef926758 Improve our ability to detect bogus pointers passed to pfree et al.
Commit c6e0fe1f2 was a shade too trusting that any pointer passed
to pfree, repalloc, etc will point at a valid chunk.  Notably,
passing a pointer that was actually obtained from malloc tended
to result in obscure assertion failures, if not worse.  (On FreeBSD
I've seen such mistakes take down the entire cluster, seemingly as
a result of clobbering shared memory.)

To improve matters, extend the mcxt_methods[] array so that it
has entries for every possible MemoryContextMethodID bit-pattern,
with the currently unassigned ID codes pointing to error-reporting
functions.  Then, fiddle with the ID assignments so that patterns
likely to be associated with bad pointers aren't valid ID codes.
In particular, we should avoid assigning bit patterns 000 (zeroed
memory) and 111 (wipe_mem'd memory).

It turns out that on glibc (Linux), malloc uses chunk headers that
have flag bits in the same place we keep MemoryContextMethodID,
and that the bit patterns 000, 001, 010 are the only ones we'll
see as long as the backend isn't threaded.  So we can have very
robust detection of pfree'ing a malloc-assigned block on that
platform, at least so long as we can refrain from using up those
ID codes.  On other platforms, we don't have such a good guarantee,
but keeping 000 reserved will be enough to catch many such cases.

While here, make GetMemoryChunkMethodID() local to mcxt.c, as there
seems no need for it to be exposed even in memutils_internal.h.

Patch by me, with suggestions from Andres Freund and David Rowley.

Discussion: https://postgr.es/m/2910981.1665080361@sss.pgh.pa.us
2022-10-06 21:24:00 -04:00
Andres Freund e5555657ba meson: Add support for building with precompiled headers
This substantially speeds up building for windows, due to the vast amount of
headers included via windows.h. A cross build from linux targetting mingw goes
from

994.11user 136.43system 0:31.58elapsed 3579%CPU
to
422.41user 89.05system 0:14.35elapsed 3562%CPU

The wins on windows are similar-ish (but I don't have a system at hand just
now for actual numbers). Targetting other operating systems the wins are far
smaller (tested linux, macOS, FreeBSD).

For now precompiled headers are disabled by default, it's not clear how well
they work on all platforms. E.g. on FreeBSD gcc doesn't seem to have working
support, but clang does.

When doing a full build precompiled headers are only beneficial for targets
with multiple .c files, as meson builds a separate precompiled header for each
target (so that different compilation options take effect). This commit
therefore only changes target with at least two .c files to use precompiled
headers.

Because this commit adds b_pch=false to the default_options new build
directories will have precompiled headers disabled by default, however
existing build directories will continue use the default value of b_pch, which
is true.

Note that using precompiled headers with ccache requires setting
CCACHE_SLOPPINESS=pch_defines,time_macros to get hits.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
Discussion: https://postgr.es/m/c5736f70-bb6d-8d25-e35c-e3d886e4e905@enterprisedb.com
Discussion: https://postgr.es/m/20190826054000.GE7005%40paquier.xyz
2022-10-06 17:19:30 -07:00
Andres Freund e0b0142959 Create subscription stats entry at CREATE SUBSCRIPTION time
Previously, the subscription stats entry was created when the first
stats, i.e., an error on apply worker or tablesync worker,  were
reported. Therefore, the stats_reset field was not updated by
pg_stat_reset_subscription_stats() if the stats entry was not
populated yet, which was different behavior than other statistics.

This change creates the subscription stats entry and initializes it at
CREATE SUBSCRIPTION time.

Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Discussion: https://postgr.es/m/CAAKRu_Zqd-e5imT_3-ZiQv1cfsWuy16OJTiUaCvqpq4V7GVdSg@mail.gmail.com
2022-10-06 17:17:16 -07:00
David Rowley cd4e8caaa0 Fix final warnings produced by -Wshadow=compatible-local
I thought I had these in d8df67bb1, but per report from Andres Freund, I
missed some.

Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/20221005214052.c4tkudawyp5wxt3c@awork3.anarazel.de
2022-10-07 13:13:27 +13:00
Andres Freund 4289263cf2 windows: Adjust FD_SETSIZE via commandline define
When using precompiled headers, we cannot pre-define macros for the system
headers from within .c files, as headers are already processed before
the #define in the C file is reached. But we can pre-define using
-DFD_SETSIZE, as long as that's also used when building the precompiled header.

A few files #define FD_SETSIZE 1024 on windows, as the default is only 64. I
am hesitant to change FD_SETSIZE globally on windows, due to
src/backend/port/win32/socket.c using it to size on-stack arrays. Instead add
-DFD_SETSIZE=1024 when building the specific targets needing it.

We likely should move away from using select() in those places, but that's a
larger change.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Reviewed-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20221005190829.lda7ttalh4mzrvf4@awork3.anarazel.de
Discussion: https://postgr.es/m/CA+hUKG+50eOUbN++ocDc0Qnp9Pvmou23DSXu=ZA6fepOcftKqA@mail.gmail.com
Discussion: https://postgr.es/m/20190826054000.GE7005%40paquier.xyz
2022-10-06 13:09:57 -07:00
Andres Freund 0fa41648d7 meson: Fix two comments
Author: Junwang Zhao <zhjwpku@gmail.com>
Discussion: https://postgr.es/m/CAEG8a3KxObc9g8NTzx1kX0Auf=J7FNiubYZXSK6G5wv5ShmP6A@mail.gmail.com
2022-10-06 13:09:37 -07:00
Tom Lane 9543eff5e0 Remove MemoryContextContains().
MemoryContextContains is no longer reliable in the wake of c6e0fe1f2,
because there's no longer very much redundancy in chunk headers.
(It wasn't *completely* reliable even before that, as there was a
chance of a false positive if you passed it something that didn't
point to an mcxt chunk at all.  But it was generally good enough.)

Hence, remove it.  There is no remaining core code that requires it.
Extensions that have been using it might be able to substitute a
test like "GetMemoryChunkContext(ptr) == context", recognizing that
this explicitly requires that the pointer point to some chunk.

Tom Lane and David Rowley

Discussion: https://postgr.es/m/1913788.1664898906@sss.pgh.pa.us
2022-10-06 13:35:31 -04:00
Tom Lane 42b746d4c9 Remove uses of MemoryContextContains in nodeAgg.c and nodeWindowAgg.c.
MemoryContextContains is no longer reliable in the wake of c6e0fe1f2,
so we need to get rid of these uses.

It appears that there's no really good reason to force the result of
an aggregate's finalfn or serialfn to be allocated in the per-tuple
context.  The only other plausible case is that the result points to
or into the aggregate's transition value, and that's fine because it
will last as long as we need it to.  (This conclusion depends on the
assumption that finalfns are not allowed to scribble on the transition
value, but we've long required that.)  So we can just drop the
MemoryContextContains plus datumCopy business, although we do need
to take care to not return a read-write pointer when the transition
value is an expanded datum.

Likewise, we don't really need to force the result of a window
function to be in the output context.  In this case, the plausible
alternative is that it's pointing into the temporary tuple slot used
by WinGetFuncArgInPartition or WinGetFuncArgInFrame (since those
functions could return such a pointer, which might become the window
function's result).  That will hold still for long enough, unless
there is another window function using the same WindowObject.
I'm content to always perform a datumCopy when there's more than one
such function.

On net, these changes should provide small speed improvements as well
as removing problematic code.

Tom Lane and David Rowley

Discussion: https://postgr.es/m/1913788.1664898906@sss.pgh.pa.us
2022-10-06 13:27:34 -04:00
Tom Lane 66c2922e76 Take care to de-duplicate entries in standby.c's table of locks.
The RecoveryLockLists data structure, which tracks all exclusive
locks that the startup process is holding on behalf of transactions
being replayed, did not have any provision for avoiding duplicate
entries for the same lock.  Maybe that was okay when the code was
first written.  However, modern practice is for checkpoints to
write fresh lists of all active exclusive locks into the WAL.
Thus, an exclusive lock that survives across multiple checkpoints
causes bloat in standbys' startup processes.  If there are a lot
of such locks this can look like a memory leak, and it's even
possible to drive the startup process into a palloc failure from
an over-length List.

To fix, use a hash table instead of simple lists to track the
locks being held.  Allowing for dynahash overhead, this requires
a little more space per lock than the old way (although it's the
same size as what we were allocating prior to c6e0fe1f2).  It's
probably a shade slower too.  However, testing indicates that the
penalty is negligible on ordinary workloads, so let's make this
change to improve robustness in extreme cases.

Patch by me, per report from Dmitriy Kuzmin.  No back-patch
(for now anyway), since it seems that a significant improvement
would only occur in corner cases.

Discussion: https://postgr.es/m/CAHLDt=_ts0A7Agn=hCpUh+RCFkxd+G6uuT=kcTfqFtGur0dp=A@mail.gmail.com
2022-10-06 12:27:36 -04:00
Tom Lane ca71131eeb Introduce t_isalnum() to replace t_isalpha() || t_isdigit() tests.
ts_locale.c omitted support for "isalnum" tests, perhaps on the
grounds that there were initially no use-cases for that.  However,
both ltree and pg_trgm need such tests, and we do also have one
use-case now in the core backend.  The workaround of testing
isalpha and isdigit separately seems quite inefficient, especially
when dealing with multibyte characters; so let's fill in the
missing support.

Discussion: https://postgr.es/m/2548310.1664999615@sss.pgh.pa.us
2022-10-06 11:08:56 -04:00
Michael Paquier 5757141cae Fix comment in xlogprefetcher.c
Author: Sho Kato
Discussion: https://postgr.es/m/TYCPR01MB684954052EC534A3261B29249F5C9@TYCPR01MB6849.jpnprd01.prod.outlook.com
2022-10-06 20:25:02 +09:00
Michael Paquier 051b096b8d Refactor TAP test authentication/001_password.pl
The test is changed to test for connection strings rather than specific
roles, and the reset logic of pg_hba.conf is extended so as the database
and user name entries can be directly specified.  This is aimed at being
used as a base for more test scenarios of pg_hba.conf and authentication
paths.

Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/Yz0xO0emJ+mxtj2a@paquier.xyz
2022-10-06 09:45:18 +09:00
David Rowley d8df67bb1a Fix final compiler warning produced by -Wshadow=compatible-local
We're now able to compile the entire tree with -Wshadow=compatible-local
without any compiler warnings.

Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvqWGMdB_pATeUqE=JCtNqNxObPOJ00jFEa2_sZ20j_Wvg@mail.gmail.com
2022-10-06 10:19:36 +13:00
David Rowley 112f0225db Add optional parameter to PG_TRY() macros
This optional parameter can be specified in cases where there are nested
PG_TRY() statements within a function in order to stop the compiler from
issuing warnings about shadowed local variables when compiling with
-Wshadow.  The optional parameter is used as a suffix on the variable
names declared within the PG_TRY(), PG_CATCH(), PG_FINALLY() and
PG_END_TRY() macros.  The parameter, if specified, must be the same in
each component macro of the given PG_TRY() block.

This also adjusts the single case where we have nested PG_TRY() statements
to add a parameter to the inner-most PG_TRY().

This reduces the number of compiler warnings when compiling with
-Wshadow=compatible-local from 5 down to 1.

Author: David Rowley
Discussion: https://postgr.es/m/CAApHDvqWGMdB_pATeUqE=JCtNqNxObPOJ00jFEa2_sZ20j_Wvg@mail.gmail.com
2022-10-06 10:08:31 +13:00
Andres Freund c3315a7da5 tests: Restrict pg_locks queries in advisory_locks.sql to current database
Otherwise testing an existing installation can fail, if there are other locks,
e.g. from one of the isolation tests.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221003234111.4ob7yph6r4g4ywhu@awork3.anarazel.de
2022-10-05 10:44:38 -07:00
Andres Freund 6a20b04f04 tests: Rename conflicting role names
These cause problems when running installcheck-world USE_MODULE_DB=1 with -j.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221003234111.4ob7yph6r4g4ywhu@awork3.anarazel.de
2022-10-05 10:43:13 -07:00
Andres Freund 902ab2fcef meson: Add windows resource files
The generated resource files aren't exactly the same ones as the old
buildsystems generate. Previously "InternalName" and "OriginalFileName" were
mostly wrong / not set (despite being required), but that was hard to fix in
at least the make build. Additionally, the meson build falls back to a
"auto-generated" description when not set, and doesn't set it in a few cases -
unlikely that anybody looks at these descriptions in detail.

Author: Andres Freund <andres@anarazel.de>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-10-05 09:56:05 -07:00
Andres Freund a1261cd16f meson: ecpg: Split definition of static and shared libraries
Required for correct resource file generation, as the resource files should
only be added to the shared library.

This also fixes a bunch of issues in the .pc files.

Previously I tried to avoid building sources twice, once for the static and
once for the shared libraries. We could still do so, but it's not clear that
it's worth the complication.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/20220927011951.j3h4o7n6bhf7dwau@awork3.anarazel.de
2022-10-05 09:56:05 -07:00
Andres Freund 089c0bc7a7 meson: libpq: Revise static / shared library setup
Improvements:
- we don't need -DFRONTEND for libpq anymore since 1d77afefbd
- the .pc file contents for a static libpq were wrong (referencing
  {pgport, common}_shlib)
- incidentally fixes meson not supporting link_whole on AIX yet
- added explanatory comments

Previously I tried to avoid building libpq's sources twice, once for the
static and once for the shared library. We could still do so, but it's not
clear that it's worth the complication.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
2022-10-05 09:56:05 -07:00
David Rowley 2d0bbedda7 Rename shadowed local variables
In a similar effort to f01592f91, here we mostly rename shadowed local
variables to remove the warnings produced when compiling with
-Wshadow=compatible-local.

This fixes 63 warnings and leaves just 5.

Author: Justin Pryzby, David Rowley
Reviewed-by: Justin Pryzby
Discussion https://postgr.es/m/20220817145434.GC26426%40telsasoft.com
2022-10-05 21:01:41 +13:00
Michael Paquier 839c2520a7 Remove definition of JUMBLE_SIZE from queryjumble.h
The same exists in queryjumble.c, and it is used only locally in this
file so let's remove the definition in the header.

Author: Tatsu Nakamori
Reviewed-by: Tom Lane, Julien Rouhaud
Discussion: https://postgr.es/m/bb4ebd0412da9b1ac87a5eb2a3646bf1@oss.nttdata.com
2022-10-05 14:27:50 +09:00
Michael Paquier 9a30e154b3 Use macros from xlog_internal.h for WAL segment logic in pg_resetwal
When scanning for the end of WAL, pg_resetwal has been maintaining its
own internal logic to calculate segment numbers and to parse a WAL
segment name for its timeline and segment number.  The code claimed for
example that XLogFromFileName() cannot be used because it lacks the
possibility of specifying a WAL segment size, which is not the case
since fc49e24, that has made the WAL segment size configurable at
initialization time, extending this routine to do so.

Similarly, this switches one segment number calculation to use
XLByteToSeg() rather than the same logic present in xlog_internal.h.
While on it, switch to TimeLineID in pg_resetwal.c for TLI numbers
parsed from segment names, to be more consistent with
XLogFromFileName().  The maths are exactly the same, but the code gets
simplified.

Author: Bharath Rupireddy
Reviewed-by: Kyotaro Horiguchi
Discussion: https://postgr.es/m/CALj2ACX+E_jnwqH_jmjhNG8BczJTNRTOLpw8K1CB1OcB48MJ8w@mail.gmail.com
2022-10-05 14:10:13 +09:00
Michael Paquier 9aa58d48f3 Add a few new patterns to the tab completion of psql
This improves the tab completion of psql on a few points:
- Provide a list of subscriptions on \dRs.
- Provide a list of publications on \dRp.
- Add CURRENT_ROLE, CURRENT_USER, SESSION_USER when OWNER TO is provided
at the end of a query (as defined by RoleSpec in gram.y).

Author: Vignesh C
Reviewed-by: Dagfinn Ilmari Mannsåker
Discussion: https://postgr.es/m/CALDaNm3toRBt6c6saY3174f7CsGztXRvVvfWbikkJEXY7x5WAA@mail.gmail.com
2022-10-05 11:46:10 +09:00
Michael Paquier bdf9b60085 Fix comment in guc_tables.c
s/ERROR_HANDLING/ERROR_HANDLING_OPTIONS/.

Author: Peter Smith
Discussion: https://postgr.es/m/CAHut+PtDj3CV+f0pVisc0XYMi2LHGBpQxQWtF0FjiSVN_nV17Q@mail.gmail.com
2022-10-04 15:39:41 +09:00
Michael Paquier c42cd05c58 Cleanup useless assignments and checks
This cleans up a couple of areas:
- Remove XLogSegNo calculation for the last WAL segment in backup in
xlog.c (7d70809 has moved this logic entirely to xlogbackup.c when
building the contents of the backup history file).
- Remove check on log_min_duration in analyze.c, as it is already true
where this code path is reached.
- Simplify call to find_option() in guc.c.

Author: Ranier Vilela
Reviewed-by: Masahiko Sawada
Discussion: https://postgr.es/m/CAEudQArCDQQiPiFR16=yu9k5s2tp4tgEe1U1ZbkW4ofx81AWWQ@mail.gmail.com
2022-10-04 13:16:23 +09:00
Michael Paquier 62be9e4cdc Add filtering capability for cross-version pg_upgrade tests
This commit expands the TAP tests of pg_upgrade when running these with
different major versions for the "old" cluster (to-be-upgraded) and the
"new" cluster (upgraded-to), by backporting some of the buildfarm
facilities directory into the script:
- Remove comments from the dump files, avoiding version-dependent
information.
- Remove empty lines from the dump files.
- Use --extra-float-digits=0 in the pg_dump command, when using an "old"
cluster with version equal to or lower than v11.
- Use --wal-segsize and --allow-group-access in initdb only when the
"old" cluster is equal to or higher than v11.

This allows the tests to pass down to v14 with the main regression test
suite, while v9.5~13 still generate some diffs, but these are minimal
compared to what happened before this commit.  Much more could be done,
especially around dump differences with function and procedures (these
can also be avoided with direct manipulation of the dumps loaded, for
example, in a way similar to the buildfarm), but at least the basics are
in place now.

Reviewed-by: Justin Pryzby, Anton A. Melnikov
Discussion: https://postgr.es/m/Yox1ME99GhAemMq1@paquier.xyz
2022-10-04 10:11:08 +09:00
Andres Freund 908e17151b meson: llvm: Use llvm-config's --cxxflags when building llvmjit
Otherwise we don't use LLVM's flags when building llvmjit_wrap.cpp and
llvmjit_inline.cpp. That can cause compile time failures if the C++ compiler
doesn't default to a new enough C++ standards version and link time failures
due to ABI influencing flags like -fno-rtti.
2022-10-03 14:55:52 -07:00
Tom Lane 4a79fd1a75 Fix psql's behavior with \g for a multiple-command string.
The pre-v15 behavior was to discard all but the last result,
but with the new behavior of printing all results by default,
we will send each such result to the \g file.  However,
we're still opening and closing the \g file for each result,
so you lose all but the last result anyway.  Move the output-file
state up to ExecQueryAndProcessResults so that we open/close the
\g file only once per command string.

To support this without changing other behavior, we must
adjust PrintQueryResult to have separate FILE * arguments
for query and status output (since status output has never
gone to the \g file).  That in turn makes it a good idea
to push the responsibility for fflush'ing output down to
PrintQueryTuples and PrintQueryStatus.

Also fix an infinite loop if COPY IN/OUT is attempted in \watch.
We used to reject that, but that error exit path got broken
somewhere along the line in v15.  There seems no real reason
to reject it anyway as the code now stands, so just remove
the error exit and make sure that COPY OUT data goes to the
right place.

Also remove PrintQueryResult's unused is_watch parameter,
and make some other cosmetic cleanups (adjust obsolete
comments, break some overly-long lines).

Daniel Vérité and Tom Lane

Discussion: https://postgr.es/m/4333844c-2244-4d6e-a49a-1d483fbe304f@manitou-mail.org
2022-10-03 15:07:17 -04:00
Tom Lane f4c7c410ee Revert "Optimize order of GROUP BY keys".
This reverts commit db0d67db24 and
several follow-on fixes.  The idea of making a cost-based choice
of the order of the sorting columns is not fundamentally unsound,
but it requires cost information and data statistics that we don't
really have.  For example, relying on procost to distinguish the
relative costs of different sort comparators is pretty pointless
so long as most such comparator functions are labeled with cost 1.0.
Moreover, estimating the number of comparisons done by Quicksort
requires more than just an estimate of the number of distinct values
in the input: you also need some idea of the sizes of the larger
groups, if you want an estimate that's good to better than a factor of
three or so.  That's data that's often unknown or not very reliable.
Worse, to arrive at estimates of the number of calls made to the
lower-order-column comparison functions, the code needs to make
estimates of the numbers of distinct values of multiple columns,
which are necessarily even less trustworthy than per-column stats.
Even if all the inputs are perfectly reliable, the cost algorithm
as-implemented cannot offer useful information about how to order
sorting columns beyond the point at which the average group size
is estimated to drop to 1.

Close inspection of the code added by db0d67db2 shows that there
are also multiple small bugs.  These could have been fixed, but
there's not much point if we don't trust the estimates to be
accurate in-principle.

Finally, the changes in cost_sort's behavior made for very large
changes (often a factor of 2 or so) in the cost estimates for all
sorting operations, not only those for multi-column GROUP BY.
That naturally changes plan choices in many situations, and there's
precious little evidence to show that the changes are for the better.
Given the above doubts about whether the new estimates are really
trustworthy, it's hard to summon much confidence that these changes
are better on the average.

Since we're hard up against the release deadline for v15, let's
revert these changes for now.  We can always try again later.

Note: in v15, I left T_PathKeyInfo in place in nodes.h even though
it's unreferenced.  Removing it would be an ABI break, and it seems
a bit late in the release cycle for that.

Discussion: https://postgr.es/m/TYAPR01MB586665EB5FB2C3807E893941F5579@TYAPR01MB5866.jpnprd01.prod.outlook.com
2022-10-03 10:56:16 -04:00
Michael Paquier f60eb3f282 Add authentication TAP test for peer authentication
This commit introduces an authentication test for the peer method, as of
a set of scenarios with and without a user name map.  The script is
automatically skipped if peer is not supported in the environment where
this test is run, checking this behavior by attempting a connection
first on a cluster up and running.

Author: Bertrand Drouvot
Discussion: https://postgr.es/m/aa60994b-1c66-ca7a-dab9-9a200dbac3d2@amazon.com
2022-10-03 16:42:25 +09:00
Peter Eisentraut a9d58bfe8a Fix tiny memory leaks
Both check_application_name() and check_cluster_name() use
pg_clean_ascii() but didn't release the memory.  Depending on when the
GUC is set, this might be cleaned up at some later time or it would
leak postmaster memory once.  In any case, it seems better not to have
to rely on such analysis and make the code locally robust.  Also, this
makes Valgrind happier.

Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Jacob Champion <jchampion@timescale.com>
Discussion: https://www.postgresql.org/message-id/CAD21AoBmFNy9MPfA0UUbMubQqH3AaK5U3mrv6pSeWrwCk3LJ8g@mail.gmail.com
2022-10-01 12:48:24 +02:00
Michael Paquier 83e42a0035 doc: Fix some grammar and typos
This fixes some areas related to logical replication and custom RMGRs.

Author: Ekaterina Kiryanova
Discussion: https://postgr.es/m/fa4773f1-1396-384a-bcd7-85b5e013f399@postgrespro.ru
Backpatch-through: 15
2022-10-01 15:28:02 +09:00
Tom Lane 2dc2e4e31a Avoid improbable PANIC during heap_update, redux.
Commit 34f581c39 intended to ensure that RelationGetBufferForTuple
would acquire a visibility-map page pin in case the otherBuffer's
all-visible bit had become set since we last had lock on that page.
But I missed a case: when we're extending the relation, VM concerns
were dealt with only in the relatively-less-likely case that we
fail to conditionally lock the otherBuffer.  I think I'd believed
that we couldn't need to worry about it if the conditional lock
succeeds, which is true for the target buffer; but the otherBuffer
was unlocked for awhile so its bit might be set anyway.  So we need
to do the GetVisibilityMapPins dance, and then also recheck the
page's free space, in both cases.

Per report from Jaime Casanova.  Back-patch to v12 as the previous
patch was (although there's still no evidence that the bug is
reachable pre-v14).

Discussion: https://postgr.es/m/E1lWLjP-00006Y-Ml@gemulon.postgresql.org
2022-09-30 19:37:00 -04:00
Andres Freund 0e497eadb1 mingw: Define PGDLLEXPORT as __declspec (dllexport) as done for msvc
While mingw would otherwise fall back to
__attribute__((visibility("default"))), that appears to only work as long as
no symbols are declared with __declspec(dllexport). But we can end up with
some, e.g. plpython's Py_Init.

It's quite possible we should do the same for cygwin, but I don't have a test
environment for that...

Discussion: http://postgr.es/m/20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
Discussion: http://postgr.es/m/20220928025242.ugf7t5ugxxgmkraa@awork3.anarazel.de
2022-09-30 10:50:05 -07:00
Tom Lane 4e4f7b9fcc Adjust PQsslAttributeNames() to match PQsslAttribute().
Currently, PQsslAttributeNames() returns the same list of attribute
names regardless of its conn parameter.  This patch changes it to
have behavior parallel to what 80a05679d installed for PQsslAttribute:
you get OpenSSL's attributes if conn is NULL or is an SSL-encrypted
connection, or an empty list if conn is a non-encrypted connection.
The point of this is to have sensible connection-dependent behavior
in case we ever support multiple SSL libraries.  The behavior for
NULL can be defined as "the attributes for the default SSL library",
parallel to what PQsslAttribute(NULL, "library") does.

Since this is mostly just future-proofing, no back-patch.

Discussion: https://postgr.es/m/17625-fc47c78b7d71b534@postgresql.org
2022-09-30 10:26:47 -04:00
Alvaro Herrera 69298db8e1
Fix tab-completion after commit 790bf615dd
I (Álvaro) broke tab-completion for GRANT .. ALL TABLES IN SCHEMA while
removing ALL from the publication syntax for schemas in the
aforementioned commit.  I also missed to update a bunch of
tab-completion rules for ALTER/CREATE PUBLICATION that match each
individual piece of ALL TABLES IN SCHEMA.  Repair those bugs.

While fixing up that commit, update a couple of outdated comments
related to the same change.

Backpatch to 15.

Author: Shi yu <shiy.fnst@fujitsu.com>
Reviewed-by: Peter Smith <smithpb2250@gmail.com>
Discussion: https://postgr.es/m/OSZPR01MB6310FCE8609185A56344EED2FD559@OSZPR01MB6310.jpnprd01.prod.outlook.com
2022-09-30 12:53:31 +02:00
Michael Paquier 65b158ae4e Remove useless argument from UnpinBuffer()
The last caller of UnpinBuffer() that did not want to adjust
CurrentResourceOwner was removed in 2d115e4, and nothing has been
introduced in bufmgr.c to do the same thing since.  This simplifies 10
code paths.

Author: Aleksander Alekseev
Reviewed-by: Nathan Bossart, Zhang Mingli, Bharath Rupireddy
Discussion: https://postgr.es/m/CAJ7c6TOmmFpb6ohurLhTC7hKNJWGzdwf8s4EAtAZxD48g-e6Jw@mail.gmail.com
2022-09-30 15:57:47 +09:00
Tom Lane 80a05679d5 Fix bogus behavior of PQsslAttribute(conn, "library").
Commit ebc8b7d44 intended to change the behavior of
PQsslAttribute(NULL, "library"), but accidentally also changed
what happens with a non-NULL conn pointer.  Undo that so that
only the intended behavior change happens.  Clarify some
associated documentation.

Per bug #17625 from Heath Lord.  Back-patch to v15.

Discussion: https://postgr.es/m/17625-fc47c78b7d71b534@postgresql.org
2022-09-29 17:28:09 -04:00
Tom Lane 551aa6b7b9 Improve wording of log messages triggered by max_slot_wal_keep_size.
The one about "terminating process to release replication slot" told
you nothing about why that was happening.  The one about "invalidating
slot because its restart_lsn exceeds max_slot_wal_keep_size" told you
what was happening, but violated our message style guideline about
keeping the primary message short.  Add DETAIL/HINT lines to carry
the appropriate detail and make the two cases more uniform.

While here, fix bogus test logic in 019_replslot_limit.pl: if it timed
out without seeing the expected log message, no test failure would be
reported.  This is flat broken since commit 549ec201d removed the test
counts; even before that it was horribly bad style, since you'd only
get told that not all tests had been run.

Kyotaro Horiguchi, reviewed by Bertrand Drouvot; test fixes by me

Discussion: https://postgr.es/m/20211214.130456.2233153190058148084.horikyota.ntt@gmail.com
2022-09-29 13:27:48 -04:00
Tom Lane d7e39d72ca Use actual backend IDs in pg_stat_get_backend_idset() and friends.
Up to now, the ID values returned by pg_stat_get_backend_idset() and
used by pg_stat_get_backend_activity() and allied functions were just
indexes into a local array of sessions seen by the last stats refresh.
This is problematic for a few reasons.  The "ID" of a session can vary
over its existence, which is surprising.  Also, while these numbers
often match the "backend ID" used for purposes like temp schema
assignment, that isn't reliably true.  We can fairly cheaply switch
things around to make these numbers actually be the sessions' backend
IDs.  The added test case illustrates that with this definition, the
temp schema used by a given session can be obtained given its PID.

While here, delete some dead code that guarded against getting
a NULL return from pgstat_fetch_stat_local_beentry().  That can't
happen as long as the caller is careful to pass an in-range array
index, as all the callers are.  (This code may not have been dead
when written, but it surely is now.)

Nathan Bossart

Discussion: https://postgr.es/m/20220815205811.GA250990@nathanxps13
2022-09-29 12:14:39 -04:00
Etsuro Fujita d5e3fe682a Update comment in ExecInsert() regarding batch insertion.
Remove the stale text that is a leftover from an earlier version of the
patch to add support for batch insertion, and adjust the wording in the
remaining text.

Back-patch to v14 where batch insertion came in.

Review and wording adjustment by Tom Lane.

Discussion: https://postgr.es/m/CAPmGK14goatHPHQv2Aeu_UTKqZ%2BBO%2BP%2Bzd3HKv5D%2BdyyfWKDSw%40mail.gmail.com
2022-09-29 16:55:00 +09:00
Michael Paquier 0823d061b0 Introduce SYSTEM_USER
SYSTEM_USER is a reserved keyword of the SQL specification that,
roughly described, is aimed at reporting some information about the
system user who has connected to the database server.  It may include
implementation-specific information about the means by the user
connected, like an authentication method.

This commit implements SYSTEM_USER as of auth_method:identity, where
"auth_method" is a keyword about the authentication method used to log
into the server (like peer, md5, scram-sha-256, gss, etc.) and
"identity" is the authentication identity as introduced by 9afffcb (peer
sets authn to the OS user name, gss to the user principal, etc.).  This
format has been suggested by Tom Lane.

Note that thanks to d951052, SYSTEM_USER is available to parallel
workers.

Bump catalog version.

Author: Bertrand Drouvot
Reviewed-by: Jacob Champion, Joe Conway, Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/7e692b8c-0b11-45db-1cad-3afc5b57409f@amazon.com
2022-09-29 15:05:40 +09:00
Michael Paquier 5ac9e86919 Mark sigint_interrupt_enabled as sig_atomic_t
This is a continuation of 78fdb1e, where this flag is set in the psql
callback handler used for SIGINT.  This was previously a boolean but the
C standard recommends the use of sig_atomic_t.  Note that this
influences PromptInterruptContext in string.h, where the same flag is
tracked.

Author: Hayato Kuroda
Discussion: https://postgr.es/m/TYAPR01MB58669A9EC96AA3078C2CD938F5549@TYAPR01MB5866.jpnprd01.prod.outlook.com
2022-09-29 14:28:13 +09:00