Commit Graph

57109 Commits

Author SHA1 Message Date
Michael Paquier befe9451fb doc: Improve example query related to pg_wait_events
Author: Pavel Luzanov
Discussion: https://postgr.es/m/4f79be75-7e30-4817-b0da-4a691ea5427f@postgrespro.ru
2023-10-24 08:40:43 +09:00
Michael Paquier 9972c7de1d Fix typos in wait_event.c
Noticed while working on a different patch.  Introduced in af720b4c50.
2023-10-24 08:05:29 +09:00
Robert Haas 5b36e8f078 Change struct tablespaceinfo's oid member from 'char *' to 'Oid'
This shouldn't change behavior except in the unusual case where
there are file in the tablespace directory that have entirely
numeric names but are nevertheless not possible names for a
tablespace directory, either because their names have leading zeroes
that shouldn't be there, or the value is actually zero, or because
the value is too large to represent as an OID.

In those cases, the directory would previously have made it into
the list of tablespaceinfo objects and no longer will. Thus, base
backups will now ignore such directories, instead of treating them
as legitimate tablespace directories. Similarly, if entries for
such tablespaces occur in a tablespace_map file, they will now
be rejected as erroneous, instead of being honored.

This is infrastructure for future work that wants to be able to
know the tablespace of each relation that is part of a backup
*as an OID*. By strengthening the up-front validation, we don't
have to worry about weird cases later, and can more easily avoid
repeated string->integer conversions.

Patch by me, reviewed by David Steele.

Discussion: http://postgr.es/m/CA+TgmoZNVeBzoqDL8xvr-nkaepq815jtDR4nJzPew7=3iEuM1g@mail.gmail.com
2023-10-23 15:17:26 -04:00
Robert Haas 5c47c6546c Refactor parse_filename_for_nontemp_relation to parse more.
Instead of returning the number of characters in the RelFileNumber,
return the RelFileNumber itself. Continue to return the fork number,
as before, and additionally return the segment number.

parse_filename_for_nontemp_relation now rejects a RelFileNumber or
segment number that begins with a leading zero. Before, we accepted
such cases as relation filenames, but if we continued to do so after
this change, the function might return the same values for two
different files (e.g. 1234.5 and 001234.5 or 1234.005) which could be
annoying for callers. Since we don't actually ever generate filenames
with leading zeroes in the names, any such files that we find must
have been created by something other than PostgreSQL, and it is
therefore reasonable to treat them as non-relation files.

Along the way, change unlogged_relation_entry to store a RelFileNumber
rather than an OID. This update should have been made in
851f4cc75c, but it was overlooked.
It's trivial to make the update as part of this commit, perhaps more
trivial than it would have been without it, so do that.

Patch by me, reviewed by David Steele.

Discussion: http://postgr.es/m/CA+TgmoZNVeBzoqDL8xvr-nkaepq815jtDR4nJzPew7=3iEuM1g@mail.gmail.com
2023-10-23 15:08:53 -04:00
Michael Paquier b6f1cca9ba Remove unnecessary break in pg_logical_replication_slot_advance()
pg_logical_replication_slot_advance() included a break condition to stop
when a targeted LSN is reached, when processing a series of WAL records
with XLogReadRecord().  Since 38a957316d, it matched with the check of
its main while loop.  This condition saved from an extra CFI check,
actually pointless, so let's remove this condition and simplify the
code.

In passing, fix an incorrect comment.

Author: Bharath Rupireddy
Reviewed-by: Tom Lane, Gurjeet Singh
Discussion: https://postgr.es/m/CALj2ACWfGDLQ2cy7ZKwxnJqbDkO6Yvqqrqxne5ZN4HYm=PRTGg@mail.gmail.com
2023-10-23 10:20:30 +09:00
Michael Paquier 40ebc41576 doc: Fix some grammar and inconsistent tags
Author: Ekaterina Kiryanova, Elena Indrupskaya, Oleg Sibiryakov, Maxim
Yablokov
Discussion: https://postgr.es/m/4c2a430b-32e2-44e2-aeca-03b7db6824e4@postgrespro.ru
2023-10-23 09:58:55 +09:00
Thomas Munro 5e4dacb987 Log LLVM library version in configure output.
When scanning build farm results, it's useful to be able to see which
version is in use.  For the Meson build system, this information was
already displayed.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/4022690.1697852728%40sss.pgh.pa.us
2023-10-22 14:22:11 +13:00
Thomas Munro dab889d60b Fix min_dynamic_shared_memory on Windows.
When min_dynamic_shared_memory is set above 0, we try to find space in a
pre-allocated region of the main shared memory area instead of calling
dsm_impl_XXX() routines to allocate more.  The dsm_pin_segment() and
dsm_unpin_segment() routines had a bug: they called dsm_impl_XXX()
routines even for main region segments.  Nobody noticed before now
because those routines do nothing on Unix, but on Windows they'd fail
while attempting to duplicate an invalid Windows HANDLE.  Add the
missing gating.

Back-patch to 14, where commit 84b1c63a added this feature.  Fixes
pgsql-bugs bug #18165.

Reported-by: Maxime Boyer <maxime.boyer@cra-arc.gc.ca>
Tested-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18165-bf4f525cea6e51de%40postgresql.org
2023-10-22 10:04:55 +13:00
Tom Lane 2d870b4aef Allow ALTER SYSTEM to set unrecognized custom GUCs.
Previously, ALTER SYSTEM failed if the target GUC wasn't present in
the session's GUC hashtable.  That is a reasonable behavior for core
(single-part) GUC names, and for custom GUCs for which we have loaded
an extension that's reserved the prefix.  But it's unnecessarily
restrictive otherwise, and it also causes inconsistent behavior:
you can "ALTER SYSTEM SET foo.bar" only if you did "SET foo.bar"
earlier in the session.  That's fairly silly.

Hence, refactor things so that we can execute ALTER SYSTEM even
if the variable doesn't have a GUC hashtable entry, as long as the
name meets the custom-variable naming requirements and does not
have a reserved prefix.  (It's safe to do this even if the
variable belongs to an extension we currently don't have loaded.
A bad value will at worst cause a WARNING when the extension
does get loaded.)

Also, adjust GRANT ON PARAMETER to have the same opinions about
whether to allow an unrecognized GUC name, and to throw the
same errors if not (it previously used a one-size-fits-all
message for several distinguishable conditions).  By default,
only a superuser will be allowed to do ALTER SYSTEM SET on an
unrecognized name, but it's possible to GRANT the ability to
do it.

Patch by me, pursuant to a documentation complaint from
Gavin Panella.  Arguably this is a bug fix, but given the
lack of other complaints I'll refrain from back-patching.

Discussion: https://postgr.es/m/2617358.1697501956@sss.pgh.pa.us
Discussion: https://postgr.es/m/169746329791.169914.16613647309012285391@wrigleys.postgresql.org
2023-10-21 13:35:19 -04:00
Alvaro Herrera 36a14afc07
Make some error strings more generic
It's undesirable to have SQL commands or configuration options in a
translatable error string, so take some of these out.
2023-10-20 22:52:15 +02:00
Andres Freund 41da94fd5c meson: Make detection of python more robust
Previously we errored out if no python installation could be found (but we did
handle not having enough of python installed to build plpython
against). Presumably nobody hit this so far, as python is likely installed due
to meson requiring python.

Author: Tristan Partin <tristan@neon.tech>
Discussion: https://postgr.es/m/CSPIJVUDZFKX.3KHMOAVGF94RV@c3po
Backpatch: 16-, where meson support was added
2023-10-20 11:11:31 -07:00
Tom Lane 52f22cd4e8 Doc: update CREATE OPERATOR's statement about => as an operator.
This doco said that use of => as an operator "is deprecated".
It's been fully disallowed since 865f14a2d back in 9.5, but
evidently that commit missed updating this statement.
Do so now.
2023-10-20 13:01:02 -04:00
Tom Lane 2b5154beab Extend ALTER OPERATOR to allow setting more optimization attributes.
Allow the COMMUTATOR, NEGATOR, MERGES, and HASHES attributes to be set
by ALTER OPERATOR.  However, we don't allow COMMUTATOR/NEGATOR to be
changed once set, nor allow the MERGES/HASHES flags to be unset once
set.  Changes like that might invalidate plans already made, and
dealing with the consequences seems like more trouble than it's worth.
The main use-case we foresee for this is to allow addition of missed
properties in extension update scripts, such as extending an existing
operator to support hashing.  So only transitions from not-set to set
states seem very useful.

This patch also causes us to reject some incorrect cases that formerly
resulted in inconsistent catalog state, such as trying to set the
commutator of an operator to be some other operator that already has a
(different) commutator.

While at it, move the InvokeObjectPostCreateHook call for CREATE
OPERATOR to not occur until after we've fixed up commutator or negator
links as needed.  The previous ordering could only be justified by
thinking of the OperatorUpd call as a kind of ALTER OPERATOR step;
but we don't call InvokeObjectPostAlterHook therein.  It seems better
to let the hook see the final state of the operator object.

In the documentation, move the discussion of how to establish
commutator pairs from xoper.sgml to the CREATE OPERATOR ref page.

Tommy Pavlicek, reviewed and editorialized a bit by me

Discussion: https://postgr.es/m/CAEhP-W-vGVzf4udhR5M8Bdv88UYnPrhoSkj3ieR3QNrsGQoqdg@mail.gmail.com
2023-10-20 12:28:46 -04:00
Michael Paquier dcd4454590 Remove unnecessary dependencies to wal_level=logical in TAP tests
A couple of TAP tests make use of wal_level=logical for nodes that do
not need to do any kind of logical decoding operations, like
subscription nodes on which changes are only applied.  This can be
confusing when reading these tests as setup examples, so let's remove
this configuration where not required (contrary to two-way logical
replication and similar more complex cases).  This simplifies the tests
a bit, making them slightly cheaper with less WAL generated overall.

Author: Hayato Kuroda
Discussion: https://postgr.es/m/TYAPR01MB5866946BCEB747ABE513ACC6F5D5A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-10-20 10:09:27 +09:00
Robert Haas afd12774ae During online checkpoints, insert XLOG_CHECKPOINT_REDO at redo point.
This allows tools that read the WAL sequentially to identify (possible)
redo points when they're reached, rather than only being able to
detect them in retrospect when XLOG_CHECKPOINT_ONLINE is found, possibly
much later in the WAL stream. There are other possible applications as
well; see the discussion links below.

Any redo location that precedes the checkpoint location should now point
to an XLOG_CHECKPOINT_REDO record, so add a cross-check to verify this.

While adjusting the code in CreateCheckPoint() for this patch, I made it
call WALInsertLockAcquireExclusive a bit later than before, since there
appears to be no need for it to be held while checking whether the system
is idle, whether this is an end-of-recovery checkpoint, or what the current
timeline is.

Bump XLOG_PAGE_MAGIC.

Patch by me, based in part on earlier work from Dilip Kumar. Review by
Dilip Kumar, Amit Kapila, Andres Freund, and Michael Paquier.

Discussion: http://postgr.es/m/CA+TgmoYy-Vc6G9QKcAKNksCa29cv__czr+N9X_QCxEfQVpp_8w@mail.gmail.com
Discussion: http://postgr.es/m/20230614194717.jyuw3okxup4cvtbt%40awork3.anarazel.de
Discussion: http://postgr.es/m/CA+hUKG+b2ego8=YNW2Ohe9QmSiReh1-ogrv8V_WZpJTqP3O+2w@mail.gmail.com
2023-10-19 14:47:29 -04:00
Tom Lane 8483a54b7d Doc: modernize comment for boolin().
Most of the behavior described by this comment was moved to
parse_bool_with_len() some time ago.  Move what's still
valuable there too, and drop the rest.

Peter Smith

Discussion: https://postgr.es/m/CAHut+PtMJURKp=U8Z=Ktp0zV40sEb1f-iEk9FvY2GQe+5ZBnwg@mail.gmail.com
2023-10-19 11:31:05 -04:00
Michael Paquier 5147ab1dd3 pg_stat_statements: Add local_blk_{read|write}_time
This commit adds to pg_stat_statements the two new fields for local
buffers introduced by 295c36c0c1, adding the time spent to read and
write these blocks.  These are similar to what is done for temp and
shared blocks.  This information available only if track_io_timing is
enabled.

Like for 5a3423ad8e, no version bump is required in the module.

Author: Nazir Bilal Yavuz
Reviewed-by: Robert Haas, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
2023-10-19 14:03:31 +09:00
Michael Paquier 295c36c0c1 Add local_blk_{read|write}_time I/O timing statistics for local blocks
There was no I/O timing statistics for counting read and write timings
on local blocks, contrary to the counterparts for temp and shared
blocks.  This information is available when track_io_timing is enabled.

The output of EXPLAIN is updated to show this information.  An update of
pg_stat_statements is planned next.

Author: Nazir Bilal Yavuz
Reviewed-by: Robert Haas, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
2023-10-19 13:39:38 +09:00
Michael Paquier 13d00729d4 Rename I/O timing statistics columns to shared_blk_{read|write}_time
These two counters, defined in BufferUsage to track respectively the
time spent while reading and writing blocks have historically only
tracked data related to shared buffers, when track_io_timing is enabled.

An upcoming patch to add specific counters for local buffers will take
advantage of this rename as it has come up that no data is currently
tracked for local buffers, and tracking local and shared buffers using
the same fields would be inconsistent with the treatment done for temp
buffers.  Renaming the existing fields clarifies what the block type of
each stats field is.

pg_stat_statement is updated to reflect the rename.  No extension
version bump is required as 5a3423ad8e has done one, affecting v17~.

Author: Nazir Bilal Yavuz
Reviewed-by: Robert Haas, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
2023-10-19 11:26:40 +09:00
Tom Lane 9b103f861e Improve pglz_decompress's defenses against corrupt compressed data.
When processing a match tag, check to see if the claimed "off"
is more than the distance back to the output buffer start.
If it is, then the data is corrupt, and what's more we would
fetch from outside the buffer boundaries and potentially incur
a SIGSEGV.  (Although the odds of that seem relatively low, given
that "off" can't be more than 4K.)

Back-patch to v13; before that, this function wasn't really
trying to protect against bad data.

Report and fix by Flavien Guedez.

Discussion: https://postgr.es/m/01fc0593-e31e-463d-902c-dd43174acee2@oopacity.net
2023-10-18 20:43:27 -04:00
Michael Paquier 7fb355db14 Install wait_event_types.h in VPATH builds
An extra rule is needed in src/include/Makefile for VPATH builds to
install any generated server-side include files, and wait_event_types.h
was forgotten from the set.

Issue introduced by fa88928470.

Reported-by: Christoph Berg
Discussion: https://postgr.es/m/ZTAA11u7CtX6NqlK@msg.df7cb.de
2023-10-19 09:42:46 +09:00
Thomas Munro 76200e5ee4 jit: Changes for LLVM 17.
Changes required by https://llvm.org/docs/NewPassManager.html.

Back-patch to 12, leaving the final release of 11 unchanged, consistent
with earlier decision not to back-patch LLVM 16 support either.

Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2BWXznXCyTgCADd%3DHWkP9Qksa6chd7L%3DGCnZo-MBgg9Lg%40mail.gmail.com
2023-10-19 05:13:23 +13:00
Thomas Munro f90b4a846b jit: Supply LLVMGlobalGetValueType() for LLVM < 8.
Commit 37d5babb used this C API function while adding support for LLVM
16 and opaque pointers, but it's not available in LLVM 7 and older.
Provide it in our own llvmjit_wrap.cpp.  It just calls a C++ function
that pre-dates LLVM 3.9, our minimum target.

Back-patch to 12, like 37d5babb.

Discussion: https://postgr.es/m/CA%2BhUKGKnLnJnWrkr%3D4mSGhE5FuTK55FY15uULR7%3Dzzc%3DwX4Nqw%40mail.gmail.com
2023-10-19 03:01:55 +13:00
Thomas Munro 37d5babb5c jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].

 * For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
 * For LLVM == 15, we'll continue to do the same, explicitly opting
   out of opaque pointer mode.
 * For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
   the extra type argument.

The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc.  The change is mostly mechanical,
except that at each site the correct type had to be provided.

In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.

Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.

[1] https://llvm.org/docs/OpaquePointers.html

Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 22:47:23 +13:00
Michael Paquier 3f9b1f26ca pg_upgrade: Fix test name in 002_pg_upgrade.pl
Author: Hou Zhijie
Discussion: https://postgr.es/m/TYAPR01MB5724A40D47E71F4717357EC694D5A@TYAPR01MB5724.jpnprd01.prod.outlook.com
Backpatch-through: 15
2023-10-18 18:23:25 +09:00
Peter Eisentraut 41a284411e Clean up some pg_dump tests
1) Remove useless entries from "unlike" lists.  Runs that are not
   listed in "like" don't need to be excluded in "unlike".

2) Ensure there is always a "like" list, even if it is empty.  This
   makes the test more self-documenting.

3) Use predefined lists such as %full_runs where appropriate, instead
   of listing all runs separately.

Also add code that checks 1 and 2 automatically and dies with an error
for violations.

Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://www.postgresql.org/message-id/flat/1f8cb371-e84e-434e-0367-6b716fb16fa1@eisentraut.org
2023-10-18 08:08:17 +02:00
Michael Paquier d17ffc734d Count write times when extending relation files for shared buffers
Relation files extended by multiple blocks at a time have been counting
the number of blocks written, but forgot to increment the write time in
this case, as single-block write and relation extension are treated as
two different I/O operations in the shared stats: IOOP_EXTEND vs
IOOP_WRITE.  In this case IOOP_EXTEND was forgotten for normal
(non-temporary) relations, still the number of blocks written was
incremented according to the relation extend done.

Write times are tracked when track_io_timing is enabled, which is not
the case by default.

Author: Nazir Bilal Yavuz
Reviewed-by: Robert Haas, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
Backpatch-through: 16
2023-10-18 14:54:33 +09:00
Michael Paquier 173b56f1ef Add flush option to pg_logical_emit_message()
Since its introduction, LogLogicalMessage() (via the SQL interface
pg_logical_emit_message()) has never included a call to XLogFlush(),
causing it to potentially lose messages on a crash when used in
non-transactional mode.  This has come up to me as a problem while
playing with ideas to design a test suite for what has become
039_end_of_wal.pl introduced in bae868caf2 by Thomas Munro, because
there are no direct ways to force a WAL flush via SQL.

The default is false, to not flush messages and influence existing
use-cases where this function could be used.  If set to true, the
message emitted is flushed before returning back to the caller, making
the message durable on crash.  This new option has no effect when using
pg_logical_emit_message() in transactional mode, as the record's flush
is guaranteed by the WAL record generated by the transaction committed.

Two queries of test_decoding are tweaked to cover the new code path for
the flush.

Bump catalog version.

Author: Michael Paquier
Reviewed-by: Andres Freund, Amit Kapila, Fujii Masao, Tung Nguyen, Tomas
Vondra
Discussion: https://postgr.es/m/ZNsdThSe2qgsfs7R@paquier.xyz
2023-10-18 11:24:59 +09:00
Tom Lane 19fa977311 Dodge a compiler bug affecting timetz_zone/timetz_izone.
Use a modulo operator instead of implementing the same behavior
with a loop.  The loop solution is doubtless microscopically
faster for the typical case of only wrapping into the very next
day, but maybe not so much for large interval values.  In any
case, timetz is such a backwater that it's doubtful anybody
would notice any performance change anyway.

This avoids a compiler bug occurring in AIX's xlc, even in pretty
late-model revisions.

We did not have test coverage for the case where the initial
result->time value is negative, so add that.

For the moment, install this only in HEAD.  My plan is to
back-patch the test case, and then the code change assuming that
buildfarm testing proves the bug occurs in the back branches.
(That seems pretty likely, but let's find out for sure.)

Per buildfarm results from commits 97957fdba and 2f0472030.
Thanks to Michael Paquier for the idea to use a modulo operation
to replace the faulty loop.

Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
2023-10-17 13:10:35 -04:00
Nathan Bossart 97550c0711 Avoid calling proc_exit() in processes forked by system().
The SIGTERM handler for the startup process immediately calls
proc_exit() for the duration of the restore_command, i.e., a call
to system().  This system() call forks a new process to execute the
shell command, and this child process inherits the parent's signal
handlers.  If both the parent and child processes receive SIGTERM,
both will attempt to call proc_exit().  This can end badly.  For
example, both processes will try to remove themselves from the
PGPROC shared array.

To fix this problem, this commit adds a check in
StartupProcShutdownHandler() to see whether MyProcPid == getpid().
If they match, this is the parent process, and we can proc_exit()
like before.  If they do not match, this is a child process, and we
just emit a message to STDERR (in a signal safe manner) and
_exit(), thereby skipping any problematic exit callbacks.

This commit also adds checks in proc_exit(), ProcKill(), and
AuxiliaryProcKill() that verify they are not being called within
such child processes.

Suggested-by: Andres Freund
Reviewed-by: Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 11
2023-10-17 10:41:48 -05:00
Robert Haas 2406c4e34c Reword messages about impending (M)XID exhaustion.
First, we shouldn't recommend switching to single-user mode, because
that's terrible advice. Especially on newer versions where VACUUM
will enter emergency mode when nearing (M)XID exhaustion, it's
perfectly fine to just VACUUM in multi-user mode. Doing it that way
is less disruptive and avoids disabling the safeguards that prevent
actual wraparound, so recommend that instead.

Second, be more precise about what is going to happen (when we're
nearing the limits) or what is happening (when we actually hit them).
The database doesn't shut down, nor does it refuse all commands. It
refuses commands that assign whichever of XIDs and MXIDs are nearly
exhausted.

No back-patch. The existing hint that advises going to single-user
mode is sufficiently awful advice that removing it or changing it
might be justifiable even though we normally avoid changing
user-facing messages in back-branches, but I (rhaas) felt that it
was better to be more conservative and limit this fix to master
only. Aside from the usual risk of breaking translations, people
might be used to the existing message, or even have monitoring
scripts that look for it.

Alexander Alekseev, John Naylor, Robert Haas, reviewed at various
times by Peter Geoghegan, Hannu Krosing, and Andres Freund.

Discussion: http://postgr.es/m/CA+TgmoZBg95FiR9wVQPAXpGPRkacSt2okVge+PKPPFppN7sfnQ@mail.gmail.com
2023-10-17 10:34:21 -04:00
Robert Haas a1a5da8cb7 Talk about assigning, rather than generating, new MultiXactIds.
The word "assign" is used in various places internally to describe what
GetNewMultiXactId does, but the user-facing messages have previously
said "generate". For consistency, standardize on "assign," which seems
(at least to me) to be slightly clearer.

Discussion: http://postgr.es/m/CA+TgmoaoE1_i3=4-7GCTtKLVZVQ2Gh6qESW2VG1OprtycxOHMA@mail.gmail.com
2023-10-17 10:23:31 -04:00
Michael Paquier d6b0c2bcb1 Improve truncation of pg_serial/, removing "apparent wraparound" LOGs
It is possible that the tail XID of pg_serial/ gets ahead of its head
XID, which would cause the truncation of pg_serial/ done during
checkpoints to show up as a "wraparound" LOG in SimpleLruTruncate(),
which is confusing.  This also wastes a bit of disk space until the head
page is reclaimed again.

CheckPointPredicate() is changed so as the cutoff page for the
truncation is switched to the head page if the tail XID has advanced
beyond the head XID, rather than the tail page.  This prevents the
confusing LOG message about a wraparound while allowing some truncation
to be done to cut in disk space.

This could be considered as a bug fix, but the original behavior is
harmless as well, resulting only in disk space temporarily wasted, so
no backpatch is done.

Author: Sami Imseih
Reviewed-by: Heikki Linnakangas, Michael Paquier
Discussion: https://postgr.es/m/755E19CA-D02C-4A4C-80D3-74F775410C48@amazon.com
2023-10-17 14:36:21 +09:00
Alexander Korotkov 6fcaeb0ea2 Run 006_login_trigger.pl only with Unix-domain sockets
Per report from buildfarm member drongo.

Reported-by: Alexander Lakhin
2023-10-17 08:11:40 +03:00
Amit Kapila 79243de13f Restart the apply worker if the privileges have been revoked.
Restart the apply worker if the subscription owner's superuser privileges
have been revoked. This is required so that the subscription connection
string gets revalidated and use the password option to connect to the
publisher for non-superusers, if required.

Author: Vignesh C
Reviewed-by: Amit Kapila
Discussion: http://postgr.es/m/CALDaNm2Dxmhq08nr4P6G+24QvdBo_GAVyZ_Q1TcGYK+8NHs9xw@mail.gmail.com
2023-10-17 08:41:44 +05:30
Tom Lane 2f04720307 Add regression test coverage for timetz_izone().
Extend the test added by commit 97957fdba so that it also covers
timetz_izone(), that is the "AT TIME ZONE interval" case.
This is mostly to see if xlc's apparent bug occurs there too,
but more code coverage is always welcome.

Discussion: https://postgr.es/m/2287835.1697464481@sss.pgh.pa.us
2023-10-16 15:45:01 -04:00
Tom Lane 54b208f909 Ensure we have a snapshot while dropping ON COMMIT DROP temp tables.
Dropping a temp table could entail TOAST table access to clean out
toasted catalog entries, such as large pg_constraint.conbin strings
for complex CHECK constraints.  If we did that via ON COMMIT DROP,
we triggered the assertion in init_toast_snapshot(), because
there was no provision for setting up a snapshot for the drop
actions.  Fix that.

(I assume here that the adjacent truncation actions for ON COMMIT
DELETE ROWS don't have a similar problem: it doesn't seem like
nontransactional truncations would need to touch any toasted fields.
If that proves wrong, we could refactor a bit to have the same
snapshot acquisition cover that too.)

The test case added here does not fail before v15, because that
assertion was added in 277692220 which was not back-patched.
However, the race condition the assertion warns of surely
exists further back, so back-patch to all supported branches.

Per report from Richard Guo.

Discussion: https://postgr.es/m/CAMbWs4-x26=_QxxgdJyNbiCDzvtr2WV5ZDso_v-CukKEe6cBZw@mail.gmail.com
2023-10-16 14:06:14 -04:00
Nathan Bossart 8fb13dd6ab Move extra code out of the Pre/PostRestoreCommand() section.
If SIGTERM is received within this section, the startup process
will immediately proc_exit() in the signal handler, so it is
inadvisable to include any more code than is required there (as
such code is unlikely to be compatible with doing proc_exit() in a
signal handler).  This commit moves the code recently added to this
section (see 1b06d7bac9 and 7fed801135) to outside of the section.
This ensures that the startup process only calls proc_exit() in its
SIGTERM handler for the duration of the system() call, which is how
this code worked from v8.4 to v14.

Reported-by: Michael Paquier, Thomas Munro
Analyzed-by: Andres Freund
Suggested-by: Tom Lane
Reviewed-by: Michael Paquier, Robert Haas, Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 15
2023-10-16 12:41:55 -05:00
Robert Haas a70bce43fb Update the documentation on recovering from (M)XID exhaustion.
The old documentation encourages entering single-user mode for no
reason, which is a bad plan in most cases. Instead, discourage users
from doing that, and explain the limited cases in which it may be
desirable.

The old documentation claims that running VACUUM as anyone but the
superuser can't possibly work, which is not really true, because it
might be that some other user has enough permissions to VACUUM all
the tables that matter. Weaken the language just a bit.

The old documentation claims that you can't run any commands
when near XID exhaustion, which is false because you can still
run commands that don't require an XID, like a SELECT without a
locking clause.

The old documentation doesn't clearly explain that it's a good idea
to get rid of prepared transactons, long-running transactions, and
replication slots that are preventing (M)XID horizon advancement.
Spell out the steps to do that.

Also, discourage the use of VACUUM FULL and VACUUM FREEZE in
this type of scenario.

Back-patch to v14. Much of this is good advice on all supported
versions, but before 60f1f09ff4
the chances of VACUUM failing in multi-user mode were much higher.

Alexander Alekseev, John Naylor, Robert Haas, reviewed at various
times by Peter Geoghegan, Hannu Krosing, and Andres Freund.

Discussion: http://postgr.es/m/CA+TgmoYtsUDrzaHcmjFhLzTk1VEv29mO_u-MT+XWHrBJ_4nD8A@mail.gmail.com
2023-10-16 12:57:39 -04:00
Alexander Korotkov 5abbd97fef List 006_login_trigger.pl test for meson
Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA%2BhUKGLuqDUaYYhJnA1H1q5Z-k18kQHoEqZ5fiXtTi4038nspg%40mail.gmail.com
2023-10-16 10:39:07 +03:00
Michael Paquier 4922173010 worker_spi: Fix test failure with BGWORKER_BYPASS_ROLELOGINCHECK
This is a consequence of 4817da51f6 that has bumped up
max_worker_processes, where now the last worker started by the test
would be able to start by itself a parallel worker because there are
more slots available.  This did not show up before as the number of
bgworkers reached exactly 8, as known as the previous limit, at the end
of the test.

Per report from buildfarm member crake, reproducible with
debug_parallel_query = regress in the same fashion as fd4d93d269.
2023-10-16 13:45:39 +09:00
Thomas Munro 63a582222c Try to handle torn reads of pg_control in frontend.
Some of our src/bin tools read the control file without any kind of
interlocking against concurrent writes from the server.  At least ext4
and ntfs can expose partially modified contents when you do that.

For now, we'll try to tolerate this by retrying up to 10 times if the
checksum doesn't match, until we get two reads in a row with the same
bad checksum.  This is not guaranteed to reach the right conclusion, but
it seems very likely to.  Thanks to Tom Lane for this suggestion.

Various ideas for interlocking or atomicity were considered too
complicated, unportable or expensive given the lack of field reports,
but remain open for future reconsideration.

Back-patch as far as 12.  It doesn't seem like a good idea to put a
heuristic change for a very rare problem into the final release of 11.

Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
2023-10-16 17:33:08 +13:00
Michael Paquier 4817da51f6 worker_spi: Bump up max_worker_processes in TAP tests
mamba has detected a failure in the last test that should start a
bgworker while bypassing the role login check.  The buildfarm did not
provide any information about its failure in the logs, but I suspect
that this is caused by an exhaustion of the max_worker_processes slots
set at 8 by default.

In "normal" test runs, the number of bgworkers running at this stage of
the test is already 7, so, if one of them spawns for example a parallel
worker all the slots would be taken, preventing the last worker of the
test to start.

Reviewed-by: Tom Lane
Discussion: https://postgr.es/m/ZSyebsiub88pyJJO@paquier.xyz
2023-10-16 13:07:36 +09:00
Alexander Korotkov 1f89b73c4e Rename 005_login_trigger.pl to 006_login_trigger.pl
In order to avoid numbering collision with 005_sspi.pl.
2023-10-16 04:01:45 +03:00
Alexander Korotkov 47ab5d2e2e Fix role names in src/test/authentication/t/005_login_trigger.pl
Per buildfarm member longfin.
2023-10-16 04:01:45 +03:00
Michael Paquier 20b7708662 Add a few recent commits to .git-blame-ignore-revs
Three commits need to be added compared to the last time this file was
updated:
- e9718b4bd3
- 01529c7040
- b6a77c6a6c
2023-10-16 09:41:59 +09:00
Michael Paquier e9718b4bd3 Fix code indentation violations in e83d1b0c40
koel has not reported this one yet, I have just bumped on it while
looking at a different patch.
2023-10-16 09:36:31 +09:00
Thomas Munro 01529c7040 Fix comment from commit 22655aa231.
Per automated complaint from BF animal koel this needed to be
re-indented, but there was also a typo.  Back-patch to 16.
2023-10-16 13:32:41 +13:00
Alexander Korotkov e83d1b0c40 Add support event triggers on authenticated login
This commit introduces trigger on login event, allowing to fire some actions
right on the user connection.  This can be useful for logging or connection
check purposes as well as for some personalization of environment.  Usage
details are described in the documentation included, but shortly usage is
the same as for other triggers: create function returning event_trigger and
then create event trigger on login event.

In order to prevent the connection time overhead when there are no triggers
the commit introduces pg_database.dathasloginevt flag, which indicates database
has active login triggers.  This flag is set by CREATE/ALTER EVENT TRIGGER
command, and unset at connection time when no active triggers found.

Author: Konstantin Knizhnik, Mikhail Gribkov
Discussion: https://postgr.es/m/0d46d29f-4558-3af9-9c85-7774e14a7709%40postgrespro.ru
Reviewed-by: Pavel Stehule, Takayuki Tsunakawa, Greg Nancarrow, Ivan Panchenko
Reviewed-by: Daniel Gustafsson, Teodor Sigaev, Robert Haas, Andres Freund
Reviewed-by: Tom Lane, Andrey Sokolov, Zhihong Yu, Sergey Shinderuk
Reviewed-by: Gregory Stark, Nikita Malakhov, Ted Yu
2023-10-16 03:18:22 +03:00
Thomas Munro c558e6fd92 Acquire ControlFileLock in relevant SQL functions.
Commit dc7d70ea added functions that read the control file, but didn't
acquire ControlFileLock.  With unlucky timing, file systems that have
weak interlocking like ext4 and ntfs could expose partially overwritten
contents, and the checksum would fail.

Back-patch to all supported releases.

Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
2023-10-16 10:43:47 +13:00