Commit Graph

56866 Commits

Author SHA1 Message Date
Michael Paquier
f76f1ef00d Log OpenSSL version in ./configure output
This information is useful to know when scanning buildfarm results, and
it is already displayed in Meson.  The output of `openssl version` is
logged, with the command retrieved from PATH.

This depends on c8e4030d1b, so backpatch down to 16.

Reviewed-by: Peter Eisentraut, Daniel Gustafsson, Tom Lane
Discussion: https://postgr.es/m/ZTW9yOlZaSVoFhTz@paquier.xyz
Backpatch-through: 16
2023-10-25 09:27:09 +09:00
Tom Lane
b1444a09dc Fix problems when a plain-inheritance parent table is excluded.
When an UPDATE/DELETE/MERGE's target table is an old-style
inheritance tree, it's possible for the parent to get excluded
from the plan while some children are not.  (I believe this is
only possible if we can prove that a CHECK ... NO INHERIT
constraint on the parent contradicts the query WHERE clause,
so it's a very unusual case.)  In such a case, ExecInitModifyTable
mistakenly concluded that the first surviving child is the target
table, leading to at least two bugs:

1. The wrong table's statement-level triggers would get fired.

2. In v16 and up, it was possible to fail with "invalid perminfoindex
0 in RTE with relid nnnn" due to the child RTE not having permissions
data included in the query plan.  This was hard to reproduce reliably
because it did not occur unless the update triggered some non-HOT
index updates.

In v14 and up, this is easy to fix by defining ModifyTable.rootRelation
to be the parent RTE in plain inheritance as well as partitioned cases.

While the wrong-triggers bug also appears in older branches, the
relevant code in both the planner and executor is quite a bit
different, so it would take a good deal of effort to develop and
test a suitable patch.  Given the lack of field complaints about the
trigger issue, I'll desist for now.  (Patching v11 for this seems
unwise anyway, given that it will have no more releases after next
month.)

Per bug #18147 from Hans Buschmann.

Amit Langote and Tom Lane

Discussion: https://postgr.es/m/18147-6fc796538913ee88@postgresql.org
2023-10-24 14:48:34 -04:00
Peter Geoghegan
cf89d3052d Doc: indexUnchanged is strictly a hint.
Clearly spell out the limitations of aminsert()'s indexUnchanged hinting
mechanism in the index AM documentation.

Oversight in commit 9dc718bd, which added the "logically unchanged
index" hint (which is used to trigger bottom-up index deletion).

Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-By: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAH2-WzmU_BQ=-H9L+bxTSMQBqHMjp1DSwGypvL0gKs+dTOfkKg@mail.gmail.com
Backpatch: 14-, where indexUnchanged hinting was introduced.
2023-10-24 09:27:26 -07:00
Thomas Munro
3b0f20f6ce Log LLVM library version in configure output.
When scanning build farm results, it's useful to be able to see which
version is in use.  For the Meson build system, this information was
already displayed.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/4022690.1697852728%40sss.pgh.pa.us
2023-10-22 14:23:11 +13:00
Thomas Munro
174ccda5e2 Fix min_dynamic_shared_memory on Windows.
When min_dynamic_shared_memory is set above 0, we try to find space in a
pre-allocated region of the main shared memory area instead of calling
dsm_impl_XXX() routines to allocate more.  The dsm_pin_segment() and
dsm_unpin_segment() routines had a bug: they called dsm_impl_XXX()
routines even for main region segments.  Nobody noticed before now
because those routines do nothing on Unix, but on Windows they'd fail
while attempting to duplicate an invalid Windows HANDLE.  Add the
missing gating.

Back-patch to 14, where commit 84b1c63a added this feature.  Fixes
pgsql-bugs bug #18165.

Reported-by: Maxime Boyer <maxime.boyer@cra-arc.gc.ca>
Tested-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/18165-bf4f525cea6e51de%40postgresql.org
2023-10-22 10:05:22 +13:00
Alvaro Herrera
b62a3fa1cc
Make some error strings more generic
It's undesirable to have SQL commands or configuration options in a
translatable error string, so take some of these out.
2023-10-20 22:52:15 +02:00
Andres Freund
c79ca0485a meson: Make detection of python more robust
Previously we errored out if no python installation could be found (but we did
handle not having enough of python installed to build plpython
against). Presumably nobody hit this so far, as python is likely installed due
to meson requiring python.

Author: Tristan Partin <tristan@neon.tech>
Discussion: https://postgr.es/m/CSPIJVUDZFKX.3KHMOAVGF94RV@c3po
Backpatch: 16-, where meson support was added
2023-10-20 11:11:36 -07:00
Tom Lane
1ad3a7bd39 Dodge a compiler bug affecting timetz_zone/timetz_izone.
This avoids a compiler bug occurring in AIX's xlc, even in pretty
late-model revisions.  Buildfarm testing has now confirmed that
only 64-bit xlc is affected.  Although we are contemplating
dropping support for xlc in v17, it's still supported in the
back branches, so we need this fix.

Back-patch of code changes from HEAD commit 19fa97731.
(The test cases were already back-patched, in 4a427b82c et al.)

Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
2023-10-20 13:40:15 -04:00
Tom Lane
1856a0ee0e Doc: update CREATE OPERATOR's statement about => as an operator.
This doco said that use of => as an operator "is deprecated".
It's been fully disallowed since 865f14a2d back in 9.5, but
evidently that commit missed updating this statement.
Do so now.
2023-10-20 13:01:24 -04:00
Peter Eisentraut
8405068f3e Remove duplicate name from list of acknowledgments
Reported-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
2023-10-19 10:37:46 +02:00
Tom Lane
cfa4eba02c Improve pglz_decompress's defenses against corrupt compressed data.
When processing a match tag, check to see if the claimed "off"
is more than the distance back to the output buffer start.
If it is, then the data is corrupt, and what's more we would
fetch from outside the buffer boundaries and potentially incur
a SIGSEGV.  (Although the odds of that seem relatively low, given
that "off" can't be more than 4K.)

Back-patch to v13; before that, this function wasn't really
trying to protect against bad data.

Report and fix by Flavien Guedez.

Discussion: https://postgr.es/m/01fc0593-e31e-463d-902c-dd43174acee2@oopacity.net
2023-10-18 20:43:17 -04:00
Bruce Momjian
418d6b4430 doc: PG 16 relnotes: fix spelling error
Reported-by: Lele Gaifax

Discussion: https://postgr.es/m/87cyximsps.fsf@metapensiero.it

Backpatch-through: 16 only
2023-10-18 13:40:44 -04:00
Thomas Munro
7741850568 jit: Changes for LLVM 17.
Changes required by https://llvm.org/docs/NewPassManager.html.

Back-patch to 12, leaving the final release of 11 unchanged, consistent
with earlier decision not to back-patch LLVM 16 support either.

Author: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKG%2BWXznXCyTgCADd%3DHWkP9Qksa6chd7L%3DGCnZo-MBgg9Lg%40mail.gmail.com
2023-10-19 05:14:07 +13:00
Thomas Munro
60596f148a jit: Supply LLVMGlobalGetValueType() for LLVM < 8.
Commit 37d5babb used this C API function while adding support for LLVM
16 and opaque pointers, but it's not available in LLVM 7 and older.
Provide it in our own llvmjit_wrap.cpp.  It just calls a C++ function
that pre-dates LLVM 3.9, our minimum target.

Back-patch to 12, like 37d5babb.

Discussion: https://postgr.es/m/CA%2BhUKGKnLnJnWrkr%3D4mSGhE5FuTK55FY15uULR7%3Dzzc%3DwX4Nqw%40mail.gmail.com
2023-10-19 03:07:18 +13:00
Thomas Munro
74d19ec096 jit: Support opaque pointers in LLVM 16.
Remove use of LLVMGetElementType() and provide the type of all pointers
to LLVMBuildXXX() functions when emitting IR, as required by modern LLVM
versions[1].

 * For LLVM <= 14, we'll still use the old LLVMBuildXXX() functions.
 * For LLVM == 15, we'll continue to do the same, explicitly opting
   out of opaque pointer mode.
 * For LLVM >= 16, we'll use the new LLVMBuildXXX2() functions that take
   the extra type argument.

The difference is hidden behind some new IR emitting wrapper functions
l_load(), l_gep(), l_call() etc.  The change is mostly mechanical,
except that at each site the correct type had to be provided.

In some places we needed to do some extra work to get functions types,
including some new wrappers for C++ APIs that are not yet exposed by in
LLVM's C API, and some new "example" functions in llvmjit_types.c
because it's no longer possible to start from the function pointer type
and ask for the function type.

Back-patch to 12, because it's a little tricker in 11 and we agreed not
to put the latest LLVM support into the upcoming final release of 11.

[1] https://llvm.org/docs/OpaquePointers.html

Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Ronan Dunklau <ronan.dunklau@aiven.io>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKGKNX_%3Df%2B1C4r06WETKTq0G4Z_7q4L4Fxn5WWpMycDj9Fw%40mail.gmail.com
2023-10-18 22:53:56 +13:00
Michael Paquier
4277bcb301 pg_upgrade: Fix test name in 002_pg_upgrade.pl
Author: Hou Zhijie
Discussion: https://postgr.es/m/TYAPR01MB5724A40D47E71F4717357EC694D5A@TYAPR01MB5724.jpnprd01.prod.outlook.com
Backpatch-through: 15
2023-10-18 18:23:43 +09:00
Michael Paquier
2308f18c07 Count write times when extending relation files for shared buffers
Relation files extended multiple blocks at a time have been counting the
number of blocks written, but forgot that to increment the write time in
this case, as write and relation extension are treated as two different
I/O operations in the shared stats: IOOP_EXTEND vs IOOP_WRITE.  In this
case IOOP_EXTEND was forgotten for normal (non-temporary) relations.

Write times are tracked when track_io_timing is enabled, which is not
the case by default.

Author: Nazir Bilal Yavuz
Reviewed-by: Robert Haas, Melanie Plageman
Discussion: https://postgr.es/m/CAN55FZ19Ss279mZuqGbuUNxka0iPbLgYuOQXqAKewrjNrp27VA@mail.gmail.com
Backpatch-through: 16
2023-10-18 14:54:39 +09:00
Tom Lane
4a427b82cf Back-patch test cases for timetz_zone/timetz_izone.
Per code coverage reports, we had zero regression test coverage
of these functions.  That came back to bite us, as apparently
that's allowed us to miss discovering misbehavior of this code
with AIX's xlc compiler.  Install relevant portions of the
test cases added in 97957fdba, 2f0472030, 19fa97731.

(Assuming the expected outcome that the xlc problem does appear
in back branches, a code fix will follow.)

Discussion: https://postgr.es/m/CA+hUKGK=DOC+hE-62FKfZy=Ybt5uLkrg3zCZD-jFykM-iPn8yw@mail.gmail.com
2023-10-17 13:55:45 -04:00
Nathan Bossart
ee06199fcb Avoid calling proc_exit() in processes forked by system().
The SIGTERM handler for the startup process immediately calls
proc_exit() for the duration of the restore_command, i.e., a call
to system().  This system() call forks a new process to execute the
shell command, and this child process inherits the parent's signal
handlers.  If both the parent and child processes receive SIGTERM,
both will attempt to call proc_exit().  This can end badly.  For
example, both processes will try to remove themselves from the
PGPROC shared array.

To fix this problem, this commit adds a check in
StartupProcShutdownHandler() to see whether MyProcPid == getpid().
If they match, this is the parent process, and we can proc_exit()
like before.  If they do not match, this is a child process, and we
just emit a message to STDERR (in a signal safe manner) and
_exit(), thereby skipping any problematic exit callbacks.

This commit also adds checks in proc_exit(), ProcKill(), and
AuxiliaryProcKill() that verify they are not being called within
such child processes.

Suggested-by: Andres Freund
Reviewed-by: Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 11
2023-10-17 10:41:58 -05:00
Tom Lane
57e6e861d4 Ensure we have a snapshot while dropping ON COMMIT DROP temp tables.
Dropping a temp table could entail TOAST table access to clean out
toasted catalog entries, such as large pg_constraint.conbin strings
for complex CHECK constraints.  If we did that via ON COMMIT DROP,
we triggered the assertion in init_toast_snapshot(), because
there was no provision for setting up a snapshot for the drop
actions.  Fix that.

(I assume here that the adjacent truncation actions for ON COMMIT
DELETE ROWS don't have a similar problem: it doesn't seem like
nontransactional truncations would need to touch any toasted fields.
If that proves wrong, we could refactor a bit to have the same
snapshot acquisition cover that too.)

The test case added here does not fail before v15, because that
assertion was added in 277692220 which was not back-patched.
However, the race condition the assertion warns of surely
exists further back, so back-patch to all supported branches.

Per report from Richard Guo.

Discussion: https://postgr.es/m/CAMbWs4-x26=_QxxgdJyNbiCDzvtr2WV5ZDso_v-CukKEe6cBZw@mail.gmail.com
2023-10-16 14:06:15 -04:00
Nathan Bossart
d1c56ad37b Move extra code out of the Pre/PostRestoreCommand() section.
If SIGTERM is received within this section, the startup process
will immediately proc_exit() in the signal handler, so it is
inadvisable to include any more code than is required there (as
such code is unlikely to be compatible with doing proc_exit() in a
signal handler).  This commit moves the code recently added to this
section (see 1b06d7bac9 and 7fed801135) to outside of the section.
This ensures that the startup process only calls proc_exit() in its
SIGTERM handler for the duration of the system() call, which is how
this code worked from v8.4 to v14.

Reported-by: Michael Paquier, Thomas Munro
Analyzed-by: Andres Freund
Suggested-by: Tom Lane
Reviewed-by: Michael Paquier, Robert Haas, Thomas Munro, Andres Freund
Discussion: https://postgr.es/m/Y9nGDSgIm83FHcad%40paquier.xyz
Discussion: https://postgr.es/m/20230223231503.GA743455%40nathanxps13
Backpatch-through: 15
2023-10-16 12:42:17 -05:00
Robert Haas
08c1795c52 Update the documentation on recovering from (M)XID exhaustion.
The old documentation encourages entering single-user mode for no
reason, which is a bad plan in most cases. Instead, discourage users
from doing that, and explain the limited cases in which it may be
desirable.

The old documentation claims that running VACUUM as anyone but the
superuser can't possibly work, which is not really true, because it
might be that some other user has enough permissions to VACUUM all
the tables that matter. Weaken the language just a bit.

The old documentation claims that you can't run any commands
when near XID exhaustion, which is false because you can still
run commands that don't require an XID, like a SELECT without a
locking clause.

The old documentation doesn't clearly explain that it's a good idea
to get rid of prepared transactons, long-running transactions, and
replication slots that are preventing (M)XID horizon advancement.
Spell out the steps to do that.

Also, discourage the use of VACUUM FULL and VACUUM FREEZE in
this type of scenario.

Back-patch to v14. Much of this is good advice on all supported
versions, but before 60f1f09ff4
the chances of VACUUM failing in multi-user mode were much higher.

Alexander Alekseev, John Naylor, Robert Haas, reviewed at various
times by Peter Geoghegan, Hannu Krosing, and Andres Freund.

Discussion: http://postgr.es/m/CA+TgmoYtsUDrzaHcmjFhLzTk1VEv29mO_u-MT+XWHrBJ_4nD8A@mail.gmail.com
2023-10-16 13:05:07 -04:00
Thomas Munro
5725e4ebe7 Try to handle torn reads of pg_control in frontend.
Some of our src/bin tools read the control file without any kind of
interlocking against concurrent writes from the server.  At least ext4
and ntfs can expose partially modified contents when you do that.

For now, we'll try to tolerate this by retrying up to 10 times if the
checksum doesn't match, until we get two reads in a row with the same
bad checksum.  This is not guaranteed to reach the right conclusion, but
it seems very likely to.  Thanks to Tom Lane for this suggestion.

Various ideas for interlocking or atomicity were considered too
complicated, unportable or expensive given the lack of field reports,
but remain open for future reconsideration.

Back-patch as far as 12.  It doesn't seem like a good idea to put a
heuristic change for a very rare problem into the final release of 11.

Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
2023-10-16 17:21:41 +13:00
Thomas Munro
3090213690 Fix comment from commit 22655aa231.
Per automated complaint from BF animal koel this needed to be
re-indented, but there was also a typo.  Back-patch to 16.
2023-10-16 13:32:55 +13:00
Thomas Munro
2371432cd6 Acquire ControlFileLock in relevant SQL functions.
Commit dc7d70ea added functions that read the control file, but didn't
acquire ControlFileLock.  With unlucky timing, file systems that have
weak interlocking like ext4 and ntfs could expose partially overwritten
contents, and the checksum would fail.

Back-patch to all supported releases.

Reviewed-by: David Steele <david@pgmasters.net>
Reviewed-by: Anton A. Melnikov <aamelnikov@inbox.ru>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20221123014224.xisi44byq3cf5psi%40awork3.anarazel.de
2023-10-16 10:45:05 +13:00
Noah Misch
bf1c21c4fa Dissociate btequalimage() from interval_ops, ending its deduplication.
Under interval_ops, some equal values are distinguishable.  One such
pair is '24:00:00' and '1 day'.  With that being so, btequalimage()
breaches the documented contract for the "equalimage" btree support
function.  This can cause incorrect results from index-only scans.
Users should REINDEX any btree indexes having interval-type columns.
After updating, pg_amcheck will report an error for almost all such
indexes.  This fix makes interval_ops simply omit the support function,
like numeric_ops does.  Back-pack to v13, where btequalimage() first
appeared.  In back branches, for the benefit of old catalog content,
btequalimage() code will return false for type "interval".  Going
forward, back-branch initdb will include the catalog change.

Reviewed by Peter Geoghegan.

Discussion: https://postgr.es/m/20231011013317.22.nmisch@google.com
2023-10-14 16:33:54 -07:00
Noah Misch
06ff064842 Don't spuriously report FD_SETSIZE exhaustion on Windows.
Starting on 2023-08-03, this intermittently terminated a "pgbench -C"
test in CI.  It could affect a high-client-count "pgbench" without "-C".
While parallel reindexdb and vacuumdb reach the same problematic check,
sufficient client count and/or connection turnover is less plausible for
them.  Given the lack of examples from the buildfarm or from manual
builds, reproducing this must entail rare operating system
configurations.  Also correct the associated error message, which was
wrong for non-Windows.  Back-patch to v12, where the pgbench check first
appeared.  While v11 vacuumdb has the problematic check, reaching it
with typical vacuumdb usage is implausible.

Reviewed by Thomas Munro.

Discussion: https://postgr.es/m/CA+hUKG+JwvTNdcyJTriy9BbtzF1veSRQ=9M_ZKFn9_LqE7Kp7Q@mail.gmail.com
2023-10-14 15:54:49 -07:00
Andres Freund
0002feb820 Fix bulk table extension when copying into multiple partitions
When COPYing into a partitioned table that does now permit the use of
table_multi_insert(), we could error out with
  ERROR: could not read block NN in file "base/...": read only 0 of 8192 bytes

because BulkInsertState->next_free was not reset between partitions. This
problem occurred only when not able to use table_multi_insert(), as a
dedicated BulkInsertState for each partition is used in that case.

The bug was introduced in 00d1e02be2, but it was hard to hit at that point,
as commonly bulk relation extension is not used when not using
table_multi_insert(). It became more likely after 82a4edabd2, which expanded
the use of bulk extension.

To fix the bug, reset the bulk relation extension state in BulkInsertState in
ReleaseBulkInsertStatePin(). That was added (in b1ecb9b3fc) to tackle a very
similar issue.  Obviously the name is not quite correct, but there might be
external callers, and bulk insert state needs to be reset in precisely in the
situations that ReleaseBulkInsertStatePin() already needed to be called.

Medium term the better fix likely is to disallow reusing BulkInsertState
across relations.

Add a test that, without the fix, reproduces #18130 in most
configurations. The test also catches the problem fixed in b1ecb9b3fc when
run with small shared_buffers.

Reported-by: Ivan Kolombet <enderstd@gmail.com>
Analyzed-by: Tom Lane <tgl@sss.pgh.pa.us>
Analyzed-by: Andres Freund <andres@anarazel.de>
Bug: #18130
Discussion: https://postgr.es/m/18130-7a86a7356a75209d%40postgresql.org
Discussion: https://postgr.es/m/257696.1695670946%40sss.pgh.pa.us
Backpatch: 16-
2023-10-13 19:17:28 -07:00
David Rowley
595db9e9cc Fix runtime partition pruning for HASH partitioned tables
This could only affect HASH partitioned tables with at least 2 partition
key columns.

If partition pruning was delayed until execution and the query contained
an IS NULL qual on one of the partitioned keys, and some subsequent
partitioned key was being compared to a non-Const, then this could result
in a crash due to the incorrect keyno being used to calculate the
stateidx for the expression evaluation code.

Here we fix this by properly skipping partitioned keys which have a
nullkey set.  Effectively, this must be the same as what's going on
inside perform_pruning_base_step().

Sergei Glukhov also provided a patch, but that's not what's being used
here.

Reported-by: Sergei Glukhov
Reviewed-by: tender wang, Sergei Glukhov
Discussion: https://postgr.es/m/d05b26fa-af54-27e1-f693-6c31590802fa@postgrespro.ru
Backpatch-through: 11, where runtime partition pruning was added.
2023-10-13 01:13:07 +13:00
David Rowley
f114e16f6c Doc: fix grammatical errors for enable_partitionwise_aggregate
Author: Andrew Atkinson
Reviewed-by: Ashutosh Bapat
Discussion: https://postgr.es/m/CAG6XLEnC%3DEgq0YHRic2kWWDs4xwQnQ_kBA6qhhzAq1-pO_9Tfw%40mail.gmail.com
Backpatch-through: 11, where enable_partitionwise_aggregate was added
2023-10-12 21:15:58 +13:00
David Rowley
6352f16276 Fix incorrect step generation in HASH partition pruning
get_steps_using_prefix_recurse() incorrectly assumed that it could stop
recursive processing of the 'prefix' list when cur_keyno was one before
the step_lastkeyno.  Since hash partition pruning can prune using IS
NULL quals, and these IS NULL quals are not present in the 'prefix'
list, then that logic could cause more levels of recursion than what is
needed and lead to there being no more items in the 'prefix' list to
process.  This would manifest itself as a crash in some code that
expected the 'start' ListCell not to be NULL.

Here we adjust the logic so that instead of stopping recursion at 1 key
before the step_lastkeyno, we just look at the llast(prefix) item and
ensure we only recursively process up until just before whichever the last
key is.  This effectively allows keys to be missing in the 'prefix' list.

This change does mean that step_lastkeyno is no longer needed, so we
remove that from the static functions.  I also spent quite some time
reading this code and testing it to try to convince myself that there
are no other issues.  That resulted in the irresistible temptation of
rewriting some comments, many of which were just not true or inconcise.

Reported-by: Sergei Glukhov
Reviewed-by: Sergei Glukhov, tender wang
Discussion: https://postgr.es/m/2f09ce72-315e-2a33-589a-8519ada8df61@postgrespro.ru
Backpatch-through: 11, where partition pruning was introduced.
2023-10-12 19:51:26 +13:00
Bruce Momjian
4ac7635fdb doc: pg_upgrade: use dynamic new cluster major version numbers
Also update docs to use more recent old version numbers

Reported-by: mark.a.sloan@gmail.com

Discussion: https://postgr.es/m/169506804412.3727336.8571753495127355296@wrigleys.postgresql.org

Backpatch-through: 16
2023-10-10 17:12:00 -04:00
Bruce Momjian
608ccdbe25 doc: clarify that SSPI and GSSAPI are interchangeable
Reported-by: tpo_deb@sourcepole.ch

Discussion: https://postgr.es/m/167846222574.1803490.15815104179136215862@wrigleys.postgresql.org

Backpatch-through: 11
2023-10-10 16:51:08 -04:00
Bruce Momjian
17fff5a6ef doc: foreign servers with pushdown need matching collation
Reported-by: Pete Storer

Discussion: https://postgr.es/m/BL0PR05MB66283C57D72E321591AE4EB1F3CE9@BL0PR05MB6628.namprd05.prod.outlook.com

Backpatch-through: 11
2023-10-10 16:04:56 -04:00
Bruce Momjian
61fcc9d440 doc: add SSL configuration section reference
Reported-by: Steve Atkins

Discussion: https://postgr.es/m/B82E80DD-1452-4175-B19C-564FE46705BA@blighty.com

Backpatch-through: 11
2023-10-10 15:54:28 -04:00
Bruce Momjian
b01a5e71d8 doc: clarify how the bootstrap user name is chosen
Discussion: https://postgr.es/m/167931662853.3349090.18217722739345182859@wrigleys.postgresql.org

Backpatch-through: 16
2023-10-10 15:27:26 -04:00
Bruce Momjian
63e7cac019 doc: document the need to analyze partitioned tables
Autovacuum does not do it.

Reported-by: Justin Pryzby

Discussion: https://postgr.es/m/20210913035409.GA10647@telsasoft.com

Backpatch-through: 11
2023-10-10 15:14:18 -04:00
Jeff Davis
b8963e8a28 Fix bug in GenericXLogFinish().
Mark the buffers dirty before writing WAL.

Discussion: https://postgr.es/m/25104133-7df8-cae3-b9a2-1c0aaa1c094a@iki.fi
Reviewed-by: Heikki Linnakangas
Backpatch-through: 11
2023-10-10 11:01:55 -07:00
Tom Lane
e943cff33d Doc: use CURRENT_USER not USER in plpgsql trigger examples.
While these two built-in functions do exactly the same thing,
CURRENT_USER seems preferable to use in documentation examples.
It's easier to look up if the reader is unsure what it is.
Also, this puts these examples in sync with an adjacent example
that already used CURRENT_USER.

Per question from Kirk Parker.

Discussion: https://postgr.es/m/CANwZ8rmN_Eb0h0hoMRS8Feftaik0z89PxVsKg+cP+PctuOq=Qg@mail.gmail.com
2023-10-09 11:29:36 -04:00
David Rowley
9154ededfc Strip off ORDER BY/DISTINCT aggregate pathkeys in create_agg_path
1349d2790 added code to adjust the PlannerInfo.group_pathkeys so that
ORDER BY / DISTINCT aggregate functions could obtain pre-sorted inputs
to allow faster execution.  That commit forgot to adjust the pathkeys in
create_agg_path().  Some code in there assumed that it was always fine
to make the AggPath's pathkeys the same as its subpath's.  That seems to
have been ok up until 1349d2790, but since that commit adds pathkeys for
columns which are within the aggregate function, those columns won't be
available above the aggregate node.  This can result in "could not find
pathkey item to sort" during create_plan().

The fix here is to strip off the additional pathkeys added by
adjust_group_pathkeys_for_groupagg().  It seems that the pathkeys here
will only ever be group_pathkeys, so all we need to do is check if the
length of the pathkey list is longer than the num_groupby_pathkeys and
get rid of the additional ones only if we see extras.

Reported-by: Justin Pryzby
Reviewed-by: Richard Guo
Discussion: https://postgr.es/m/ZQhYYRhUxpW3PSf9%40telsasoft.com
Backpatch-through: 16, where 1349d2790 was introduced
2023-10-09 16:37:33 +13:00
Etsuro Fujita
3ed98ed88a Remove extra parenthesis from comment. 2023-10-06 18:30:01 +09:00
David Rowley
31b2b2d72d Fix memory leak in Memoize code
Ensure we switch to the per-tuple memory context to prevent any memory
leaks of detoasted Datums in MemoizeHash_hash() and MemoizeHash_equal().

Reported-by: Orlov Aleksej
Author: Orlov Aleksej, David Rowley
Discussion: https://postgr.es/m/83281eed63c74e4f940317186372abfd%40cft.ru
Backpatch-through: 14, where Memoize was added
2023-10-05 20:31:25 +13:00
Michael Paquier
75f31a3f26 Avoid memory size overflow when allocating backend activity buffer
The code in charge of copying the contents of PgBackendStatus to local
memory could fail on memory allocation because of an overflow on the
amount of memory to use.  The overflow can happen when combining a high
value track_activity_query_size (max at 1MB) with a large
max_connections, when both multiplied get higher than INT32_MAX as both
parameters treated as signed integers.  This could for example trigger
with the following functions, all calling pgstat_read_current_status():
- pg_stat_get_backend_subxact()
- pg_stat_get_backend_idset()
- pg_stat_get_progress_info()
- pg_stat_get_activity()
- pg_stat_get_db_numbackends()

The change to use MemoryContextAllocHuge() has been introduced in
8d0ddccec6, so backpatch down to 12.

Author: Jakub Wartak
Discussion: https://postgr.es/m/CAKZiRmw8QSNVw2qNK-dznsatQqz+9DkCquxP0GHbbv1jMkGHMA@mail.gmail.com
Backpatch-through: 12
2023-10-03 15:37:18 +09:00
Michael Paquier
a06efbc3ac Fail hard on out-of-memory failures in xlogreader.c
This commit changes the WAL reader routines so as a FATAL for the
backend or exit(FAILURE) for the frontend is triggered if an allocation
for a WAL record decode fails in walreader.c, rather than treating this
case as bogus data, which would be equivalent to the end of WAL.  The
key is to avoid palloc_extended(MCXT_ALLOC_NO_OOM) in walreader.c,
relying on plain palloc() calls.

The previous behavior could make WAL replay finish too early than it
should.  For example, crash recovery finishing earlier may corrupt
clusters because not all the WAL available locally was replayed to
ensure a consistent state.  Out-of-memory failures would show up
randomly depending on the memory pressure on the host, but one simple
case would be to generate a large record, then replay this record after
downsizing a host, as Ethan Mertz originally reported.

This relies on bae868caf2, as the WAL reader routines now do the
memory allocation required for a record only once its header has been
fully read and validated, making xl_tot_len trustable.  Making the WAL
reader react differently on out-of-memory or bogus record data would
require ABI changes, so this is the safest choice for stable branches.
Also, it is worth noting that 3f1ce97346 has been using a plain
palloc() in this code for some time now.

Thanks to Noah Misch and Thomas Munro for the discussion.

Like the other commit, backpatch down to 12, leaving out v11 that will
be EOL'd soon.  The behavior of considering a failed allocation as bogus
data comes originally from 0ffe11abd3, where the record length
retrieved from its header was not entirely trustable.

Reported-by: Ethan Mertz
Discussion: https://postgr.es/m/ZRKKdI5-RRlta3aF@paquier.xyz
Backpatch-through: 12
2023-10-03 10:25:12 +09:00
Tom Lane
aaaf8fbb67 Fix omission of column-level privileges in selective pg_restore.
In a selective restore, ACLs for a table should be dumped if the
table is selected to be dumped.  However, if the table has both
table-level and column-level ACLs, only the table-level ACL was
restored.  This happened because _tocEntryRequired assumed that
an ACL could have only one dependency (the one on its table),
and punted if there was more than one.  But since commit ea9125304,
column-level ACLs also depend on the table-level ACL if any, to
ensure correct ordering in parallel restores.  To fix, adjust the
logic in _tocEntryRequired to ignore dependencies on ACLs.

I extended a test case in 002_pg_dump.pl so that it purports to
test for this; but in fact the test passes even without the fix.
That's because this bug only manifests during a selective restore,
while the scenarios 002_pg_dump.pl tests include only selective dumps.
Perhaps somebody would like to extend the script so that it can test
scenarios including selective restore, but I'm not touching that.

Euler Taveira and Tom Lane, per report from Kong Man.
Back-patch to all supported branches.

Discussion: https://postgr.es/m/DM4PR11MB73976902DBBA10B1D652F9498B06A@DM4PR11MB7397.namprd11.prod.outlook.com
2023-10-02 13:27:51 -04:00
Heikki Linnakangas
4a97a43a73 Flush WAL stats in bgwriter
bgwriter can write out WAL, but did not flush the WAL pgstat counters,
so the writes were not seen in pg_stat_wal.

Back-patch to v14, where pg_stat_wal was introduced.

Author: Nazir Bilal Yavuz
Reviewed-by: Matthias van de Meent, Kyotaro Horiguchi
Discussion: https://www.postgresql.org/message-id/CAN55FZ2FPYngovZstr%3D3w1KSEHe6toiZwrurbhspfkXe5UDocg%40mail.gmail.com
2023-10-02 12:49:25 +03:00
Tom Lane
5c34a7374c Fix datalen calculation in tsvectorrecv().
After receiving position data for a lexeme, tsvectorrecv()
advanced its "datalen" value by (npos+1)*sizeof(WordEntry)
where the correct calculation is (npos+1)*sizeof(WordEntryPos).
This accidentally failed to render the constructed tsvector
invalid, but it did result in leaving some wasted space
approximately equal to the space consumed by the position data.
That could have several bad effects:

* Disk space is wasted if the received tsvector is stored into a
  table as-is.

* A legal tsvector could get rejected with "maximum total lexeme
  length exceeded" if the extra space pushes it over the MAXSTRPOS
  limit.

* In edge cases, the finished tsvector could be assigned a length
  larger than the allocated size of its palloc chunk, conceivably
  leading to SIGSEGV when the tsvector gets copied somewhere else.
  The odds of a field failure of this sort seem low, though valgrind
  testing could probably have found this.

While we're here, let's express the calculation as
"sizeof(uint16) + npos * sizeof(WordEntryPos)" to avoid the type
pun implicit in the "npos + 1" formulation.  It's not wrong
given that WordEntryPos had better be 2 bytes to avoid padding
problems, but it seems clearer this way.

Report and patch by Denis Erokhin.  Back-patch to all supported
versions.

Discussion: https://postgr.es/m/009801d9f2d9$f29730c0$d7c59240$@datagile.ru
2023-10-01 13:16:59 -04:00
Tom Lane
ea0e7cd6b8 In COPY FROM, fail cleanly when unsupported encoding conversion is needed.
In recent releases, such cases fail with "cache lookup failed for
function 0" rather than complaining that the conversion function
doesn't exist as prior versions did.  Seems to be a consequence of
sloppy refactoring in commit f82de5c46.  Add the missing error check.

Per report from Pierre Fortin.  Back-patch to v14 where the
oversight crept in.

Discussion: https://postgr.es/m/20230929163739.3bea46e5.pfortin@pfortin.com
2023-10-01 12:09:26 -04:00
Andrew Dunstan
910eb61b2b Only evaluate default values as required when doing COPY FROM
Commit 9f8377f7a2 was a little too eager in fetching default values.
Normally this would not matter, but if the default value is not valid
for the type (e.g. a varchar that's too long) it caused an unnecessary
error.

Complaint and fix from Laurenz Albe

Backpatch to release 16.

Discussion: https://postgr.es/m/75a7b7483aeb331aa017328d606d568fc715b90d.camel@cybertec.at
2023-10-01 10:25:33 -04:00
Andres Freund
d5c5312dc8 meson: macos: Correct -exported_symbols_list syntax for Sonoma compat
-exported_symbols_list=... works on Ventura and earlier, but not on
Sonoma. The easiest way to fix it is to -Wl,-exported_symbols_list,@0@ which
actually seems more appropriate anyway, it's obviously a linker argument. It
is easier to use the -Wl,, syntax than passing multiple arguments, due to the
way the export_fmt is used (a single string that's formatted), but if it turns
out to be necessary, we can go for multiple arguments as well.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20230928222248.jw6s7yktpfsfczha@alap3.anarazel.de
Backpatch: 16-, where the meson based buildsystem was added
2023-09-30 12:10:52 -07:00