Commit Graph

57121 Commits

Author SHA1 Message Date
Heikki Linnakangas b8bff07daa Make ResourceOwners more easily extensible.
Instead of having a separate array/hash for each resource kind, use a
single array and hash to hold all kinds of resources. This makes it
possible to introduce new resource "kinds" without having to modify
the ResourceOwnerData struct. In particular, this makes it possible
for extensions to register custom resource kinds.

The old approach was to have a small array of resources of each kind,
and if it fills up, switch to a hash table. The new approach also uses
an array and a hash, but now the array and the hash are used at the
same time. The array is used to hold the recently added resources, and
when it fills up, they are moved to the hash. This keeps the access to
recent entries fast, even when there are a lot of long-held resources.

All the resource-specific ResourceOwnerEnlarge*(),
ResourceOwnerRemember*(), and ResourceOwnerForget*() functions have
been replaced with three generic functions that take resource kind as
argument. For convenience, we still define resource-specific wrapper
macros around the generic functions with the old names, but they are
now defined in the source files that use those resource kinds.

The release callback no longer needs to call ResourceOwnerForget on
the resource being released. ResourceOwnerRelease unregisters the
resource from the owner before calling the callback. That needed some
changes in bufmgr.c and some other files, where releasing the
resources previously always called ResourceOwnerForget.

Each resource kind specifies a release priority, and
ResourceOwnerReleaseAll releases the resources in priority order. To
make that possible, we have to restrict what you can do between
phases. After calling ResourceOwnerRelease(), you are no longer
allowed to remember any more resources in it or to forget any
previously remembered resources by calling ResourceOwnerForget.  There
was one case where that was done previously. At subtransaction commit,
AtEOSubXact_Inval() would handle the invalidation messages and call
RelationFlushRelation(), which temporarily increased the reference
count on the relation being flushed. We now switch to the parent
subtransaction's resource owner before calling AtEOSubXact_Inval(), so
that there is a valid ResourceOwner to temporarily hold that relcache
reference.

Other end-of-xact routines make similar calls to AtEOXact_Inval()
between release phases, but I didn't see any regression test failures
from those, so I'm not sure if they could reach a codepath that needs
remembering extra resources.

There were two exceptions to how the resource leak WARNINGs on commit
were printed previously: llvmjit silently released the context without
printing the warning, and a leaked buffer io triggered a PANIC. Now
everything prints a WARNING, including those cases.

Add tests in src/test/modules/test_resowner.

Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 13:30:50 +02:00
Heikki Linnakangas b70c2143bb Move a few ResourceOwnerEnlarge() calls for safety and clarity.
These are functions where a lot of things happen between the
ResourceOwnerEnlarge and ResourceOwnerRemember calls. It's important
that there are no unrelated ResourceOwnerRemember calls in the code in
between, otherwise the reserved entry might be used up by the
intervening ResourceOwnerRemember and not be available at the intended
ResourceOwnerRemember call anymore. I don't see any bugs here, but the
longer the code path between the calls is, the harder it is to verify.

In bufmgr.c, there is a function similar to ResourceOwnerEnlarge,
ReservePrivateRefCountEntry(), to ensure that the private refcount
array has enough space. The ReservePrivateRefCountEntry() calls were
made at different places than the ResourceOwnerEnlargeBuffers()
calls. Move the ResourceOwnerEnlargeBuffers() and
ReservePrivateRefCountEntry() calls together for consistency.

Reviewed-by: Aleksander Alekseev, Michael Paquier, Julien Rouhaud
Reviewed-by: Kyotaro Horiguchi, Hayato Kuroda, Álvaro Herrera, Zhihong Yu
Reviewed-by: Peter Eisentraut, Andres Freund
Discussion: https://www.postgresql.org/message-id/cbfabeb0-cd3c-e951-a572-19b365ed314d%40iki.fi
2023-11-08 13:30:46 +02:00
Peter Eisentraut e9f075f9a1 Don't install ldap_password_func in meson
It should be handled as a test module per commit b6a0d469ca.
2023-11-08 11:27:28 +01:00
Michael Paquier b64c8b01c2 Fix use of OPENSSL in SSL tests if command is not found
`openssl` is an optional dependency in the meson build as it may not be
installed in an environment even if SSL libraries are around.  The meson
scripts assume that, but the SSL tests thought that it was a hard
dependency, causing a meson installation to fail if `openssl` could not
be found.  Like similar tests that depend on external commands, and to
be consistent with ./configure for the SSL tests, this commit makes the
command existence optional in the tests.

Author: Tristan Partin
Discussion: https://postgr.es/m/CWSX6P5OUUM5.N7B74KQ06ZP6@neon.tech
Backpatch-through: 16
2023-11-08 17:29:02 +09:00
Michael Paquier 1b2c6b756e Enlarge assertion in bloom_init() for false_positive_rate
false_positive_rate is a parameter that can be set with the bloom
opclass in BRIN, and setting it to a value of exactly 0.25 would trigger
an assertion in the first INSERT done on the index with value set.

The assertion changed here relied on BLOOM_{MIN|MAX}_FALSE_POSITIVE_RATE
that are somewhat arbitrary values, and specifying an out-of-range value
would also trigger a failure when defining such an index.  So, as-is,
the assertion was just doubling on the min-max check of the reloption.
This is now enlarged to check that it is a correct percentage value,
instead, based on a suggestion by Tom Lane.

Author: Alexander Lakhin
Reviewed-by: Tom Lane, Shihao Zhong
Discussion: https://postgr.es/m/17969-a6c54de48026d694@postgresql.org
Backpatch-through: 14
2023-11-08 14:06:26 +09:00
Alvaro Herrera 615f5f6faa
Stop including parsenodes.h in plannodes.h
I added it by mistake in commit 7103ebb7aa.  To clean up, struct
MergeAction needs to be moved to primnodes.h from parsenodes.h.  (This
forces us to also move OverridingKind to primnodes.h).

Having to add parsenodes.h to bootstrap.h as fallout is a bit
surprising, since nothing nominally needs it there.  However, per
comments in bootscanner.l, it is needed so that YYSTYPE can be declared.
I think this only started with commit dac048f71e, but I didn't
actually verify that.

In passing, stop including parsenodes.h in tcopprot.h.  Nothing needs it
there.

Per discussion on a patch by Ashutosh Bapat.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/202311071106.6y7b2ascqjlz@alvherre.pgsql
2023-11-07 19:26:39 +01:00
Peter Eisentraut 3c551ebede citext: Allow tests to pass in OpenSSL FIPS mode
citext doesn't define an md5() function, so the value of using it in
its tests is dubious.  At best this shows in an indirect way that the
cast from citext to text works.  Avoid the issue and remove the test.

Discussion: https://www.postgresql.org/message-id/flat/dbbd927f-ef1f-c9a1-4ec6-c759778ac852%40enterprisedb.com
2023-11-07 07:58:14 +01:00
Amit Kapila 8af917be6b Fix the test 003_logical_slots.
pg_upgrade test 003_logical_slots was leaving files like
delete_old_cluster.sh in the source directory for VPATH and meson builds.
The fix is to change the directory to tmp_check before running the test as
is done in the similar test in 002_pg_upgrade.

Reported-by: Peter Eisentraut, Andrew Dunstan
Author: Hayato Kuroda based on a suggestion by Andrew Dunstan
Reviewed-by: Peter Smith, Hou Zhijie
Discussion: http://postgr.es/m/b4fb612d-ef0b-4db7-81b9-cf0701275491@eisentraut.org
Discussion: http://postgr.es/m/TYAPR01MB5866D7B89DC5688256D980C2F5A9A@TYAPR01MB5866.jpnprd01.prod.outlook.com
2023-11-07 11:32:33 +05:30
Michael Paquier c2bdd2c5b1 Reorder two functions in inval.c
This file separates public and static functions with a separator
comment, but two routines were not defined in a location reflecting
that, so reorder them.

Author: Aleksander Alekseev
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/CAJ7c6TMX2dd0g91UKvcC+CVygKQYJkKJq1+ZzT4rOK42+b53=w@mail.gmail.com
2023-11-07 11:55:13 +09:00
David Rowley ac7d6f5f83 Make use of initReadOnlyStringInfo() in more places
f0efa5aec introduced the concept of "read-only" StringInfos which makes
use of an existing, possibly not NUL terminated, buffer.

Here we adjust two places that make use of StringInfos to receive data
to avoid using appendBinaryStringInfo() in cases where a NUL termination
character is not required.  This saves a possible palloc() and saves
having to needlessly memcpy() from one buffer to another.

Here we adjust two places which were using appendBinaryStringInfo().
Neither of these cases seem particularly performance-critical.  In the
case of XLogWalRcvProcessMsg(), the appendBinaryStringInfo() was only
appending 24 bytes.  The change made here does mean that we can get rid
of the incoming_message global variable and make that local instead.

The apply_spooled_messages() case applies in logical decoding when
applying (possibly large) changes which have been serialized to a file.

Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/CAApHDvoxYUDHwqPf-ShvchsERf1RzmkGoLwg63JNvHCkDCuyKQ@mail.gmail.com
2023-11-07 11:16:43 +13:00
Tom Lane 18b585155a Detect integer overflow while computing new array dimensions.
array_set_element() and related functions allow an array to be
enlarged by assigning to subscripts outside the current array bounds.
While these places were careful to check that the new bounds are
allowable, they neglected to consider the risk of integer overflow
in computing the new bounds.  In edge cases, we could compute new
bounds that are invalid but get past the subsequent checks,
allowing bad things to happen.  Memory stomps that are potentially
exploitable for arbitrary code execution are possible, and so is
disclosure of server memory.

To fix, perform the hazardous computations using overflow-detecting
arithmetic routines, which fortunately exist in all still-supported
branches.

The test cases added for this generate (after patching) errors that
mention the value of MaxArraySize, which is platform-dependent.
Rather than introduce multiple expected-files, use psql's VERBOSITY
parameter to suppress the printing of the message text.  v11 psql
lacks that parameter, so omit the tests in that branch.

Our thanks to Pedro Gallegos for reporting this problem.

Security: CVE-2023-5869
2023-11-06 10:56:43 -05:00
Tom Lane 3b0776fde5 Compute aggregate argument types correctly in transformAggregateCall().
transformAggregateCall() captures the datatypes of the aggregate's
arguments immediately to construct the Aggref.aggargtypes list.
This seems reasonable because the arguments have already been
transformed --- but there is an edge case where they haven't been.
Specifically, if we have an unknown-type literal in an ANY argument
position, nothing will have been done with it earlier.  But if we
also have DISTINCT, then addTargetToGroupList() converts the literal
to "text" type, resulting in the aggargtypes list not matching the
actual runtime type of the argument.  The end result is that the
aggregate tries to interpret a "text" value as being of type
"unknown", that is a zero-terminated C string.  If the text value
contains no zero bytes, this could result in disclosure of server
memory following the text literal value.

To fix, move the collection of the aggargtypes list to the end
of transformAggregateCall(), after DISTINCT has been handled.
This requires slightly more code, but not a great deal.

Our thanks to Jingzhou Fu for reporting this problem.

Security: CVE-2023-5868
2023-11-06 10:38:00 -05:00
Peter Eisentraut 721856ff24 Remove distprep
A PostgreSQL release tarball contains a number of prebuilt files, in
particular files produced by bison, flex, perl, and well as html and
man documentation.  We have done this consistent with established
practice at the time to not require these tools for building from a
tarball.  Some of these tools were hard to get, or get the right
version of, from time to time, and shipping the prebuilt output was a
convenience to users.

Now this has at least two problems:

One, we have to make the build system(s) work in two modes: Building
from a git checkout and building from a tarball.  This is pretty
complicated, but it works so far for autoconf/make.  It does not
currently work for meson; you can currently only build with meson from
a git checkout.  Making meson builds work from a tarball seems very
difficult or impossible.  One particular problem is that since meson
requires a separate build directory, we cannot make the build update
files like gram.h in the source tree.  So if you were to build from a
tarball and update gram.y, you will have a gram.h in the source tree
and one in the build tree, but the way things work is that the
compiler will always use the one in the source tree.  So you cannot,
for example, make any gram.y changes when building from a tarball.
This seems impossible to fix in a non-horrible way.

Second, there is increased interest nowadays in precisely tracking the
origin of software.  We can reasonably track contributions into the
git tree, and users can reasonably track the path from a tarball to
packages and downloads and installs.  But what happens between the git
tree and the tarball is obscure and in some cases non-reproducible.

The solution for both of these issues is to get rid of the step that
adds prebuilt files to the tarball.  The tarball now only contains
what is in the git tree (*).  Getting the additional build
dependencies is no longer a problem nowadays, and the complications to
keep these dual build modes working are significant.  And of course we
want to get the meson build system working universally.

This commit removes the make distprep target altogether.  The make
dist target continues to do its job, it just doesn't call distprep
anymore.

(*) - The tarball also contains the INSTALL file that is built at make
dist time, but not by distprep.  This is unchanged for now.

The make maintainer-clean target, whose job it is to remove the
prebuilt files in addition to what make distclean does, is now just an
alias to make distprep.  (In practice, it is probably obsolete given
that git clean is available.)

The following programs are now hard build requirements in configure
(they were already required by meson.build):

- bison
- flex
- perl

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://www.postgresql.org/message-id/flat/e07408d9-e5f2-d9fd-5672-f53354e9305e@eisentraut.org
2023-11-06 15:18:04 +01:00
Noah Misch b72de09a1b Set GUC "is_superuser" in all processes that set AuthenticatedUserId.
It was always false in single-user mode, in autovacuum workers, and in
background workers.  This had no specifically-identified security
consequences, but non-core code or future work might make it
security-relevant.  Back-patch to v11 (all supported versions).

Jelte Fennema-Nio.  Reported by Jelte Fennema-Nio.
2023-11-06 06:14:13 -08:00
Noah Misch 3a9b18b309 Ban role pg_signal_backend from more superuser backend types.
Documentation says it cannot signal "a backend owned by a superuser".
On the contrary, it could signal background workers, including the
logical replication launcher.  It could signal autovacuum workers and
the autovacuum launcher.  Block all that.  Signaling autovacuum workers
and those two launchers doesn't stall progress beyond what one could
achieve other ways.  If a cluster uses a non-core extension with a
background worker that does not auto-restart, this could create a denial
of service with respect to that background worker.  A background worker
with bugs in its code for responding to terminations or cancellations
could experience those bugs at a time the pg_signal_backend member
chooses.  Back-patch to v11 (all supported versions).

Reviewed by Jelte Fennema-Nio.  Reported by Hemanth Sandrana and
Mahendrakar Srinivasarao.

Security: CVE-2023-5870
2023-11-06 06:14:13 -08:00
Daniel Gustafsson 526fe0d799 Add XMLText function (SQL/XML X038)
This function implements the standard XMLTest function, which
converts text into xml text nodes. It uses the libxml2 function
xmlEncodeSpecialChars to escape predefined entities (&"<>), so
that those do not cause any conflict when concatenating the text
node output with existing xml documents.

This also adds a note in  features.sgml about not supporting
XML(SEQUENCE). The SQL specification defines a RETURNING clause
to a set of XML functions, where RETURNING CONTENT or RETURNING
SEQUENCE can be defined. Since PostgreSQL doesn't support
XML(SEQUENCE) all of these functions operate with an
implicit RETURNING CONTENT.

Author: Jim Jones <jim.jones@uni-muenster.de>
Reviewed-by: Vik Fearing <vik@postgresfriends.org>
Discussion: https://postgr.es/m/86617a66-ec95-581f-8d54-08059cca8885@uni-muenster.de
2023-11-06 09:38:29 +01:00
Peter Eisentraut 7b5275eec3 pg_resetwal: Add more tests and test coverage
pg_resetwal had poor test coverage.  There are some TAP tests, but
they all run with -n, so they don't actually test the full
functionality.  (There is a non-dry-run call of pg_resetwal in the
recovery test suite, but that is incidental.)

This adds a bunch of more tests to test all the different options and
scenarios.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-11-06 09:16:00 +01:00
Peter Eisentraut 6ceec8a1fe doc: pg_resetwal: Add comments how the multipliers are derived
Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/0f3ab4a1-ae80-56e8-3426-6b4a02507687@eisentraut.org
2023-11-06 09:16:00 +01:00
Alexander Korotkov 93c85db3b5 Fix allocation of UniqueRelInfo
Reported-by: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4_STsG1PKQBuvQC8W4sPo3KvML3=jOTjKLUYQuK3g8cpQ@mail.gmail.com
2023-11-06 10:04:01 +02:00
Peter Eisentraut 2c7c6c417f More consistent behavior of GetDataDirectoryCreatePerm on Windows
On Windows, GetDataDirectoryCreatePerm() just did nothing.  The way
the code in some callers is structured, this is the first function
that tries to access the data directory.  So it also ends up the place
that is responsible for reporting that a data directory does not exist
or similar.  Therefore, on Windows, these scenarios end up on
potentially completely different code paths.

To unify this, to make testing more consistent across platforms, have
GetDataDirectoryCreatePerm() run the stat() call on Windows as well,
even though it won't do anything with the result.  That way, file
system errors are reporting to callers in the same way as on
non-Windows.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/15a59bca-0383-183c-9383-0446da9b87e1%40eisentraut.org
2023-11-05 21:59:04 +01:00
David Rowley 151ffcf6d8 Try again to fix the MSVC build
My last attempt in 39c959ef2 mistakenly conditionally added the missing
file based on some unrelated condition.

Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGLovvAXim9Fytn=jxks9s=JhP5=8Oyy0cbxGG-ggALJtg@mail.gmail.com
2023-11-04 15:41:16 +13:00
Alexander Korotkov ec63622c03 Fix usage of the parse tree for estimate_num_groups() in set operations
recurse_set_operations() uses the parse tree for the group number estimation,
because of the "varno 0" hack.  At the same time 2489d76c49 made root->parse
and corresponding parent_root->simple_rte_array[]->subquery distinct copies
of the parse tree, while d3d55ce571 introduced self-join removal replacing
relid of removed relation only in one of the copies.

The present commit fixes this bug by making recurse_set_operations() call
estimate_num_groups() with the copy of the parse tree processed by self-join
removal.

In future, we may think about maintaining just one copy of the parse tree
and/or keeping removed relids as aliases.

Reported-by: Zuming Jiang
Bug: #18170
Discussion: https://postgr.es/m/flat/18170-f1d17bf9a0d58b24%40postgresql.org
Author: Richard Guo, Alexander Korotkov
Reviewed-by: Andrei Lepikhov
2023-11-04 03:30:18 +02:00
Andres Freund a237a07d53 meson: docs: Install all manpages, not just ones in man1
In f13eb16485 I made a mistake leading to only man1 being installed. I will
report a bug suggesting that meson warn about mistakes of this sort.

Reported-by: Christoph Berg <myon@debian.org>
Discussion: https://postgr.es/m/ZUU5pRQO6ZUeBsi6@msg.df7cb.de
Backpatch: 16-, where the meson build was introduced
2023-11-03 14:48:52 -07:00
Bruce Momjian d594e0daf7 doc: move HBA reload instructions above the syntax details
Reported-by: John <johrss@amazon.com>

Discussion: https://postgr.es/m/165947088723.651.7641196693246068619@wrigleys.postgresql.org

Backpatch-through: master
2023-11-03 14:03:22 -04:00
Bruce Momjian 42d3125ada doc: \copy can get data values \. and end-of-input confused
Reported-by: Svante Richter

Discussion: https://postgr.es/m/fcd57e4-8f23-4c3e-a5db-2571d09208e2@beta.fastmail.com

Backpatch-through: 11
2023-11-03 13:57:59 -04:00
Bruce Momjian d391f6dcc3 doc: CREATE DATABASE doesn't copy db-level perms. from template
Reported-by: david@kapitaltrading.com

Discussion: https://postgr.es/m/166007719137.995877.13951579839074751714@wrigleys.postgresql.org

Backpatch-through: 11
2023-11-03 13:39:50 -04:00
Bruce Momjian 7751352942 doc: mention ORDER BY for some aggregates, add ORDER BY examples
Discussion: https://postgr.es/m/CAKFQuwb+4SWnfrfQKB-UM1P1x97Xk+ybSar4xM32XGLd=fq9bA@mail.gmail.com

Co-authored-by: David G. Johnston

Backpatch-through: master
2023-11-03 13:05:27 -04:00
Tom Lane 95a610b097 Doc: update CREATE RULE ref page's hoary discussion of views.
This text left one with the impression that an ON SELECT rule could
be attached to a plain table, which has not been true since commit
264c06820 (meaning the text was already misleading when written,
evidently by me in 96bd67f61).  However, it didn't get really bad
until b23cd185f removed the convert-a-table-to-a-view logic, which
had made it possible for scripts that thought they were attaching
ON SELECTs to tables to still work.

Rewrite into a form that makes it clear that an ON SELECT rule
is better regarded as an implementation detail of a view.
Pre-v16, point out that adding ON SELECT to a table actually
converts it to a view.

Per bug #18178 from Joshua Uyehara.  Back-patch to all supported
branches.

Discussion: https://postgr.es/m/18178-05534d7064044d2d@postgresql.org
2023-11-03 11:48:23 -04:00
Bruce Momjian a8510a7d96 doc: ALTER DEFAULT PRIVILEGES does not affect inherited roles
Reported-by: Jordi Gutiérrez Hermoso

Discussion: https://postgr.es/m/72652d72e1816bfc3c05d40f9e0e0373d07823c8.camel@octave.org

Co-authored-by: Laurenz Albe

Backpatch-through: 11
2023-11-03 09:51:53 -04:00
David Rowley 39c959ef25 Add missing unicode_category.c to MSVC build scripts
Fixes MSVC build failure introduced by a02b37fc0
2023-11-03 20:12:36 +13:00
David Rowley b690e5facb Stabilize postgres_fdw tests on 32-bit machines
cac169d68 adjusted DEFAULT_FDW_TUPLE_COST and that seems to have caused
a test to become unstable on 32-bit machines.

4b14e1871 tried to fix this as originally the plan was flipping between
a Nested Loop and Hash Join.  That commit forced the Nested Loop, but
there's still flexibility to push or not push the sort to the remote
server and 32-bit seems to prefer to push and on 64-bit, the costs
prefer not to.

Here let's just turn off enable_sort to significantly encourage the sort
to take place on the remote server.

Reported-by: Michael Paquier, Richard Guo
Discussion: https://postgr.es/m/ZUM2IhA8X2lrG50K@paquier.xyz
2023-11-03 12:35:37 +13:00
Tom Lane 0bc726d95a Make GetConfigOption/GetConfigOptionResetString return "" for NULL.
As per the preceding commit, GUC APIs generally expose NULL-valued
string variables as empty strings.  Extend that policy to
GetConfigOption() and GetConfigOptionResetString(), eliminating
a crash hazard for unwary callers, as well as a fundamental
ambiguity in GetConfigOption()'s API.

No back-patch, since this is an API change and conceivably somebody
somewhere is depending on this corner case.

Xing Guo, Aleksander Alekseev, Tom Lane

Discussion: https://postgr.es/m/CACpMh+AyDx5YUpPaAgzVwC1d8zfOL4JoD-uyFDnNSa1z0EsDQQ@mail.gmail.com
2023-11-02 11:53:36 -04:00
Tom Lane 7704a1a72e Be more wary about NULL values for GUC string variables.
get_explain_guc_options() crashed if a string GUC marked GUC_EXPLAIN
has a NULL boot_val.  Nosing around found a couple of other places
that seemed insufficiently cautious about NULL string values, although
those are likely unreachable in practice.  Add some commentary
defining the expectations for NULL values of string variables,
in hopes of forestalling future additions of more such bugs.

Xing Guo, Aleksander Alekseev, Tom Lane

Discussion: https://postgr.es/m/CACpMh+AyDx5YUpPaAgzVwC1d8zfOL4JoD-uyFDnNSa1z0EsDQQ@mail.gmail.com
2023-11-02 11:47:33 -04:00
David Rowley 4b14e18714 Attempt to stabilize postgres_fdw tests
cac169d68 adjusted DEFAULT_FDW_TUPLE_COST and that seems to have caused
a test to become unstable on 32-bit machines.  Try to make it stable
again.

Reported-by: Michael Paquier
Discussion: https://postgr.es/m/ZUM2IhA8X2lrG50K@paquier.xyz
2023-11-02 23:16:34 +13:00
Jeff Davis a02b37fc08 Additional unicode primitive functions.
Introduce unicode_version(), icu_unicode_version(), and
unicode_assigned().

The latter requires introducing a new lookup table for the Unicode
General Category, which is generated along with the other Unicode
lookup tables.

Discussion: https://postgr.es/m/CA+TgmoYzYR-yhU6k1XFCADeyj=Oyz2PkVsa3iKv+keM8wp-F_A@mail.gmail.com
Reviewed-by: Peter Eisentraut
2023-11-01 22:47:06 -07:00
Michael Paquier 7021d3b176 Prevent startup of logical replication launcher during pg_upgrade
The logical replication launcher may start apply workers during an
upgrade.  This could be the cause of corruptions on a new cluster if
these are able to apply changes before the physical files are copied
over to the new cluster.

The chance of being able to do so is small as pg_upgrade uses its own
port and unix domain directory (the latter is customizable with
--socketdir), but just preventing the launcher to start is safer at the
end, because we are then sure that no changes will be applied.  Like
29d0a77fa6 for max_slot_wal_keep_size, this is only set when a cluster
uses v17 or newer.

Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm2g9ZKf=y8X6z6MsLCuh8WwU-=Q6pLj35NFi2M5BZNS_A@mail.gmail.com
2023-11-02 14:34:51 +09:00
Michael Paquier 40d5e5981c Fix 003_check_guc.pl when loading modules with custom GUCs
The test missed that custom GUCs need to be ignored from the list of
parameters that can exist in postgresql.conf.sample.  This caused the
test to fail on a server where such a module is loaded, when using
EXTRA_INSTALL and TEMP_CONFIG, for instance.

Author: Anton A. Melnikov
Discussion: https://postgr.es/m/fc5509ce-5144-4dac-8d13-21793da44fc5@postgrespro.ru
Backpatch-through: 15
2023-11-02 12:38:05 +09:00
David Rowley cac169d686 Increase DEFAULT_FDW_TUPLE_COST from 0.01 to 0.2
0.01 was unrealistically low as it's the same as the default
cpu_tuple_cost and 10x cheaper than the default parallel_tuple_cost.
It's hard to imagine a situation where fetching a tuple from a foreign
server would be cheaper than fetching one from a parallel worker.

After some experimentation on a loopback server, somewhere between 0.15
and 0.3 seems more realistic.  Here we split the difference and set it
to 0.2.

This will cause operations that reduce the number of tuples (e.g.
aggregation) to be more likely to take place on the foreign server.

Adjusting this causes some plan changes in the postgres_fdw regression
tests.  This is because penalizing each Path with the additional tuple
costs causes some dilution of the costs of the other operations being
charged for and results in various paths appearing to be closer to the
same costs such that add_path's STD_FUZZ_FACTOR is more likely to see two
paths as costing (fuzzily) the same.  This isn't ideal, but it shouldn't
be reason enough to use artificially low costs.

Discussion: https://postgr.es/m/CAApHDvopVjjfh5c1Ed2HRvDdfom2dEpMwwiu5-f1AnmYprJngA@mail.gmail.com
2023-11-02 14:30:15 +13:00
Michael Paquier 4210b55f59 doc: Replace reference to ERRCODE_RAISE_EXCEPTION by "raise_exception"
This part of the documentation refers to exceptions as handled by
PL/pgSQL, and using the internal error code is confusing.

Per thinko in 66bde49d96.

Reported-by: Euler Taveira, Bruce Momjian
Discussion: https://postgr.es/m/ZUEUnLevXyW7DlCs@momjian.us
Backpatch-through: 11
2023-11-02 07:33:02 +09:00
Bruce Momjian 783e816666 doc: add reference to wire protocol details
Discussion: https://postgr.es/m/143A51B2-80B1-4ECD-AF67-F7061377FF63@hotmail.com

Author: Li Japin

Backpatch-through: master
2023-11-01 13:57:29 -04:00
Daniel Gustafsson 0f852cccd9 Fix function name in comment
The name of the function resulting from the macro expansion was
incorrectly stated.

Backpatch to 16 where it was introduced.

Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20231101.172308.1740861597185391383.horikyota.ntt@gmail.com
Backpatch-through: v16
2023-11-01 11:46:30 +01:00
Michael Paquier fe705ef6fc doc: Expand section related to LWLocks and shared memory
The documentation includes a section describing how to define custom
LWLocks in extensions using the shmem hooks.  However, it has never
mentioned the second, more flexible method based on the following
routines:
- LWLockNewTrancheId() to allocate a tranche ID.
- LWLockRegisterTranche() to associate a name to a tranche ID.
- LWLockInitialize() to initialize a LWLock with a tranche ID.

autoprewarm.c is the only example of extension in the tree that
allocates a LWLock this way.

This commit adds some documentation about all that.  While on it, a
comment is added about the need of AddinShmemInitLock.  This is required
especially for EXEC_BACKEND builds (aka Windows, normally), as per a
remark from Alexander, because backends can execute shmem initialization
paths concurrently.

Author: Aleksander Alekseev, Michael Paquier
Discussion: https://postgr.es/m/CAJ7c6TPKhFgL+54cdTD9yGpG4+sNcyJ+N1GvQqAxgWENAOa3VA@mail.gmail.com
2023-11-01 14:54:13 +09:00
Bruce Momjian 6ec62b7799 doc: add missing word to sentence about Paris
Reported-by: Tang <tanghy.fnst@fujitsu.com>

Discussion: https://postgr.es/m/OS0PR01MB6113393560A1DF115ADFB153FB2D9@OS0PR01MB6113.jpnprd01.prod.outlook.com

Backpatch-through: master
2023-10-31 13:18:42 -04:00
Bruce Momjian 3fdc1d2693 C comment: mention why no setting lasterrno in dir_existsfile()
Reported-by: Wei Sun

Discussion: https://postgr.es/m/tencent_1276C08F98579CC19D8A4488C848A8411806@qq.com

Backpatch-through: master
2023-10-31 11:59:14 -04:00
Bruce Momjian b706172d22 C comment: improve statistics computation comment example
Discussion: https://postgr.es/m/CAKFQuwbD672Sc0EXv0ifx3pzfQ5UAEpiAeaBGKz_Ox-4d2NGCA@mail.gmail.com

Author: David G. Johnston

Backpatch-through: master
2023-10-31 11:42:02 -04:00
Bruce Momjian 741ed2065c C comment: adjust statistics mention
No need to talk about the statistics collector.

Discussion: https://postgr.es/m/8a82417cdb6e8038fe276d4960e3207a@oss.nttdata.com

Author: Álvaro Herrera

Backpatch-through: master
2023-10-31 11:02:04 -04:00
Bruce Momjian 87cf7b63c4 doc: improve ALTER SYSTEM description of value list quoting
Reported-by: splarv@ya.ru

Discussion: https://postgr.es/m/167105927893.1897.13227723035830709578@wrigleys.postgresql.org

Backpatch-through: 11
2023-10-31 10:21:32 -04:00
Bruce Momjian b69db51736 doc: improve bpchar and character type length details
Reported-by: Jeff Davis

Discussion: https://postgr.es/m/32a9b8357e8e29b04f395f92c53b64e015a4caf1.camel@j-davis.com

Author: Jeff Davis, adjustments by me

Backpatch-through: 16
2023-10-31 10:13:11 -04:00
Bruce Momjian 4a6286fd93 doc: add function argument and query parameter limits
Also reorder entries and add commas.

Reported-by: David G. Johnston

Discussion: https://postgr.es/m/CAKFQuwYeNPxeocV3_0+Zx=_Xwvg+sNyEMdzyG5s2E2e0hZLQhg@mail.gmail.com

Author: David G. Johnston (partial)

Backpatch-through: 12
2023-10-31 09:23:09 -04:00
Bruce Momjian 989adace3f doc: 1-byte varlena headers can be used for user PLAIN storage
This also updates some C comments.

Reported-by: suchithjn22@gmail.com

Discussion: https://postgr.es/m/167336599095.2667301.15497893107226841625@wrigleys.postgresql.org

Author: Laurenz Albe (doc patch)

Backpatch-through: 11
2023-10-31 09:10:35 -04:00