Commit Graph

56962 Commits

Author SHA1 Message Date
Tom Lane 1a6dcfecab Doc: improve a couple of comments in postgresql.conf.sample.
Clarify comments associated with max_parallel_workers and
related settings.

Per bug #18343 from Christopher Kline.

Discussion: https://postgr.es/m/18343-3a5e903d1d3692ab@postgresql.org
2024-02-15 16:45:03 -05:00
Daniel Gustafsson aafa4513fd doc: Remove links to further reading from pgcrypto docs
The pgcrypto docs contained a set of links for useful reading and
technical references. These sets of links were however not actively
curated and had stale content and dead links. Rather than investing
time into maintining these, this removes them altogether since there
are lots of resources online which are actively maintained.

Backpatch to all supported versions since these links have been in
the docs for a long time.

Reported-by: Hanefi Onaldi <hanefi.onaldi@microsoft.com>
Reviewed-by: Magnus Hagander <magnus@hagander.net>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/170774255387.3279713.2822272755998870925@wrigleys.postgresql.org
Backpatch-through: v12
2024-02-14 11:05:10 +01:00
Heikki Linnakangas f2f09b825c Fix 'mmap' DSM implementation with allocations larger than 4 GB
Fixes bug #18341. Backpatch to all supported versions.

Discussion: https://www.postgresql.org/message-id/18341-ce16599e7fd6228c@postgresql.org
2024-02-13 21:24:45 +02:00
Tom Lane f15d01cb5d Use a safer outfuncs/readfuncs representation for BitStrings.
For a long time, our outfuncs.c code has supposed that the string
contents of a BitString node could just be printed literally with
no concern for quoting/escaping.  Now, that's okay if the string
literal contains only valid binary or hex digits ... but our lexer
doesn't check that, preferring to let bitin() be the sole authority
on what's valid.  So we could have raw parse trees that contain
incorrect BitString literals, and that can result in failures when
WRITE_READ_PARSE_PLAN_TREES debugging is enabled.

Fix by using outToken() to print the string field, and debackslash()
to read it.  This results in a change in the emitted representation
only in cases that would have failed before, and don't represent valid
SQL in the first place.  Between that and the fact that we don't store
raw parse trees in the catalogs, I judge this safe to apply without a
catversion bump.

Per bug #18340 from Alexander Lakhin.  Back-patch to v16; before that,
we lacked readfuncs support for BitString nodes, so that the problem
was only cosmetic.

Discussion: https://postgr.es/m/18340-4aa1ae6ed4121912@postgresql.org
2024-02-13 12:18:25 -05:00
Daniel Gustafsson 103235888d Skip .DS_Store files in server side utils
The macOS Finder application creates .DS_Store files in directories
when opened,  which creates problems for serverside utilities which
expect all files to be PostgreSQL specific files.  Skip these files
when encountered in pg_checksums, pg_rewind and pg_basebackup.

This was extracted from a larger patchset for skipping hidden files
and system files, where the concencus was to just skip these. Since
this is equally likely to happen in every version, backpatch to all
supported versions.

Reported-by: Mark Guertin <markguertin@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Tobias Bussmann <t.bussmann@gmx.net>
Discussion: https://postgr.es/m/E258CE50-AB0E-455D-8AAD-BB4FE8F882FB@gmail.com
Backpatch-through: v12
2024-02-13 13:47:12 +01:00
Thomas Munro 0460e4ecc0 Fix gai_strerror() thread-safety on Windows.
Commit 5579388d removed code that supplied a fallback implementation of
getaddrinfo(), which was dead code on modern systems.  One tiny piece of
the removed code was still doing something useful on Windows, though:
that OS's own gai_strerror()/gai_strerrorA() function returns a pointer
to a static buffer that it overwrites each time, so it's not
thread-safe.  In rare circumstances, a multi-threaded client program
could get an incorrect or corrupted error message.

Restore the replacement gai_strerror() function, though now that it's
only for Windows we can put it into a win32-specific file and cut it
down to the errors that Windows documents.  The error messages here are
taken from FreeBSD, because Windows' own messages seemed too verbose.

Back-patch to 16.

Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/CA%2BhUKGKz%2BF9d2PTiXwfYV7qJw%2BWg2jzACgSDgPizUw7UG%3Di58A%40mail.gmail.com
2024-02-12 11:14:42 +13:00
Tom Lane 4eb261165d Remove race condition in pg_get_expr().
Since its introduction, pg_get_expr() has intended to silently
return NULL if called with an invalid relation OID, as can happen
when scanning the catalogs concurrently with relation drops.
However, there is a race condition: we check validity of the OID
at the start, but it could get dropped just afterward, leading to
failures.  This is the cause of some intermittent instability we're
seeing in a proposed new test case, and presumably it's a hazard in
the field as well.

We can fix this by AccessShareLock-ing the target relation for the
duration of pg_get_expr().  Since we don't require any permissions
on the target relation, this is semantically a bit undesirable.  But
it turns out that the set_relation_column_names() subroutine already
takes a transient AccessShareLock on that relation, and has done since
commit 2ffa740be in 2012.  Given the lack of complaints about that, it
seems like there should be no harm in holding the lock a bit longer.

Back-patch to all supported branches.

Discussion: https://postgr.es/m/31ddcc01-a71b-4e8c-9948-01d1c47293ca@eisentraut.org
2024-02-09 12:29:41 -05:00
Tom Lane 52afe56320 Avoid concurrent calls to bindtextdomain().
We previously supposed that it was okay for different threads to
call bindtextdomain() concurrently (cf. commit 1f655fdc3).
It now emerges that there's at least one gettext implementation
in which that triggers an abort() crash, so let's stop doing that.
Add mutexes guarding libpq's and ecpglib's calls, which are the
only ones that need worry about multithreaded callers.

Note: in libpq, we could perhaps have piggybacked on
default_threadlock() to avoid defining a new mutex variable.
I judge that not terribly safe though, since libpq_gettext could
be called from code that is holding the default mutex.  If that
were the first such call in the process, it'd fail.  An extra
mutex is cheap insurance against unforeseen interactions.

Per bug #18312 from Christian Maurer.  Back-patch to all
supported versions.

Discussion: https://postgr.es/m/18312-bbbabc8113592b78@postgresql.org
Discussion: https://postgr.es/m/264860.1707163416@sss.pgh.pa.us
2024-02-09 11:21:08 -05:00
Tom Lane 9440d23a01 Clean up Windows-specific mutex code in libpq and ecpglib.
Fix pthread-win32.h and pthread-win32.c to provide a more complete
emulation of POSIX pthread mutexes: define PTHREAD_MUTEX_INITIALIZER
and make sure that pthread_mutex_lock() can operate on a mutex
object that's been initialized that way.  Then we don't need the
duplicative platform-specific logic in default_threadlock() and
pgtls_init(), which we'd otherwise need yet a third copy of for
an upcoming bug fix.

Also, since default_threadlock() supposes that pthread_mutex_lock()
cannot fail, try to ensure that that's actually true, by getting
rid of the malloc call that was formerly involved in initializing
an emulated mutex.  We can define an extra state for the spinlock
field instead.

Also, replace the similar code in ecpglib/misc.c with this version.
While ecpglib's version at least had a POSIX-compliant API, it
also had the potential of failing during mutex init (but here,
because of CreateMutex failure rather than malloc failure).  Since
all of misc.c's callers ignore failures, it seems like a wise idea
to avoid failures here too.

A further improvement in this area could be to unify libpq's and
ecpglib's implementations into a src/port/pthread-win32.c file.
But that doesn't seem like a bug fix, so I'll desist for now.

In preparation for the aforementioned bug fix, back-patch to all
supported branches.

Discussion: https://postgr.es/m/264860.1707163416@sss.pgh.pa.us
2024-02-09 11:11:39 -05:00
Alexander Korotkov e3e05addee Fix wrong logic in TransactionIdInRecentPast()
The TransactionIdInRecentPast() should return false for all the transactions
older than TransamVariables->oldestClogXid.  However, the function contains
a bug in comparison FullTransactionId to TransactionID allowing full
transactions between nextXid - 2^32 and oldestClogXid - 2^31.

This commit fixes TransactionIdInRecentPast() by turning the oldestClogXid into
FullTransactionId first, then performing the comparison.

Backpatch to all supported versions.

Reported-by: Egor Chindyaskin
Bug: 18212
Discussion: https://postgr.es/m/18212-547307f8adf57262%40postgresql.org
Author: Karina Litskevich
Reviewed-by: Kyotaro Horiguchi
Backpatch-through: 12
2024-02-09 12:37:21 +02:00
Peter Eisentraut 86d2b434c9 Fix propagation of persistence to sequences in ALTER TABLE / ADD COLUMN
Fix for 344d62fb9a9: That commit introduced unlogged sequences and
made it so that identity/serial sequences automatically get the
persistence level of their owning table.  But this works only for
CREATE TABLE and not for ALTER TABLE / ADD COLUMN.  The latter would
always create the sequence as logged (default), independent of the
persistence setting of the table.  This is fixed here.

Note: It is allowed to change the persistence of identity sequences
directly using ALTER SEQUENCE.  So mistakes in existing databases can
be fixed manually.

Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/c4b6e2ed-bcdf-4ea7-965f-e49761094827%40eisentraut.org
2024-02-09 08:09:59 +01:00
Daniel Gustafsson e13205a5e4 doc: Remove superfluous bracket in synopsis
Commit 9c08aea6a3 accidentally added one too many end brackets
in the synopsis for CREATE DATABASE .. strategy = strat. Fix by
removing. Backpatch to v15 where it was introduced.

Reported-by: tim.needham2@gmail.com
Discussion: https://postgr.es/m/170734160862.3279712.810853722572951776@wrigleys.postgresql.org
Backpatch-through: v15
2024-02-08 12:19:34 +01:00
Peter Eisentraut 5c699c857c Update comment
The documentation output format htmlhelp is no longer supported, but a
comment still mentioned it.
2024-02-08 10:19:55 +01:00
Alvaro Herrera 17f3196599
Update PQparameterStatus and ParameterStatus docs
Cover scram_iterations, which was added in commit b577743000.  While
at it, turn the list into a <simplelist> with 2 columns, which is much
nicer to read.

In master, remove mentions of antediluvian versions before which some
parameters were not reported.

Noticed while investigating a question by Maiquel Grassi.

Backpatch to 16.

Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Jelte Fennema-Nio <postgres@jeltef.nl>
Discussion: https://postgr.es/m/202401301236.mc5ebrohhtsd@alvherre.pgsql
2024-02-07 19:25:07 +01:00
Tom Lane b78fa8547d Stamp 16.2. 2024-02-05 16:41:37 -05:00
Tom Lane da21a3a7d5 Last-minute updates for release notes.
Security: CVE-2024-0985 (not CVE-2023-5869 as claimed in prior commit msg)
2024-02-05 11:51:11 -05:00
Peter Eisentraut 246d16eb87 Translation updates
Source-Git-URL: https://git.postgresql.org/git/pgtranslation/messages.git
Source-Git-Hash: 7465ae7935588bbbafa9aac1c2b8c5863de50cbb
2024-02-05 14:45:29 +01:00
Heikki Linnakangas fb3836855c Fix assertion if index is dropped during REFRESH CONCURRENTLY
When assertions are disabled, the built SQL statement is invalid and
you get a "syntax error". So this isn't a serious problem, but let's
avoid the assertion failure.

Backpatch to all supported versions.

Reviewed-by: Noah Misch
2024-02-05 11:03:03 +02:00
Heikki Linnakangas d6a61cb3be Run REFRESH MATERIALIZED VIEW CONCURRENTLY in right security context
The internal commands in REFRESH MATERIALIZED VIEW CONCURRENTLY are
correctly executed in SECURITY_RESTRICTED_OPERATION mode, except for
creating the temporary "diff" table, because you cannot create
temporary tables in SRO mode. But creating the temporary "diff" table
is a pretty complex CTAS command that selects from another temporary
table created earlier in the command. If you can cajole that CTAS
command to execute code defined by the table owner, the table owner
can run code with the privileges of the user running the REFRESH
command.

The proof-of-concept reported to the security team relied on CREATE
RULE to convert the internally-built temp table to a view. That's not
possible since commit b23cd185fd, and I was not able to find a
different way to turn the SELECT on the temp table into code
execution, so as far as I know this is only exploitable in v15 and
below. That's a fiddly assumption though, so apply this patch to
master and all stable versions.

Thanks to Pedro Gallegos for the report.

Security: CVE-2023-5869
Reviewed-by: Noah Misch
2024-02-05 11:02:56 +02:00
Tom Lane 272aecfc2f Release notes for 16.2, 15.6, 14.11, 13.14, 12.18. 2024-02-04 14:17:14 -05:00
Heikki Linnakangas cd307c38f0 Fix typo in comments
Backpatch-through: v16
2024-02-03 00:19:21 +02:00
Tom Lane a15378100f Translate ENOMEM to ERRCODE_OUT_OF_MEMORY in errcode_for_file_access().
Previously you got ERRCODE_INTERNAL_ERROR, which seems inappropriate,
especially given that we're trying to avoid emitting that in reachable
cases.

Alexander Kuzmenkov

Discussion: https://postgr.es/m/CALzhyqzgQph0BY8-hFRRGdHhF8CoqmmDHW9S=hMZ-HMzLxRqDQ@mail.gmail.com
2024-02-02 15:34:29 -05:00
Tom Lane 87dcc5e45f First-draft release notes for 16.2.
As usual, the release notes for other branches will be made by cutting
these down, but put them up for community review first.
2024-02-02 12:52:29 -05:00
Noah Misch 48a6bf5c4e Sync PG_VERSION file in CREATE DATABASE.
An OS crash could leave PG_VERSION empty or missing.  The same symptom
appeared in a backup by block device snapshot, taken after the next
checkpoint and before the OS flushes the PG_VERSION blocks.  Device
snapshots are not a documented backup method, however.  Back-patch to
v15, where commit 9c08aea6a3 introduced
STRATEGY=WAL_LOG and made it the default.

Discussion: https://postgr.es/m/20240130195003.0a.nmisch@google.com
2024-02-01 13:44:22 -08:00
Noah Misch 6d423e9ff9 Handle interleavings between CREATE DATABASE steps and base backup.
Restoring a base backup taken in the middle of CreateDirAndVersionFile()
or write_relmap_file() would lose the function's effects.  The symptom
was absence of the database directory, PG_VERSION file, or
pg_filenode.map.  If missing the directory, recovery would fail.  Either
missing file would not fail recovery but would render the new database
unusable.  Fix CreateDirAndVersionFile() with the transam/README "action
first and then write a WAL entry" strategy.  That has a side benefit of
moving filesystem mutations out of a critical section, reducing the ways
to PANIC.  Fix the write_relmap_file() call with a lock acquisition, so
it interacts with checkpoints like non-CREATE DATABASE calls do.
Back-patch to v15, where commit 9c08aea6a3
introduced STRATEGY=WAL_LOG and made it the default.

Discussion: https://postgr.es/m/20240130195003.0a.nmisch@google.com
2024-02-01 13:44:22 -08:00
Tom Lane b4fb76fb52 Update time zone data files to tzdata release 2024a.
DST law changes in Ittoqqortoormiit, Greenland (America/Scoresbysund),
Kazakhstan (Asia/Almaty and Asia/Qostanay) and Palestine; as well as
updates for the Antarctic stations Casey and Vostok.

Historical corrections for Vietnam, Toronto, and Miquelon.
2024-02-01 15:57:53 -05:00
Andrew Dunstan 90bad72cbb Avoid package qualification of $windows_os
Further fallout from commit 6ee26c6a4b. To keep code in sync and avoid
issues on older releases with different package names, simply use the
unqualified name like many other places in our code.
2024-02-01 15:33:58 -05:00
Tom Lane 62f1202031 Apply band-aid fix for an oversight in reparameterize_path_by_child.
The path we wish to reparameterize is not a standalone object:
in particular, it implicitly references baserestrictinfo clauses
in the associated RelOptInfo, and if it's a SampleScan path then
there is also the TableSampleClause in the RTE to worry about.
Both of those could contain lateral references to the join partner
relation, which would need to be modified to refer to its child.
Since we aren't doing that, affected queries can give wrong answers,
or odd failures such as "variable not found in subplan target list",
or executor crashes.  But we can't just summarily modify those
expressions, because they are shared with other paths for the rel.
We'd break things if we modify them and then end up using some
non-partitioned-join path.

In HEAD, we plan to fix this by postponing reparameterization
until create_plan(), when we know that those other paths are
no longer of interest, and then adjusting those expressions along
with the ones in the path itself.  That seems like too big a change
for stable branches however.  In the back branches, let's just detect
whether any troublesome lateral references actually exist in those
expressions, and fail reparameterization if so.  This will result in
not performing a partitioned join in such cases.  Given the lack of
field complaints, nobody's likely to miss the optimization.

Report and patch by Richard Guo.  Apply to 12-16 only, since
the intended fix for HEAD looks quite different.  We're not quite
ready to push the HEAD fix, but with back-branch releases coming
up soon, it seems wise to get this stopgap fix in place there.

Discussion: https://postgr.es/m/CAMbWs496+N=UAjOc=rcD3P7B6oJe4rZw08e_TZRUsWbPxZW3Tw@mail.gmail.com
2024-02-01 12:34:21 -05:00
Bruce Momjian 88be3f076c doc: improve role option documentation
Role option management was changed in Postgres 16.  This patch improves
the docs around these changes, including CREATE ROLE's INHERIT option,
inheritance handling, and grant's ability to change role options.

Discussion: https://postgr.es/m/Zab9GiV63EENDcWG@momjian.us

Co-authored-by: David G. Johnston

Backpatch-through: 16
2024-02-01 06:11:53 -05:00
Daniel Gustafsson 7743046b75 doc: remove incorrect grammar for ALTER EVENT TRIGGER
The Parameters subsection had an extra TRIGGER in the grammar
for DISABLE/ENABLE which is incorrect.  Backpatch down to all
supported versions since it's been like this all along.

Discussion: https://postgr.es/m/0AFB171E-7E78-4A90-A140-46AB270212CA@yesql.se
Backpatch-through: v12
2024-02-01 10:45:37 +01:00
Daniel Gustafsson 0df4f75eea doc: Fix incorrect openssl option
The openssl command for displaying the DN of a client certificate was
using --subject and not the single-dash option -subject. While recent
versions of openssl handles double dash options,  earlier does not so
fix by using just -subject  (which is per the openssl documentation).

Backpatch to v14 where this was introduced.

Reported-by: konkove@gmail.com
Discussion: https://postgr.es/m/170672168899.666.10442618407194498217@wrigleys.postgresql.org
Backpatch-through: v14
2024-02-01 09:36:34 +01:00
Michael Paquier 781bc121de Fix stats_fetch_consistency with stats for fixed-numbered objects
This impacts the statistics retrieved in transactions for the following
views when updating the value of stats_fetch_consistency, leading to
behaviors contrary to what is documented since 605994651b as an update
of this parameter should discard all statistics snapshot data:
- pg_stat_archiver
- pg_stat_bgwriter
- pg_stat_checkpointer
- pg_stat_io
- pg_stat_slru
- pg_stat_wal

For example, updating stats_fetch_consistency from "snapshot" to "cache"
in a transaction did not re-fetch any fresh data, using data cached from
the time when "snapshot" was in use.

Author: Shinya Kato
Discussion: https://postgr.es/m/d77fc5190d4dbe1738d77231488e768b@oss.nttdata.com
Backpatch-through: 15
2024-02-01 17:13:09 +09:00
Michael Paquier f33e83285a Fix various issues with ALTER TEXT SEARCH CONFIGURATION
This commit addresses a set of issues when changing token type mappings
in a text search configuration when using duplicated token names:
- ADD MAPPING would fail on insertion because of a constraint failure
after inserting the same mapping.
- ALTER MAPPING with an "overridden" configuration failed with "tuple
already updated by self" when the token mappings are removed.
- DROP MAPPING failed with "tuple already updated by self", like
previously, but in a different code path.

The code is refactored so the token names (with their numbers) are
handled as a List with unique members rather than an array with numbers,
ensuring that no duplicates mess up with the catalog inserts, updates
and deletes.  The list is generated by getTokenTypes(), with the same
error handling as previously while duplicated tokens are discarded from
the list used to work on the catalogs.

Regression tests are expanded to cover much more ground for the cases
fixed by this commit, as there was no coverage for the code touched in
this commit.  A bit more is done regarding the fact that a token name
not supported by a configuration's parser should result in an error even
if IF EXISTS is used in a DROP MAPPING clause.  This is implied in the
code but there was no coverage for that, and it was very easy to miss.

These issues exist since at least their introduction in core with
140d4ebcb4, so backpatch all the way down.

Reported-by: Alexander Lakhin
Author: Tender Wang, Michael Paquier
Discussion: https://postgr.es/m/18310-1eb233c5908189c8@postgresql.org
Backpatch-through: 12
2024-01-31 13:16:43 +09:00
Andrew Dunstan 52a9af3b8b Fix 003_extrafiles.pl test for the Windows
File::Find converts backslashes to slashes in the newer Perl versions.
See: 414f14df98

So, do the same conversion for Windows before comparing paths. To
support all Perl versions, always convert them on Windows regardless of
the Perl's version.

Author: Nazir Bilal Yavuz <byavuz81@gmail.com>

Backpatch to all live branches
2024-01-30 17:11:47 -05:00
Daniel Gustafsson 5d726c72cd pgcrypto: Fix check for buffer size
The code copying the PGP block into the temp buffer failed to
account for the extra 2 bytes in the buffer which are needed
for the prefix. If the block was oversized, subsequent checks
of the prefix would have exceeded the buffer size.  Since the
block sizes are hardcoded in the list of supported ciphers it
can be verified that there is no live bug here. Backpatch all
the way for consistency though, as this bug is old.

Author: Mikhail Gribkov <youzhick@gmail.com>
Discussion: https://postgr.es/m/CAMEv5_uWvcMCMdRFDsJLz2Q8g16HEa9xWyfrkr+FYMMFJhawOw@mail.gmail.com
Backpatch-through: v12
2024-01-30 11:15:46 +01:00
David Rowley 1c8336e3e2 Doc: mention foreign keys can reference unique indexes
We seem to have only documented a foreign key can reference the columns of
a primary key or unique constraint.  Here we adjust the documentation
to mention columns in a non-partial unique index can be mentioned too.

The header comment for transformFkeyCheckAttrs() also didn't mention
unique indexes, so fix that too.  In passing make that header comment
reflect reality in the various other aspects where it deviated from it.

Bug: 18295
Reported-by: Gilles PARC
Author: Laurenz Albe, David Rowley
Discussion: https://www.postgresql.org/message-id/18295-0ed0fac5c9f7b17b%40postgresql.org
Backpatch-through: 12
2024-01-30 10:15:51 +13:00
Nathan Bossart 1b924a86e6 Move is_valid_ascii() to ascii.h.
This function requires simd.h, which is a rather large dependency
for a widely-used header file like pg_wchar.h.  Furthermore, there
is a report of a third-party tool that is struggling to use
pg_wchar.h due to its dependence on simd.h (presumably because
simd.h uses several intrinsics).  Moving the function to the much
less popular ascii.h resolves these issues for now.

This commit is back-patched for the benefit of the aforementioned
third-party tool.  The simd.h dependency was only added in v16,
but we've opted to back-patch to v15 so that is_valid_ascii() lives
in the same file for all versions where it exists.  This could
break existing third-party code that uses the function, but we
couldn't find any examples of such code.  It should be possible to
fix any code that this commit breaks by including ascii.h in the
file that uses is_valid_ascii().

Author: Jubilee Young
Reviewed-by: Tom Lane, John Naylor, Andres Freund, Eric Ridge
Discussion: https://postgr.es/m/CAPNHn3oKJJxMsYq%2BqLYzVJOFrUcOr4OF1EC-KtFT-qh8nOOOtQ%40mail.gmail.com
Backpatch-through: 15
2024-01-29 12:09:03 -06:00
Tom Lane e02fea093e Fix incompatibilities with libxml2 >= 2.12.0.
libxml2 changed the required signature of error handler callbacks
to make the passed xmlError struct "const".  This is causing build
failures on buildfarm member caiman, and no doubt will start showing
up in the field quite soon.  Add a version check to adjust the
declaration of xml_errorHandler() according to LIBXML_VERSION.

2.12.x also produces deprecation warnings for contrib/xml2/xpath.c's
assignment to xmlLoadExtDtdDefaultValue.  I see no good reason for
that to still be there, seeing that we disabled external DTDs (at a
lower level) years ago for security reasons.  Let's just remove it.

Back-patch to all supported branches, since they might all get built
with newer libxml2 once it gets a bit more popular.  (The back
branches produce another deprecation warning about xpath.c's use of
xmlSubstituteEntitiesDefault().  We ought to consider whether to
back-patch all or part of commit 65c5864d7 to silence that.  It's
less urgent though, since it won't break the buildfarm.)

Discussion: https://postgr.es/m/1389505.1706382262@sss.pgh.pa.us
2024-01-29 12:06:07 -05:00
Heikki Linnakangas b899e00e71 Fix locking when fixing an incomplete split of a GIN internal page
ginFinishSplit() expects the caller to hold an exclusive lock on the
buffer, but when finishing an earlier "leftover" incomplete split of
an internal page, the caller held a shared lock. That caused an
assertion failure in MarkBufferDirty(). Without assertions, it could
lead to corruption if two backends tried to complete the split at the
same time.

On master, add a test case using the new injection point facility.

Report and analysis by Fei Changhong. Backpatch the fix to all
supported versions.

Reviewed-by: Fei Changhong, Michael Paquier
Discussion: https://www.postgresql.org/message-id/tencent_A3CE810F59132D8E230475A5F0F7A08C8307@qq.com
2024-01-29 13:46:30 +02:00
Michael Paquier edbd1b41ab Add more LOG messages when starting and ending recovery from a backup
Three LOG messages are added in the recovery code paths, providing
information that can be useful to track corruption issues depending on
the state of the cluster, telling that:
- Recovery has started from a backup_label.
- Recovery is restarting from a backup start LSN, without a
backup_label.
- Recovery has completed from a backup.

This was originally applied on HEAD as of 1d35f705e1, and there is
consensus that this can be useful for older versions.  This applies
cleanly down to 15, so do it down to this version for now (older
versions have heavily refactored the WAL recovery paths, making the
change less straight-forward to do).

Author: Andres Freund
Reviewed-by: David Steele, Laurenz Albe, Michael Paquier
Discussion: https://postgr.es/m/20231117041811.vz4vgkthwjnwp2pp@awork3.anarazel.de
Backpatch-through: 15
2024-01-29 09:04:51 +09:00
Michael Paquier f57a580fd2 Fix DROP ROLE when specifying duplicated roles
This commit fixes failures with "tuple already updated by self" when
listing twice the same role and in a DROP ROLE query.

This is an oversight in 6566133c5f, that has introduced a two-phase
logic in DropRole() where dependencies of all the roles to drop are
removed in a first phase, with the roles themselves removed from
pg_authid in a second phase.

The code is simplified to not rely on a List of ObjectAddress built in
the first phase used to remove the pg_authid entries in the second
phase, switching to a list of OIDs.  Duplicated OIDs can be simply
avoided in the first phase thanks to that.  Using ObjectAddress was not
necessary for the roles as they are not used for anything specific to
dependency.c, building all the ObjectAddress in the List with
AuthIdRelationId as class ID.

In 15 and older versions, where a single phase is used, DROP ROLE with
duplicated role names would fail on "role \"blah\" does not exist" for
the second entry after the CCI() done by the first deletion.  This is
not really incorrect, but it does not seem worth changing based on a
lack of complaints.

Reported-by: Alexander Lakhin
Reviewed-by: Tender Wang
Discussion: https://postgr.es/m/18310-1eb233c5908189c8@postgresql.org
Backpatch-through: 16
2024-01-29 08:06:03 +09:00
Tom Lane 807369d803 Compare varnullingrels too in assign_param_for_var().
Oversight in 2489d76c4.  Preliminary analysis suggests that the
problem may be unreachable --- but if we did have instances of
the same column with different varnullingrels, we'd surely need
to treat them as different Params.

Discussion: https://postgr.es/m/412552.1706203379@sss.pgh.pa.us
2024-01-26 15:54:17 -05:00
Tom Lane 7204aea835 Detect Julian-date overflow in timestamp[tz]_pl_interval.
We perform addition of the days field of an interval via
arithmetic on the Julian-date representation of the timestamp's date.
This step is subject to int32 overflow, and we also should not let
the Julian date become very negative, for fear of weird results from
j2date.  (In the timestamptz case, allow a Julian date of -1 to pass,
since it might convert back to zero after timezone rotation.)

The additions of the months and microseconds fields could also
overflow, of course.  However, I believe we need no additional
checks there; the existing range checks should catch such cases.
The difficulty here is that j2date's magic modular arithmetic could
produce something that looks like it's in-range.

Per bug #18313 from Christian Maurer.  This has been wrong for
a long time, so back-patch to all supported branches.

Discussion: https://postgr.es/m/18313-64d2c8952d81e84b@postgresql.org
2024-01-26 13:39:37 -05:00
Thomas Munro 60ba7cae7c Track LLVM 18 changes.
A function was given a newly standard name from C++20 in LLVM 16.  Then
LLVM 18 added a deprecation warning for the old name, and it is about to
ship, so it's time to adjust that.

Back-patch to all supported releases.

Discussion: https://www.postgresql.org/message-id/CA+hUKGLbuVhH6mqS8z+FwAn4=5dHs0bAWmEMZ3B+iYHWKC4-ZA@mail.gmail.com
2024-01-25 13:45:22 +13:00
Michael Paquier 51193e7a7d Fix ALTER TABLE .. ADD COLUMN with complex inheritance trees
This command, when used to add a column on a parent table with a complex
inheritance tree, tried to update multiple times the same tuple in
pg_attribute for a child table when incrementing attinhcount, causing
failures with "tuple already updated by self" because of a missing
CommandCounterIncrement() between two updates.

This exists for a rather long time, so backpatch all the way down.

Reported-by: Alexander Lakhin
Author: Tender Wang
Reviewed-by: Richard Guo
Discussion: https://postgr.es/m/18297-b04cd83a55b51e35@postgresql.org
Backpatch-through: 12
2024-01-24 14:20:08 +09:00
Peter Eisentraut 8b31b689f4 meson: portname was set too early
portname is set to host_system, but host_system might still be changed
later in the file.  As a consequence, platforms where host_system is
changed later wouldn't find some of their platform-specific files
(driven by portname), and the build would fail.  Move the assignment
of portname further down after the last change of host_system (but
before further overriding assignments to portname).

Discussion: https://www.postgresql.org/message-id/flat/CAC7zN94TdsHhY88XkroJzSMx7E%3DBQpV9LKKjNSEnTM04ihoWCA%40mail.gmail.com
2024-01-23 12:16:29 +01:00
Heikki Linnakangas c5a6d53371 Revert "libpqwalreceiver: Convert to libpq-be-fe-helpers.h"
This reverts commit 728f86fec6.

The signal handling was a few bricks shy of a load in that commit,
which made the walreceiver non-responsive to SIGTERM while it was
waiting for the connection to be established. That prevented a standby
from being promoted.

Since it was non-essential refactoring, let's revert it to make v16
work the same as earlier releases. I reverted it in 'master' too, to
keep the branches in sync. The refactoring was a good idea as such,
but it needs a bit more work. Once we have developed a complete patch
with this issue fixed, let's re-apply that to 'master'.

Reported-by: Kyotaro Horiguchi
Backpatch-through: 16
Discussion: https://www.postgresql.org/message-id/20231231.200741.1078989336605759878.horikyota.ntt@gmail.com
2024-01-23 10:53:23 +02:00
Michael Paquier cae25d110e Improve stability of recovery test 035_standby_logical_decoding
This commit tweaks a couple of things in 035_standby_logical_decoding to
hopefully stabilize it:
- Autovacuum is now disabled, as it could hold a global xmin with a
transaction.
- Conflicts are generated with command sequences that removed rows (on
catalogs, shared or non-shared, or just plain relations) followed by a
VACUUM.  This was unstable because these did not check that the horizon
moved between the SQL commands and the VACUUM.  The logic is refactored
as follows, to ensure that VACUUM removes dead rows before testing for
slot invalidation on a standby (idea suggested by Andres Freund):
-- Grab the current horizon.
-- Launch SQL commands removing rows.
-- Check that the snapshot horizon has been updated.
-- Launch VACUUM on the relation whose rows have been removed by the
first step.

Note that there are still some issues because of standby snapshot WAL
records generated by the bgwriter, but this makes the test much more
stable.

Per reports from buildfarm members dikkop and skink, with analysis and
tests from Alexander Lakhin.

While on it, fix a couple of incorrect comments.

Author: Bertrand Drouvot
Reviewed-by: Alexander Lakhin, Michael Paquier
Discussion: https://postgr.es/m/OSZPR01MB6310ED3CEDB531BCEDBC6AF2FD479@OSZPR01MB6310.jpnprd01.prod.outlook.com
Discussion: https://postgr.es/m/bf67e076-b163-9ba3-4ade-b9fc51a3a8f6@gmail.com
Backpatch-through: 16
2024-01-23 14:46:05 +09:00
Alvaro Herrera 07b53de700
Abort pgbench if script end is reached with an open pipeline
When a pipeline is opened with \startpipeline and not closed, pgbench
will either error on the next transaction with a "already in pipeline
mode" error or successfully end if this was the last transaction --
despite not sending anything that was piped in the pipeline.

Make it an error to reach end of script is reached while there's an
open pipeline.

Backpatch to 14, where pgbench got support for pipelines.

Author: Anthonin Bonnefoy <anthonin.bonnefoy@datadoghq.com>
Reported-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/Za4IObZkDjrO4TcS@paquier.xyz
2024-01-22 17:48:30 +01:00
David Rowley 74f770ef20 Re-disallow Memoize for parameterized nested loops with join filters
This was previously fixed in 9e215378d but got broken again as a result
of 2489d76c4.  It seems that commit causes ppi_clauses to contain
duplicate clauses and it's no longer safe to check the list_length of
that list to determine if there are join conditions other than what's
mentioned in ppi_clauses.

Here we adjust the check to count the distinct rinfo_serial mentioned in
ppi_clauses.  We expect that extra->restrictlist won't have duplicate
rinfo_serials.

Reported-by: Amadeo Gallardo
Author: Richard Guo
Discussion: https://postgr.es/m/CADFREbW-BLJd7-a5J%2B5wjVumeFG1ByXiSOFzMtkmY_SDWckTxw%40mail.gmail.com
Backpatch-through: 16, where 2489d76c4 was introduced.
2024-01-22 22:45:33 +13:00