Commit Graph

7375 Commits

Author SHA1 Message Date
Alvaro Herrera bcdfa5f2e2
Rename SLRU elements in view pg_stat_slru
The new names are intended to match those in an upcoming patch that adds
a few GUCs to configure the SLRU buffer sizes.

Backwards compatibility concern: this changes the accepted names for
function pg_stat_slru_rest().  Since this function recognizes "any other
string" as a request to reset the entry for "other", this means that
calling it with the old names would silently reset "other" instead of
doing nothing or throwing an error.

Reviewed-by: Andrey M. Borodin <x4mmm@yandex-team.ru>
Discussion: https://postgr.es/m/202402261616.dlriae7b6emv@alvherre.pgsql
2024-02-28 09:39:52 +01:00
Andrew Dunstan 92d2ab7554 Rationalize and improve error messages for some jsonpath items
This is a followup to commit 66ea94e8e6.

Error mssages concerning incorrect formats for date-time types are
unified and parameterized, instead of using a fully separate error
message for each type.

Similarly, error messages regarding numeric and string arguments to
certain items are standardized, and instead of saying that the argument
is out of range simply say that it is invalid. The actual invalid
arguments to these itesm are now shown in the error message.

Error messages relating to numeric inputs of Nan or Infinity are
made more informative.

Jeevan Chalke and Kyotaro Horiguchi, with some input from Tom Lane.

Discussion: https://postgr.es/m/20240129.121200.235012930453045390.horikyota.ntt@gmail.com
2024-02-27 02:07:22 -05:00
Michael Paquier 449e798c77 Introduce sequence_*() access functions
Similarly to tables and indexes, these functions are able to open
relations with a sequence relkind, which is useful to make a distinction
with the other relation kinds.  Previously, commands/sequence.c used a
mix of table_{close,open}() and relation_{close,open}() routines when
manipulating sequence relations, so this clarifies the code.

A direct effect of this change is to align the error messages produced
when attempting DDLs for sequences on relations with an unexpected
relkind, like a table or an index with ALTER SEQUENCE, providing an
extra error detail about the relkind of the relation used in the DDL
query.

Author: Michael Paquier
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/ZWlohtKAs0uVVpZ3@paquier.xyz
2024-02-26 16:04:59 +09:00
Alexander Korotkov a661bf7b0f Remove flaky isolation tests for timeouts
51efe38cb9 introduced bunch of tests for idle_in_transaction_session_timeout,
transaction_timeout and statement_timeout. These tests were too flaky on some
slow buildfarm machines, so we plan to replace them with TAP tests using
injection points. This commit removes flaky tests.

Discussion: https://postgr.es/m/CAAhFRxiQsRs2Eq5kCo9nXE3HTugsAAJdSQSmxncivebAxdmBjQ%40mail.gmail.com
Author: Andrey Borodin
2024-02-25 20:20:04 +02:00
Alexander Korotkov 874d817baa Multiple revisions to the GROUP BY reordering tests
Discussion: https://postgr.es/m/CAMbWs4-NKLa%2BSs%2BX%3DWR6h0x%3DT07YBJoAz70ZGHzc-2zcHUHb0A%40mail.gmail.com
Author: Richard Guo
Reviewed-by: Andrei Lepikhov, Alexander Korotkov
2024-02-24 01:49:37 +02:00
Alexander Korotkov 466979ef03 Replace lateral references to removed rels in subqueries
This commit introduces a new field 'sublevels_up' in ReplaceVarnoContext,
and enhances replace_varno_walker() to:
  1) recurse into subselects with sublevels_up increased, and
  2) perform the replacement only when varlevelsup is equal to sublevels_up.

This commit also fixes some outdated comments.  And besides adding relevant
test cases, it makes some unification over existing SJE test cases.

Discussion: https://postgr.es/m/CAMbWs4-%3DPO6Mm9gNnySbx0VHyXjgnnYYwbN9dth%3DTLQweZ-M%2Bg%40mail.gmail.com
Author: Richard Guo
Reviewed-by: Andrei Lepikhov, Alexander Korotkov
2024-02-24 00:35:17 +02:00
Amit Kapila d13ff82319 Fix BF failure in commit 93db6cbda0.
The code to match the required LOG in the test was not robust enough to
match it. It was using a very specific format to search the required
message which doesn't work when one uses log_error_verbosity = verbose.

Author: Hou Zhijie
Discussion: https://postgr.es/m/CAA4eK1KcQSk7wzC7Zfrth9OhrjW2HvxL4tKgU42qqH7p6jn+FA@mail.gmail.com
2024-02-22 18:26:40 +05:30
Amit Kapila 93db6cbda0 Add a new slot sync worker to synchronize logical slots.
By enabling slot synchronization, all the failover logical replication
slots on the primary (assuming configurations are appropriate) are
automatically created on the physical standbys and are synced
periodically. The slot sync worker on the standby server pings the primary
server at regular intervals to get the necessary failover logical slots
information and create/update the slots locally. The slots that no longer
require synchronization are automatically dropped by the worker.

The nap time of the worker is tuned according to the activity on the
primary. The slot sync worker waits for some time before the next
synchronization, with the duration varying based on whether any slots were
updated during the last cycle.

A new parameter sync_replication_slots enables or disables this new
process.

On promotion, the slot sync worker is shut down by the startup process to
drop any temporary slots acquired by the slot sync worker and to prevent
the worker from trying to fetch the failover slots.

A functionality to allow logical walsenders to wait for the physical will
be done in a subsequent commit.

Author: Shveta Malik, Hou Zhijie based on design inputs by Masahiko Sawada and Amit Kapila
Reviewed-by: Masahiko Sawada, Bertrand Drouvot, Peter Smith, Dilip Kumar, Ajin Cherian, Nisha Moond, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-02-22 15:25:15 +05:30
Amit Kapila b6df0798a5 Fix the intermittent buildfarm failures in 031_column_list.
The reason was that the ALTER SUBSCRIPTION .. SET PUBLICATION will lead to
the restarting of apply worker and after the restart, the apply worker
will use the existing slot and replication origin corresponding to the
subscription. Now, it is possible that before restart the origin has not
been updated and the WAL start location points to a location before where
PUBLICATION exists which can lead to the error "publication ... does not
exist".

Fix it by recreating the subscription as a newly created subscription will
start processing WAL from the recent WAL location and will see the
required publication.

This behavior has existed from the time logical replication was introduced
but is exposed by this test and we have started a discussion for a better
fix for this problem.

As per Buildfarm

Diagnosed-by: Amit Kapila
Author: Vignesh C
Discussion: https://postgr.es/m/3307255.1706911634@sss.pgh.pa.us
2024-02-22 12:06:44 +05:30
Amit Kapila 801792e528 Improve ERROR/LOG messages added by commits ddd5f4f54a and 7a424ece48.
Additionally, in slotsync.c, replace one StringInfoData variable usage
with a constant string to avoid palloc/pfree. Also, replace the inclusion
of "logical.h" with "slot.h" to prevent the exposure of unnecessary
implementation details.

Reported-by: Kyotaro Horiguchi, Masahiko Sawada
Author: Shveta Malik based on suggestions by Robert Haas and Amit Kapila
Reviewed-by: Kyotaro Horiguchi, Amit Kapila
Discussion: https://postgr.es/m/20240214.162652.773291409747353211.horikyota.ntt@gmail.com
Discussion: https://postgr.es/m/20240219.134015.1888940527023074780.horikyota.ntt@gmail.com
Discussion: https://postgr.es/m/CAD21AoCYXhDYOQDAS-rhGasC2T+tYbV=8Y18o94sB=5AxcW+yA@mail.gmail.com
2024-02-22 11:17:00 +05:30
Alvaro Herrera 4989ce7264
MERGE ... DO NOTHING: require SELECT privileges
Verify that a user running MERGE with a DO NOTHING clause has
privileges to read the table, even if no columns are referenced.  Such
privileges were already required if the ON clause or any of the WHEN
conditions referenced any column at all, so there's no functional change
in practice.

This change fixes an assertion failure in the case where no column is
referenced by the command and the WHEN clauses are all DO NOTHING.

Backpatch to 15, where MERGE was introduced.

Reported-by: Alena Rybakina <a.rybakina@postgrespro.ru>
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/4d65a385-7efa-4436-a825-0869f89d9d92@postgrespro.ru
2024-02-21 17:18:52 +01:00
Michael Paquier ff9e1e764f Add option force_initdb to PostgreSQL::Test::Cluster:init()
This option is useful to bypass the default behavior of init() which
would create the data folder of a new cluster by copying it from a
template previously initdb'd, if any.  Copying the data folder is much
cheaper than running initdb, but some tests may want to force that.  For
example, one scenario of pg_combinebackup updated in this commit needs a
different system ID for two nodes.

Previously, this could only be achieved by unsetting
$ENV{'INITDB_TEMPLATE'}, which could become a problem in complex node
setups by making tests less efficient.

Author: Amul Sul
Reviewed-by: Robert Haas, Michael Paquier
Discussion: https://postgr.es/m/Zc1tX9lLonLGu6oH@paquier.xyz
2024-02-21 13:28:51 +09:00
Alexander Korotkov 489072ab7a Replace relids in lateral subquery parse tree during SJE
Reported-by: Alexander Lakhin
Discussion: https://postgr.es/m/56ee4520-e9d1-d519-54fe-c8bff880ce9b%40gmail.com
Author: Alexander Korotkov, Andrei Lepikhov
2024-02-20 14:10:10 +02:00
Peter Eisentraut 74563f6b90 Revert "Improve compression and storage support with inheritance"
This reverts commit 0413a55699.

pg_dump cannot currently dump all the structures that are allowed by
this patch.  This needs more work in pg_dump and more test coverage.

Discussion: https://www.postgresql.org/message-id/flat/24656cec-d6ef-4d15-8b5b-e8dfc9c833a7@eisentraut.org
2024-02-20 11:10:59 +01:00
David Rowley 4c2369ac5d Fix incorrect pruning of NULL partition for boolean IS NOT clauses
Partition pruning wrongly assumed that, for a table partitioned on a
boolean column, a clause in the form "boolcol IS NOT false" and "boolcol
IS NOT true" could be inverted to correspondingly become "boolcol IS true"
and "boolcol IS false".  These are not equivalent as the NOT version
matches the opposite boolean value *and* NULLs.  This incorrect assumption
meant that partition pruning pruned away partitions that could contain
NULL values.

Here we fix this by correctly not pruning partitions which could store
NULLs.

To be affected by this, the table must be partitioned by a NULLable boolean
column and queries would have to contain "boolcol IS NOT false" or "boolcol
IS NOT true".  This could result in queries filtering out NULL values
with a LIST partitioned table and "ERROR:  invalid strategy number 0"
for RANGE and HASH partitioned tables.

Reported-by: Alexander Lakhin
Bug: #18344
Discussion: https://postgr.es/m/18344-8d3f00bada6d09c6@postgresql.org
Backpatch-through: 12
2024-02-20 12:49:37 +13:00
Noah Misch 0e162810df Fix test race between primary XLOG_RUNNING_XACTS and standby logical slot.
Before the previous commit, the test could hang until
LOG_SNAPSHOT_INTERVAL_MS (15s), until checkpoint_timeout (300s), or
indefinitely.  An indefinite hang was awfully improbable.  It entailed
the test reaching checkpoint_timeout before the
DecodingContextFindStartpoint() of a CREATE SUBSCRIPTION, yet after the
preceding WAL record.  Back-patch to v16, which introduced the test.

Bertrand Drouvot, reported by Noah Misch.

Discussion: https://postgr.es/m/20240211010227.a2.nmisch@google.com
2024-02-19 12:52:28 -08:00
Noah Misch 4791f87f34 Bound waits in 035_standby_logical_decoding.pl.
One IPC::Run::start() used an IPC::Run::timer() without checking for
expiration.  The other used no timeout or timer.  Back-patch to v16,
which introduced the test.

Reviewed by Bertrand Drouvot.

Discussion: https://postgr.es/m/20240211010227.a2.nmisch@google.com
2024-02-19 12:52:07 -08:00
Alexander Korotkov e1b7fde418 Get rid of pg_class usage in SJE regression tests
Usage of pg_class led to instability, see the buildfarm failure.
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=prion&dt=2024-02-15%2021%3A58%3A04

Reported-by: Andrei Lepikhov
Discussion: https://postgr.es/m/250c4cec-e459-4311-8fd6-da2558b3fb0a@postgrespro.ru
2024-02-19 14:11:50 +02:00
Alexander Korotkov a6c21887a9 Remove remaining references to timeouts-long
Reported-by: Alexander Lakhin
2024-02-16 23:59:27 +02:00
Alexander Korotkov eb49e1bdd9 Remove timeouts-long.out
bf82f43790 removes timeouts-long.spec, but forgets to remove timeouts-long.out.

Reported-by: Alexander Lakhin
2024-02-16 15:57:37 +02:00
Peter Eisentraut 0413a55699 Improve compression and storage support with inheritance
A child table can specify a compression or storage method different
from its parents.  This was previously an error.  (But this was
inconsistently enforced because for example the settings could be
changed later using ALTER TABLE.)  This now also allows an explicit
override if multiple parents have different compression or storage
settings, which was previously an error that could not be overridden.

The compression and storage properties remains unchanged in a child
inheriting from parent(s) after its creation, i.e., when using ALTER
TABLE ...  INHERIT.  (This is not changed.)

Before this change, the error detail would mention the first pair of
conflicting parent compression or storage methods.  But with this
change it waits till the child specification is considered by which
time we may have encountered many such conflicting pairs.  Hence the
error detail after this change does not include the conflicting
compression/storage methods.  Those can be obtained from parent
definitions if necessary.  The code to maintain list of all
conflicting methods or even the first conflicting pair does not seem
worth the convenience it offers.  This change is inline with what we
do with conflicting default values.

Before this commit, the specified storage method could be stored in
ColumnDef::storage (CREATE TABLE ... LIKE) or ColumnDef::storage_name
(CREATE TABLE ...).  This caused the MergeChildAttribute() and
MergeInheritedAttribute() to ignore a storage method specified in the
child definition since it looked only at ColumnDef::storage.  This
commit removes ColumnDef::storage and instead uses
ColumnDef::storage_name to save any storage method specification. This
is similar to how compression method specification is handled.

Author: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/24656cec-d6ef-4d15-8b5b-e8dfc9c833a7@eisentraut.org
2024-02-16 13:27:46 +01:00
Amit Kapila b7bdade6a4 Disable autovacuum on primary in 040_standby_failover_slots_sync test.
Disable autovacuum to avoid generating xid during stats update as
otherwise the new XID could then be replicated to standby at some random
point making slots at primary lag behind standby during slot sync.

As per buildfarm

Author: Hou Zhijie
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
Discussion: https://postgr.es/m/CAA4eK1Jun8SGCoc6JEktxY_+L7GmoJWrdsx-KCEP=GL-SsWggQ@mail.gmail.com
2024-02-16 14:42:50 +05:30
Amit Kapila d9e225f275 Change the LOG level in 040_standby_failover_slots_sync.pl to DEBUG2.
Temporarily change the log level of 040_standby_failover_slots_sync.pl to
DEBUG2. This is to get more information about BF failures. We will reset
it back to default once the tests are stabilized.

Author: Hou Zhijie
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
Discussion: https://postgr.es/m/OS0PR01MB571633C23B2A4CAC5FB0371A944C2@OS0PR01MB5716.jpnprd01.prod.outlook.com
2024-02-16 10:13:51 +05:30
David Rowley 1fe66680c0 Attempt to stabilize flapping regression test
Per buildfarm animal mylodon, the plan for this test was sometimes
swapping the join order for tenk1 and tenk2.  Given that add_path() has
no code that would cause this fluctuation when given paths with consistent
costs, this indicates that the costs must be fluctuating in some runs.
The only proven reason I've seen where that could happen was slight
variations in pg_class.relpages for some tables.  This was demonstrated to
be true by f03a9ca43 and related discussion.  Manually adjusting tenk2's
pg_class.relpages by subtracting just 1 page does cause the plan to change
for this test.

Here we've not gone to the same lengths to prove that's what's going on
in this case.  Proving that does not seem worth the time.  Let's just
shrink one side of the join so the additional cost of the swapped join
order is sufficiently different that if the relpages estimate is off a few
pages that the planner still shouldn't swap the join order.

Reported-by: Thomas Munro
Author: Andy Fan, David Rowley
Discussion: https://postgr.es/m/CA+hUKGLqC-NobKYfjxNM3Gexv9OJ-Fhvy9bugUcXsZjTqH7W=Q@mail.gmail.com
2024-02-16 15:01:29 +13:00
Alexander Korotkov bf82f43790 Followup fixes for transaction_timeout
Don't deal with transaction timeout in PostgresMain().  Instead, release
transaction timeout activated by StartTransaction() in
CommitTransaction()/AbortTransaction()/PrepareTransaction().  Deal with both
enabling and disabling transaction timeout in assign_transaction_timeout().

Also, remove potentially flaky timeouts-long isolation test, which has no
guarantees to pass on slow/busy machines.

Reported-by: Andres Freund
Discussion: https://postgr.es/m/20240215230856.pc6k57tqxt7fhldm%40awork3.anarazel.de
2024-02-16 03:36:38 +02:00
Alexander Korotkov 51efe38cb9 Introduce transaction_timeout
This commit adds timeout that is expected to be used as a prevention
of long-running queries. Any session within the transaction will be
terminated after spanning longer than this timeout.

However, this timeout is not applied to prepared transactions.
Only transactions with user connections are affected.

Discussion: https://postgr.es/m/CAAhFRxiQsRs2Eq5kCo9nXE3HTugsAAJdSQSmxncivebAxdmBjQ%40mail.gmail.com
Author: Andrey Borodin <amborodin@acm.org>
Author: Japin Li <japinli@hotmail.com>
Author: Junwang Zhao <zhjwpku@gmail.com>
Reviewed-by: Nikolay Samokhvalov <samokhvalov@gmail.com>
Reviewed-by: Andres Freund <andres@anarazel.de>
Reviewed-by: Fujii Masao <masao.fujii@oss.nttdata.com>
Reviewed-by: bt23nguyent <bt23nguyent@oss.nttdata.com>
Reviewed-by: Yuhang Qiu <iamqyh@gmail.com>
2024-02-15 23:56:12 +02:00
Alexander Korotkov 9f13376396 Pull up ANY-SUBLINK with the necessary lateral support.
For ANY-SUBLINK, we adopted a two-stage pull-up approach to handle
different types of scenarios. In the first stage, the sublink is pulled up
as a subquery. Because of this, when writing this code, we did not have
the ability to perform lateral joins, and therefore, we were unable to
pull up Var with varlevelsup=1. Now that we have the ability to use
lateral joins, we can eliminate this limitation.

Author: Andy Fan <zhihui.fan1213@gmail.com>
Author: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Alena Rybakina <lena.ribackina@yandex.ru>
Reviewed-by: Andrey Lepikhov <a.lepikhov@postgrespro.ru>
2024-02-15 12:06:12 +02:00
Peter Eisentraut 995d400cec Allow passing extra options to initdb for tests
Setting the environment variable PG_TEST_INITDB_EXTRA_OPTS passes
extra options to initdb run by pg_regress or
PostgreSQL::Test::Cluster's init.

This can be useful for a wide variety of uses, like running all tests
with checksums enabled, or with JIT enabled, or with different GUC
settings, or with different locale settings.  (Not all tests are going
to pass with arbitrary options, but it is useful to run this against
specific test suites.)

Reviewed-by: Ian Lawrence Barwick <barwick@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/d4d2ad9f-1c1d-47a1-bb4d-c10a747d4f15%40eisentraut.org
2024-02-15 10:29:18 +01:00
Amit Kapila 9bc1eee988 Another try to fix BF failure introduced in commit ddd5f4f54a.
Before attempting to sync the slot on standby by
pg_sync_replication_slots(), ensure that on the primary restart_lsn for
the slot has moved to a recent WAL position, by re-creating the
subscription and the logical slot.

Author: Hou Zhijie
Discussion: https://postgr.es/m/CAA4eK1+d5Lne8vCAn0un4SP9x-ZBr2-xfxg01uSfeBTSCKFZoQ@mail.gmail.com
2024-02-15 10:37:28 +05:30
Nathan Bossart 8d8afd48d3 Allow pg_monitor to execute pg_current_logfile().
We allow roles with privileges of pg_monitor to execute functions
like pg_ls_logdir(), so it seems natural that such roles would also
be able to execute this function.

Bumps catversion.

Co-authored-by: Pavlo Golub
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CAK7ymcLmEYWyQkiCZ64WC-HCzXAB0omM%3DYpj9B3rXe8vUAFMqw%40mail.gmail.com
2024-02-14 11:48:29 -06:00
Tom Lane 3e8235ba4f Fix multiranges to behave more like dependent types.
For most purposes, multiranges act like dependent objects of the
associated range type: you can't create them separately or drop them
separately.  This is like the way that autogenerated array types
behave.  However, a couple of points were overlooked: array types
automatically track the ownership of their base type, and array types
do not have their own permissions but use those of the base type,
while multiranges didn't emulate those behaviors.  This is fairly
broken, mainly because pg_dump doesn't think it needs to worry about
multiranges as separate objects, and thus it fails to dump/restore
ownership or permissions of multiranges.

There's no apparent value in letting a multirange diverge from
its parent's ownership or permissions, so let's make them act like
arrays in these respects.  However, we continue to let multiranges
be renamed or moved to a different schema independently of their
parent, since that doesn't break anything.

Discussion: https://postgr.es/m/1580383.1705343264@sss.pgh.pa.us
2024-02-14 11:30:39 -05:00
Amit Kapila bd8fc1677b Fix BF introduced in commit ddd5f4f54a.
The failure is that the remote slot is not synchronized after the same
slot on standby gets invalidated. The reason was that remote_slot's
restart_lsn was lagged behind the standby's oldest WAL segment. The test
didn't ensure that remote_slot's LSN was advanced to the latest position
before we tried to sync the slots via new the function
pg_sync_replication_slots().

Discussion: https://postgr.es/m/CAA4eK1JLBi3HzenB6do3_hd78kN0UDD1mz-vumWE52XHHEq5Bw@mail.gmail.com
2024-02-14 16:16:08 +05:30
Amit Kapila ddd5f4f54a Add a slot synchronization function.
This commit introduces a new SQL function pg_sync_replication_slots()
which is used to synchronize the logical replication slots from the
primary server to the physical standby so that logical replication can be
resumed after a failover or planned switchover.

A new 'synced' flag is introduced in pg_replication_slots view, indicating
whether the slot has been synchronized from the primary server. On a
standby, synced slots cannot be dropped or consumed, and any attempt to
perform logical decoding on them will result in an error.

The logical replication slots on the primary can be synchronized to the
hot standby by using the 'failover' parameter of
pg-create-logical-replication-slot(), or by using the 'failover' option of
CREATE SUBSCRIPTION during slot creation, and then calling
pg_sync_replication_slots() on standby. For the synchronization to work,
it is mandatory to have a physical replication slot between the primary
and the standby aka 'primary_slot_name' should be configured on the
standby, and 'hot_standby_feedback' must be enabled on the standby. It is
also necessary to specify a valid 'dbname' in the 'primary_conninfo'.

If a logical slot is invalidated on the primary, then that slot on the
standby is also invalidated.

If a logical slot on the primary is valid but is invalidated on the
standby, then that slot is dropped but will be recreated on the standby in
the next pg_sync_replication_slots() call provided the slot still exists
on the primary server. It is okay to recreate such slots as long as these
are not consumable on standby (which is the case currently). This
situation may occur due to the following reasons:
- The 'max_slot_wal_keep_size' on the standby is insufficient to retain
WAL records from the restart_lsn of the slot.
- 'primary_slot_name' is temporarily reset to null and the physical slot
is removed.

The slot synchronization status on the standby can be monitored using the
'synced' column of pg_replication_slots view.

A functionality to automatically synchronize slots by a background worker
and allow logical walsenders to wait for the physical will be done in
subsequent commits.

Author: Hou Zhijie, Shveta Malik, Ajin Cherian based on an earlier version by Peter Eisentraut
Reviewed-by: Masahiko Sawada, Bertrand Drouvot, Peter Smith, Dilip Kumar, Nisha Moond, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-02-14 09:45:36 +05:30
Tom Lane 5ebc9c9017 Catch overflow when rounding intervals in AdjustIntervalForTypmod.
Previously, an interval microseconds field close to INT64_MAX or
INT64_MIN could overflow, producing a result with not even the
correct sign, while being rounded to match a precision specification.

This seems worth fixing, but not worth back-patching, in part
because the ereturn() notation doesn't exist very far back.

Report and patch by Joseph Koshakow (some cosmetic mods by me)

Discussion: https://postgr.es/m/CAAvxfHfpuLgqJYzkUcher466Z1LpmE+5Sm+zc8L6zKCOQ+6TDQ@mail.gmail.com
2024-02-13 15:58:40 -05:00
Tom Lane 0736a8ef6f Use a safer outfuncs/readfuncs representation for BitStrings.
For a long time, our outfuncs.c code has supposed that the string
contents of a BitString node could just be printed literally with
no concern for quoting/escaping.  Now, that's okay if the string
literal contains only valid binary or hex digits ... but our lexer
doesn't check that, preferring to let bitin() be the sole authority
on what's valid.  So we could have raw parse trees that contain
incorrect BitString literals, and that can result in failures when
WRITE_READ_PARSE_PLAN_TREES debugging is enabled.

Fix by using outToken() to print the string field, and debackslash()
to read it.  This results in a change in the emitted representation
only in cases that would have failed before, and don't represent valid
SQL in the first place.  Between that and the fact that we don't store
raw parse trees in the catalogs, I judge this safe to apply without a
catversion bump.

Per bug #18340 from Alexander Lakhin.  Back-patch to v16; before that,
we lacked readfuncs support for BitString nodes, so that the problem
was only cosmetic.

Discussion: https://postgr.es/m/18340-4aa1ae6ed4121912@postgresql.org
2024-02-13 12:18:25 -05:00
Peter Eisentraut e70abd67c3 Use extensible buffers to assemble command lines
This makes use of StringInfo to assemble command lines, instead of
using fixed-size buffers and the (remote) possibility of "command too
long" errors.  Also makes the code a bit simpler.

This covers the test driver programs pg_regress and
pg_isolation_regress.

Similar to the changes done for pg_rewind in a33e17f210.

Discussion: https://www.postgresql.org/message-id/2be4fee5-738f-4749-b9f8-b452032c7ade%40eisentraut.org
2024-02-11 09:27:48 +01:00
Andrew Dunstan 4697454686 Disallow jsonpath methods involving TZ in immutable functions
Timezones are not immutable and so neither is any function that relies on
them. In commit 66ea94e8, we introduced a few methods which do casting
from one time to another and thus may involve the current timezone.  To
preserve the immutability of jsonpath functions currently marked
immutable, disallow these methods from being called from non-TZ aware
functions.

Jeevan Chalke, per a report from Jian He.
2024-02-10 12:12:39 -05:00
Alexander Korotkov c01f6ef46c Fix usage of aggregate pathkeys in group_keys_reorder_by_pathkeys()
group_keys_reorder_by_pathkeys() function searched for matching pathkeys
within root->group_pathkeys.  That could lead to picking an aggregate pathkey
and using its pathkey->pk_eclass->ec_sortref as an argument of
get_sortgroupref_clause_noerr().  Given that ec_sortref of an aggregate pathkey
references aggregate targetlist not query targetlist, this leads to incorrect
query optimization.

Fix this by looking for matching pathkeys only within the first
num_groupby_pathkeys pathkeys.

Reported-by: David G. Johnston
Discussion: https://postgr.es/m/CAKFQuwY3Ek%3DcLThgd8FdaSc5JRDVt0FaV00gMcWra%2BTAR4gGUw%40mail.gmail.com
Author: Andrei Lepikhov, Alexander Korotkov
2024-02-09 12:56:54 +02:00
Peter Eisentraut 6743c5ae64 Fix propagation of persistence to sequences in ALTER TABLE / ADD COLUMN
Fix for 344d62fb9a9: That commit introduced unlogged sequences and
made it so that identity/serial sequences automatically get the
persistence level of their owning table.  But this works only for
CREATE TABLE and not for ALTER TABLE / ADD COLUMN.  The latter would
always create the sequence as logged (default), independent of the
persistence setting of the table.  This is fixed here.

Note: It is allowed to change the persistence of identity sequences
directly using ALTER SEQUENCE.  So mistakes in existing databases can
be fixed manually.

Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/c4b6e2ed-bcdf-4ea7-965f-e49761094827%40eisentraut.org
2024-02-09 08:09:22 +01:00
Peter Eisentraut a89fa00714 Apply pg_dump test cleanups to test_pg_dump as well
Apply the changes from 41a284411e to the test_pg_dump module as well.
Here, we just apply the new test consistency checks, but we don't need
to fix any existing tests.

Discussion: https://www.postgresql.org/message-id/flat/1f8cb371-e84e-434e-0367-6b716fb16fa1@eisentraut.org
2024-02-08 21:19:03 +01:00
Masahiko Sawada 1aa67a5ea6 Fix meson installation of xid_wraparound test.
Fix for e255b646a, to prevent installation of xid_wraparound test
module during main install.

Reported-by: Peter Eisentraut
Discussion: https://postgr.es/m/84cd416a-0e37-4019-8380-1c8a3cdd8c5c%40eisentraut.org
2024-02-08 17:03:59 +09:00
Heikki Linnakangas b96115acb8 Fix assertion if index is dropped during REFRESH CONCURRENTLY
When assertions are disabled, the built SQL statement is invalid and
you get a "syntax error". So this isn't a serious problem, but let's
avoid the assertion failure.

Backpatch to all supported versions.

Reviewed-by: Noah Misch
2024-02-05 11:01:30 +02:00
David Rowley 7e0ade0ffe Allow Gather Merge in more cases for parallel DISTINCT
Here we adjust the partial path generation for parallel DISTINCT queries
to add Sort nodes on top of any unsorted partial distinct paths.

This increases the likelihood of the planner pushing a Sort below a Gather
Merge which enables the final phase of the parallel distinct to be
implemented using a Unique node in more cases.

Sorting the partial distinct paths is particularly useful when the
DISTINCT query has an ORDER BY and LIMIT clause as this can allow cheaper
plans by having the workers Hash Aggregate then Sort before feeding the
results into the Gather Merge.  The non-parallel portion of the plan then
becomes very cheap as it leaves only Unique and Limit to do in the leader
process.

Author: Richard Guo
Reviewed-by: David Rowley
Discussion: https://postgr.es/m/CAMbWs48u9VoVOouJsys1qOaC9WVGVmBa+wT1dx8KvxF5GPzezA@mail.gmail.com
2024-02-03 00:20:18 +13:00
David Rowley 9d1a5354f5 Fix costing bug in MergeAppend
When building a MergeAppendPath which has child paths that are not
sorted correctly for the MergeAppend's sort order, we apply the cost of
sorting those paths to the MergeAppendPath costs.

Here we fix a bug where the number of tuples specified that needed to be
sorted was effectively pg_class.reltuples rather than the number of
expected row in the subpath.  This effectively penalizes MergeAppend
plans any time any filter is present on the MergeAppend subpath as the
sort cost added is to sort all tuples in the table rather than just the
ones expected the path to return.

This did not affect UNION ALL type queries as the RelOptInfo tuples is
set from the subquery's path rows.  It does affect MergeAppends uses for
inheritance and partitioned tables.

This is a long-standing bug introduced when MergeAppend was first added
in 11cad29c9.  No backpatch as this could result in plan changes.

Author: Alexander Kuzmenkov
Reviewed-by: Ashutosh Bapat, Aleksander Alekseev, David Rowley
Discussion: https://postgr.es/m/CALzhyqyhoXQDR-Usd_0HeWk%3DuqNLzoVeT8KhRoo%3DpV_KzgO3QQ%40mail.gmail.com
2024-02-01 09:48:26 +13:00
Michael Paquier 0fda392138 Add .gitignore to src/test/modules/gin/
This has been forgotten in 6a1ea02c49.
2024-01-31 15:12:22 +09:00
Michael Paquier 5f19412e4e Add tests for int4_bool() in int.c
This cast was previously not covered at all by the regression tests.

Author: Christoph Berg
Discussion: https://postgr.es/m/ZYQZ1hNfLd_4rzkn@msg.df7cb.de
2024-01-31 15:02:28 +09:00
David Rowley b588cad688 Consider the "LIMIT 1" optimization with parallel DISTINCT
Similar to what was done in 5543677ec for non-parallel DISTINCT, apply
the same optimization when the distinct_pathkeys are empty for the
partial paths too.

This can be faster than the non-parallel version when the first row
matching the WHERE clause of the query takes a while to find.  Parallel
workers could speed that process up considerably.

Author: Richard Guo
Reviewed-by: David Rowley
Discussion: https://postgr.es/m/CAMbWs49JC0qvfUbzs-TVzgMpSSBiMJ_6sN=BaA9iohBgYkr=LA@mail.gmail.com
2024-01-31 17:22:02 +13:00
Michael Paquier 3e91dba8b0 Fix various issues with ALTER TEXT SEARCH CONFIGURATION
This commit addresses a set of issues when changing token type mappings
in a text search configuration when using duplicated token names:
- ADD MAPPING would fail on insertion because of a constraint failure
after inserting the same mapping.
- ALTER MAPPING with an "overridden" configuration failed with "tuple
already updated by self" when the token mappings are removed.
- DROP MAPPING failed with "tuple already updated by self", like
previously, but in a different code path.

The code is refactored so the token names (with their numbers) are
handled as a List with unique members rather than an array with numbers,
ensuring that no duplicates mess up with the catalog inserts, updates
and deletes.  The list is generated by getTokenTypes(), with the same
error handling as previously while duplicated tokens are discarded from
the list used to work on the catalogs.

Regression tests are expanded to cover much more ground for the cases
fixed by this commit, as there was no coverage for the code touched in
this commit.  A bit more is done regarding the fact that a token name
not supported by a configuration's parser should result in an error even
if IF EXISTS is used in a DROP MAPPING clause.  This is implied in the
code but there was no coverage for that, and it was very easy to miss.

These issues exist since at least their introduction in core with
140d4ebcb4, so backpatch all the way down.

Reported-by: Alexander Lakhin
Author: Tender Wang, Michael Paquier
Discussion: https://postgr.es/m/18310-1eb233c5908189c8@postgresql.org
Backpatch-through: 12
2024-01-31 13:15:21 +09:00
David Rowley 8ee9c25087 Simplify partial path generation in GROUP BY/ORDER BY
Here we consolidate the generation of partial sort and partial incremental
sort paths in a similar way to what was done in 4a29eabd1.  Since the cost
penalty for incremental sort was removed by that commit, there's no
point in creating a sort path on the cheapest partial path if an
incremental sort could be done instead.

This has the added benefit of reducing the amount of code required to
build these paths.

Author: Richard Guo
Reviewed-by: Etsuro Fujita, Shubham Khanna, David Rowley
Discussion: https://postgr.es/m/CAMbWs49PaKxBZU9cN7k3DKB7id+YfGfOfS9H_Fo5tkqPMt=fDg@mail.gmail.com
2024-01-31 10:10:59 +13:00
Amit Kapila 776621a5e4 Add a failover option to subscriptions.
This commit introduces a new subscription option named 'failover', which
provides users with the ability to set the failover property of the
replication slot on the publisher when creating or altering a
subscription.

This uses the replication commands introduced by commit 7329240437 to
enable the failover option for a logical replication slot.

If the failover option is set to true, the associated replication slots
(i.e. the main slot and the table sync slots) in the upstream database are
enabled to be synchronized to the standbys. Note that the capability to
sync the replication slots will be added in subsequent commits.

Thanks to Masahiko Sawada for the design inputs.

Author: Shveta Malik, Hou Zhijie, Ajin Cherian
Reviewed-by: Peter Smith, Bertrand Drouvot, Dilip Kumar, Masahiko Sawada, Nisha Moond, Kuroda Hayato, Amit Kapila
Discussion: https://postgr.es/m/514f6f2f-6833-4539-39f1-96cd1e011f23@enterprisedb.com
2024-01-30 16:49:28 +05:30