Commit Graph

55521 Commits

Author SHA1 Message Date
Tom Lane e35bb9f158 Doc: use less-awkward phrasing.
Improve wording in note about tools required to build
from the source repository.

Laurenz Albe, per gripe from Riivo Kolka

Discussion: https://postgr.es/m/167463493588.2667301.13267758265445155872@wrigleys.postgresql.org
2023-01-26 11:34:17 -05:00
Robert Haas 14fb38626f DROP ROLE regress_role_limited_admin at end of test
This is required by project policy, and I overlooked the need for
it (again) by accident.

Reported by Álvaro Herrara.

Discussion: http://postgr.es/m/20230126114659.x3yuypot7p6zj73c@alvherre.pgsql
2023-01-26 08:14:41 -05:00
Peter Eisentraut 37e2673350 Don't install postmaster symlink anymore
This has long been deprecated.  Some of the build systems didn't even
install it.

Also remove man page.

Reviewed-by: Karl O. Pinc <kop@karlpinc.com>
Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-26 11:33:01 +01:00
Peter Eisentraut 81266442fb Remove gratuitous references to postmaster program
"postgres" has long been officially preferred over "postmaster" as the
name of the program to invoke to run the server.  Some example scripts
and code comments still used the latter.  Change those.

Discussion: https://www.postgresql.org/message-id/flat/ece84b69-8f94-8b88-925f-64207cb3a2f0@enterprisedb.com
2023-01-26 10:48:32 +01:00
Peter Geoghegan 6c6b497266 Revert "Add eager and lazy freezing strategies to VACUUM."
This reverts commit 4d41799261.  Broad
concerns about regressions caused by eager freezing strategy have been
raised.  Whether or not these concerns can be worked through in any time
frame is far from certain.

Discussion: https://postgr.es/m/20230126004347.gepcmyenk2csxrri@awork3.anarazel.de
2023-01-25 22:22:27 -08:00
Jeff Davis 8b5f36bb6c Clarify documentation for CLUSTER on partitioned tables.
Author: Nathan Bossart
Discussion: https://postgr.es/m/20230114224000.GA2505377@nathanxps13
2023-01-25 20:23:32 -08:00
Michael Paquier 9d2d9728b8 Make auto_explain print the query identifier in verbose mode
When auto_explain.log_verbose is on, auto_explain should print in the
logs plans equivalent to the EXPLAIN (VERBOSE).  However, when
compute_query_id is on, query identifiers were not showing up, being
only handled by EXPLAIN (VERBOSE).  This brings auto_explain on par with
EXPLAIN regarding that.  Note that like EXPLAIN, auto_explain does not
show the query identifier when compute_query_id=regress.

The change is done so as the choice of printing the query identifier is
done in ExplainPrintPlan() rather than in ExplainOnePlan(), to avoid a
duplication of the logic dealing with the query ID.  auto_explain is the
only in-core caller of ExplainPrintPlan().

While looking at the area, I have noticed that more consolidation
between EXPLAIN and auto_explain would be in order for the logging of
the plan duration and the buffer usage.  This refactoring is left as a
future change.

Author: Atsushi Torikoshi
Reviewed-by: Justin Pryzby, Julien Rouhaud
Discussion: https://postgr.es/m/1ea21936981f161bccfce05765c03bee@oss.nttdata.com
2023-01-26 12:23:16 +09:00
Thomas Munro ffcf6f4cfc Fix rare sharedtuplestore.c corruption.
If the final chunk of an oversized tuple being written out to disk was
exactly 32760 bytes, it would be corrupted due to a fencepost bug.

Bug #17619.  Back-patch to 11 where the code arrived.

While testing that (see test module in archives), I (tmunro) noticed
that the per-participant page counter was not initialized to zero as it
should have been; that wasn't a live bug when it was written since DSM
memory was originally always zeroed, but since 14
min_dynamic_shared_memory might be configured and it supplies non-zeroed
memory, so that is also fixed here.

Author: Dmitry Astapov <dastapov@gmail.com>
Discussion: https://postgr.es/m/17619-0de62ceda812b8b5%40postgresql.org
2023-01-26 14:52:19 +13:00
Michael Paquier 9aeff092c0 Revert "Rename contrib module basic_archive to basic_wal_module"
This reverts commit 0ad3c60, as per feedback from Tom Lane, Robert Haas
and Andres Freund.  The new name used for the module had little
support.

This moves back to basic_archive as module name, and we will likely use
that as template for recovery modules, as well.

Discussion: https://postgr.es/m/CA+TgmoYG5uGOp7DGFT5gzC1kKFWGjkLSj_wOQxGhfMcvVEiKGA@mail.gmail.com
2023-01-26 09:13:39 +09:00
Peter Geoghegan 1a8e72bff7 Doc: update VACUUM VERBOSE freezing tip.
VACUUM VERBOSE/autovacuuming logging have reported on the number of
pages frozen by VACUUM since commit d977ffd9 added that capability.
This information is directly related to relfrozenxid advancement, so
update an older tip from the documentation about how relfrozenxid is
reported on by the same instrumentation code.  Now the tip directly
mentions newly frozen pages, too.
2023-01-25 14:31:41 -08:00
Peter Geoghegan 4d41799261 Add eager and lazy freezing strategies to VACUUM.
Eager freezing strategy avoids large build-ups of all-visible pages.  It
makes VACUUM trigger page-level freezing whenever doing so will enable
the page to become all-frozen in the visibility map.  This is useful for
tables that experience continual growth, particularly strict append-only
tables such as pgbench's history table.  Eager freezing significantly
improves performance stability by spreading out the cost of freezing
over time, rather than doing most freezing during aggressive VACUUMs.
It complements the insert autovacuum mechanism added by commit b07642db.

VACUUM determines its freezing strategy based on the value of the new
vacuum_freeze_strategy_threshold GUC (or reloption) with logged tables.
Tables that exceed the size threshold use the eager freezing strategy.
Unlogged tables and temp tables always use eager freezing strategy,
since the added cost is negligible there.  Non-permanent relations won't
incur any extra overhead in WAL written (for the obvious reason), nor in
pages dirtied (since any extra freezing will only take place on pages
whose PD_ALL_VISIBLE bit needed to be set either way).

VACUUM uses lazy freezing strategy for logged tables that fall under the
GUC size threshold.  Page-level freezing triggers based on the criteria
established in commit 1de58df4, which added basic page-level freezing.

Eager freezing is strictly more aggressive than lazy freezing.  Settings
like vacuum_freeze_min_age still get applied in just the same way in
every VACUUM, independent of the strategy in use.  The only mechanical
difference between eager and lazy freezing strategies is that only the
former applies its own additional criteria to trigger freezing pages.
Note that even lazy freezing strategy will trigger freezing whenever a
page happens to have required that an FPI be written during pruning,
provided that the page will thereby become all-frozen in the visibility
map afterwards (due to the FPI optimization from commit 1de58df4).

The vacuum_freeze_strategy_threshold default setting is 4GB.  This is a
relatively low setting that prioritizes performance stability.  It will
be reviewed at the end of the Postgres 16 beta period.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Jeff Davis <pgsql@j-davis.com>
Reviewed-By: Andres Freund <andres@anarazel.de>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Discussion: https://postgr.es/m/CAH2-WzkFok_6EAHuK39GaW4FjEFQsY=3J0AAd6FXk93u-Xq3Fg@mail.gmail.com
2023-01-25 14:15:38 -08:00
Andres Freund 642e8821d7 plpython: Stop undefining _POSIX_C_SOURCE, _XOPEN_SOURCE
We undefined them to avoid warnings about macro redefinitions. But we haven't
fully followed the necessary include order, since at least 147c248254, in
2011.  Recently the combination of the include order rules not being followed
and undefining _POSIX_C_SOURCE started to cause a compile failure, starting
with 03023a2664. Undefining _POSIX_C_SOURCE hides clock_gettime(), which is
referenced in an inline function as of 03023a2664, whereas it was a macro
before.

After seeing some evidence that undefining _POSIX_C_SOURCE et al isn't
required, I tried to build postgres with plpython on most of our supported
platforms (except DragonFlyBSD and Illumos, but similar systems were tested),
with/without the #undefines. No compiler warning / behavioral difference.

The oldest supported python version, 3.2, defines _POSIX_C_SOURCE to 200112L
ad _XOPEN_SOURCE to 600, whereas newer versions of python use 200809L/700
respectively. As _POSIX_C_SOURCE/_XOPEN_SOURCE will default to the newer
operating system on most platforms, it's possible that when using python 3.2
new warnings would be emitted - but that seems acceptable.

It's possible that this approach won't work on some older platforms. But
getting rid of most of the include-order complexity seems promising, and it's
an easily revertible patch if we end up having to go another way.

Discussion: https://postgr.es/m/20230124165814.2njc7gnvubn2amh6@awork3.anarazel.de
2023-01-25 09:59:26 -08:00
Andres Freund 23c12329a7 plpython: Avoid the need to redefine *printf macros
Until now we undefined and then redefined a lot of *printf macros due to
worries about conflicts with Python.h macro definitions. Current Python.h
doesn't define any *printf macros, and older versions just defined snprintf,
vsnprintf, guarded by #if defined(MS_WIN32) && !defined(HAVE_SNPRINTF).

Thus we can replace the undefine/define section with a single
 #define HAVE_SNPRINTF 1

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20230124165814.2njc7gnvubn2amh6@awork3.anarazel.de
2023-01-25 09:59:26 -08:00
Tom Lane 3b4ac33254 Avoid type cheats for invalid dsa_handles and dshash_table_handles.
Invent separate macros for "invalid" values of these types, so that
we needn't embed knowledge of their representations into calling code.
These are all zeroes anyway ATM, so this is not fixing any live bug,
but it makes the code cleaner and more future-proof.

I (tgl) also chose to move DSM_HANDLE_INVALID into dsm_impl.h,
since it seems like it should live beside the typedef for dsm_handle.

Hou Zhijie, Nathan Bossart, Kyotaro Horiguchi, Tom Lane

Discussion: https://postgr.es/m/OS0PR01MB5716860B1454C34E5B179B6694C99@OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-01-25 11:48:38 -05:00
Michael Paquier d7c4830abb doc: Fix network_ops -> inet_ops in SpGiST operator class list
network_ops is an opclass family of SpGiST, and the opclass able to
work on the inet type is named inet_ops.

Oversight in 7a1cd52, that reworked the design of the table listing all
the operators available.

Reported-by: Laurence Parry
Reviewed-by: Tom Lane, David G. Johnston
Discussion: https://postgr.es/m/167458110639.2667300.14741268666497110766@wrigleys.postgresql.org
Backpatch-through: 14
2023-01-25 20:00:36 +09:00
Michael Paquier 0ad3c60caf Rename contrib module basic_archive to basic_wal_module
This rename is in preparation for the introduction of recovery modules,
where basic_wal_module will be used as a base template for the set of
callbacks introduced.  The former name did not really reflect all that.

Author: Nathan Bossart
Discussion: https://postgr.es/m/20221227192449.GA3672473@nathanxps13
2023-01-25 14:36:51 +09:00
Thomas Munro 239b175342 Process pending postmaster work before connections.
Modify the new event loop code from commit 7389aad6 so that it checks
for work requested by signal handlers even if it doesn't see a latch
event yet.

This gives priority to shutdown and reload requests where the latch will
be reported later in the event array, or in a later call to
WaitEventSetWait(), due to scheduling details.  In particular, this
guarantees that a SIGHUP-then-connect sequence (as seen in
authentication tests) causes the postmaster to process the reload before
accepting the connection.  If the WaitEventSetWait() call saw the socket
as ready, and the reload signal was generated before the connection,
then the latest time the signal handler should be able to run is after
poll/epoll_wait/kevent returns but before we check the
pending_pm_reload_request flag.

While here, also shift the handling of child exit below reload requests,
per Tom Lane's observation that that might start new processes, so we
should make sure we pick up new settings first.

This probably explains the one-off failure of build farm animal
malleefowl.

Reported-by: Hou Zhijie <houzj.fnst@fujitsu.com>
Reported-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/OS0PR01MB57163D3BF2AB42ECAA94E5C394C29%40OS0PR01MB5716.jpnprd01.prod.outlook.com
2023-01-25 15:00:05 +13:00
Peter Geoghegan 8f8f115932 Update more obsolete multixact.c comments.
Update some remaining comments in multixact.c that still described SLRU
truncation as happening in the checkpointer, rather than during VACUUM.

Follow-up to commit 5212d447.

Shi yu, with tweaks by me.

Author: Shi yu <shiy.fnst@fujitsu.com>
Discussion: https://postgr.es/m/OSZPR01MB631066BF246F8F74E83222FCFDC69@OSZPR01MB6310.jpnprd01.prod.outlook.com
2023-01-24 15:15:33 -08:00
Andrew Dunstan 1249371632 Improve exclude pattern file processing in pgindent
This makes two small changes that will improve pgindent's usefulness in
a git hook. First, it looks for the exclude file relative to the current
directory. And second, it applies the filters to filenames given on the
command line as well as those found in a directory sweep.

It might prove necessary to make further efforts to find the exclude
file, and even to allow multiple exclude files, but for now this should
be enough for most purposes.

Reviewed by Jelte Fennema
2023-01-24 16:09:09 -05:00
Robert Haas f1358ca52d Adjust interaction of CREATEROLE with role properties.
Previously, a CREATEROLE user without SUPERUSER could not alter
REPLICATION users in any way, and could not set the BYPASSRLS
attribute. However, they could manipulate the CREATEDB property
even if they themselves did not possess it.

With this change, a CREATEROLE user without SUPERUSER can set or
clear the REPLICATION, BYPASSRLS, or CREATEDB property on a new
role or a role that they have rights to manage if and only if
that property is set for their own role.

This implements the standard idea that you can't give permissions
you don't have (but you can give the ones you do have). We might
in the future want to provide more powerful ways to constrain
what a CREATEROLE user can do - for example, to limit whether
CONNECTION LIMIT can be set or the values to which it can be set -
but that is left as future work.

Patch by me, reviewed by Nathan Bossart, Tushar Ahuja, and Neha
Sharma.

Discussion: http://postgr.es/m/CA+TgmobX=LHg_J5aT=0pi9gJy=JdtrUVGAu0zhr-i5v5nNbJDg@mail.gmail.com
2023-01-24 10:57:09 -05:00
Amit Kapila 6c6d6ba3ee Fix the Drop Database hang.
The drop database command waits for the logical replication sync worker to
accept ProcSignalBarrier and the worker's slot creation waits for the drop
database to finish which leads to a deadlock. This happens because the
tablesync worker holds interrupts while creating a slot.

We prevent cancel/die interrupts while creating a slot in the table sync
worker because it is possible that before the server finishes this
command, a concurrent drop subscription happens which would complete
without removing this slot and that leads to the slot existing until the
end of walsender. However, the slot will eventually get dropped at the
walsender exit time, so there is no danger of the dangling slot.

This patch reallows cancel/die interrupts while creating a slot and
modifies the test to wait for slots to become zero to prevent finding an
ephemeral slot.

The reported hang doesn't happen in PG14 as the drop database starts to
wait for ProcSignalBarrier with PG15 (commits 4eb2176318 and e2f65f4255)
but it is good to backpatch this till PG14 as it is not a good idea to
prevent interrupts during a network call that could block indefinitely.

Reported-by: Lakshmi Narayanan Sreethar
Diagnosed-by: Andres Freund
Author: Hou Zhijie
Reviewed-by: Vignesh C, Amit Kapila
Backpatch-through: 14, where it was introduced in commit 6b67d72b60
Discussion: https://postgr.es/m/CA+kvmZELXQ4ZD3U=XCXuG3KvFgkuPoN1QrEj8c-rMRodrLOnsg@mail.gmail.com
2023-01-24 09:25:36 +05:30
Andres Freund 728f86fec6 libpqwalreceiver: Convert to libpq-be-fe-helpers.h
In contrast to the changes to dblink and postgres_fdw, this does not fix a
bug, as libpqwalreceiver did already process interrupts.

Besides reducing code duplication, the conversion leads to libpqwalreceiver
now using reserving file descriptors for libpq connections. While not strictly
required for the use in walreceiver, we are also using libpqwalreceiver for
logical replication, where it does seem more important.

Even if we eventually decide to backpatch the prior commits, there'd be no
need to backpatch this commit, due to not fixing an active bug.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2@awork3.anarazel.de
2023-01-23 19:25:23 -08:00
Andres Freund e4602483e9 dblink, postgres_fdw: Handle interrupts during connection establishment
Until now dblink and postgres_fdw did not process interrupts during connection
establishment. Besides preventing query cancellations etc, this can lead to
undetected deadlocks, as global barriers are not processed.

These aforementioned undetected deadlocks are the reason for the spate of CI
test failures in the FreeBSD 'test_running' step.

Fix the bug by using the helper from libpq-be-fe-helpers.h, introduced in a
prior commit. Besides fixing the bug, this also removes duplicated code
around reserving file descriptors.

As the change is relatively large and there are no field reports of the
problem, don't backpatch for now.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2@awork3.anarazel.de
Backpatch:
2023-01-23 19:25:23 -08:00
Andres Freund 28a591711d Add helper library for use of libpq inside the server environment
Currently dblink and postgres_fdw don't process interrupts during connection
establishment. Besides preventing query cancellations etc, this can lead to
undetected deadlocks, as global barriers are not processed.

Libpqwalreceiver in contrast, processes interrupts during connection
establishment. The required code is not trivial, so duplicating it into
additional places does not seem like a good option.

These aforementioned undetected deadlocks are the reason for the spate of CI
test failures in the FreeBSD 'test_running' step.

For now the helper library is just a header, as it needs to be linked into
each extension using libpq, and it seems too small to be worth adding a
dedicated static library for.

The conversion to the helper are done in subsequent commits.

Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/20220925232237.p6uskba2dw6fnwj2@awork3.anarazel.de
2023-01-23 19:25:23 -08:00
Andres Freund bc54ef4ec2 Fix error handling in libpqrcv_connect()
When libpqrcv_connect (also known as walrcv_connect()) failed, it leaked the
libpq connection. In most paths that's fairly harmless, as the calling process
will exit soon after. But e.g. CREATE SUBSCRIPTION could lead to a somewhat
longer lived leak.

Fix by releasing resources, including the libpq connection, on error.

Add a test exercising the error code path. To make it reliable and safe, the
test tries to connect to port=-1, which happens to fail during connection
establishment, rather than during connection string parsing.

Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20230121011237.q52apbvlarfv6jm6@awork3.anarazel.de
Backpatch: 11-
2023-01-23 18:27:42 -08:00
David Rowley 9567686ec8 Use OFFSET 0 instead of ORDER BY to stop subquery pullup
b762fed64 recently changed this test to prevent subquery pullup to allow
us to test Memoize with lateral_vars.  As pointed out by Tom Lane, OFFSET
0 is our standard way of preventing subquery pullups, so do it that way
instead.

Discussion: https://postgr.es/m/2144818.1674517061@sss.pgh.pa.us
Backpatch-through: 14, same as b762fed64
2023-01-24 13:49:10 +13:00
David Rowley b762fed648 Fix LATERAL join test in test memoize.sql
The test in question was meant to be testing Memoize to ensure it worked
correctly when the inner side of the join contained lateral vars, however,
nothing in the lateral subquery stopped it from being pulled up into the
main query, so the planner did that, and that meant no more lateral vars.

Here we add a simple ORDER BY to stop the planner from being able to
pullup the lateral subquery.

Author: Richard Guo
Discussion: https://postgr.es/m/CAMbWs4_LHJaN4L-tXpKMiPFnsCJWU1P8Xh59o0W7AA6UN99=cQ@mail.gmail.com
Backpatch-through: 14, where Memoize was added.
2023-01-24 12:30:30 +13:00
Peter Eisentraut 8dd43894b1 Fix XLogPageRead() comment
7fcbf6a and 2ff6555 changed the function signature of XLogPageRead()
but did not update the comment.

XLogReaderRoutine contains up to date information about the API, so no
need to repeat all that at XLogPageRead(), but fix the mentions of the
no longer existing function arguments.
2023-01-23 21:46:30 +01:00
Dean Rasheed 6dfacbf72b Add non-decimal integer support to type numeric.
This enhances the numeric type input function, adding support for
hexadecimal, octal, and binary integers of any size, up to the limits
of the numeric type.

Since 6fcda9aba8, such non-decimal integers have been accepted by the
parser as integer literals and passed through to numeric_in(). This
commit gives numeric_in() the ability to handle them.

While at it, simplify the handling of NaN and infinities, reducing the
number of calls to pg_strncasecmp(), and arrange for pg_strncasecmp()
to not be called at all for regular numbers. This gives a significant
performance improvement for decimal inputs, more than offsetting the
small performance hit of checking for non-decimal input.

Discussion: https://postgr.es/m/CAEZATCV8XShnmT9HZy25C%2Bo78CVOFmUN5EM9FRAZ5xvYTggPMg%40mail.gmail.com
2023-01-23 19:21:22 +00:00
Tom Lane 62e1e28bf7 Fix pgindent --show-diff option.
At least on my machine, the initial coding of this didn't actually
work, because interpolation of "$post_fh->filename" doesn't act
as intended.

I threw in some double quotes too, just in case anybody tries
to run this in a path containing spaces.
2023-01-23 13:50:49 -05:00
Tom Lane 3cece34be8 Remove special outfuncs/readfuncs handling of RangeVar.catalogname.
Historically we skipped writing/reading this field, but that no
longer works under WRITE_READ_PARSE_PLAN_TREES since we expanded
the coverage of that option to include utility commands (787102b56).
Remove the special case and just treat this field normally.

Bump catversion out of an abundance of caution --- I do not think
we currently ever store RangeVar nodes in the catalogs, but
perhaps I'm wrong.

Per report from Pavel Stehule.

Discussion: https://postgr.es/m/CAFj8pRAYvYu-qU7-NieqRRyaQZk-yr3UjtHQ2LR62PS9M1dZMA@mail.gmail.com
2023-01-23 13:33:19 -05:00
Andrew Dunstan a9dc7f9419 Add a test using ldapbindpasswd in pg_hba.conf
This feature has not been covered in tests up to now.

John Naylor and Andrew Dunstan

Discussion: https://postgr.es/m/06005bfb-0fd7-9d08-e0e5-440f277b73b4@dunslane.net
2023-01-23 08:40:18 -05:00
Andrew Dunstan ee4613d2b7 Restructure Ldap TAP test
The code for detecting the Ldap installation and setting up a test
server is broken out into a reusable module that can be used  for
additional tests to be added in later patches.

Discussion: https://postgr.es/m/06005bfb-0fd7-9d08-e0e5-440f277b73b4@dunslane.net
2023-01-23 08:31:37 -05:00
Andrew Dunstan b90f0b5747 Add non-destructive modes to pgindent
This adds two modes of running pgindent, neither of which results in
any changes being made to the source code. The --show-diff option shows
what changes would have been made, and the --silent-diff option just
exits with a status of 2 if any changes would be made. The second of
these is intended for scripting use in places such as git hooks.

Along the way some code cleanup is done, and a --help option is also
added.

Reviewed by Tom Lane

Discussion: https://postgr.es/m/c9c9fa6d-6de6-48c2-4f8b-0fbeef026439@dunslane.net
2023-01-23 07:09:51 -05:00
Dean Rasheed 0aa38db56b Optimise numeric division for 3 and 4 base-NBASE digit divisors.
On platforms with 128-bit integer support, introduce a new function
div_var_int64(), along the same lines as div_var_int() added in
d1b307eef2 for divisors with 1 or 2 base-NBASE digits, and use it to
speed up div_var() and div_var_fast() in a similar way when the
divisor has 3 or 4 base-NBASE digits.

This gives significant performance gains for divisors with 9-16
decimal digits.

Joel Jacobson.

Discussion:
  https://postgr.es/m/b7a5893d-af18-4c0b-8918-96de5f1bbf39%40app.fastmail.com
  https://postgr.es/m/CAEZATCXGm%3DDyTq%3DFrcOqC0gPMVveKUYTaD5KRRoajrUTiWxVMw%40mail.gmail.com
2023-01-23 11:58:28 +00:00
David Rowley 009dbdea02 Run pgindent on heapam.c
An upcoming patch by Melanie Plageman does some refactoring work in this
area.  Run pgindent on that file now before making any changes so that
it's easier to maintain/evolve each of the individual patches doing the
refactor work.  Additionally, add a few new required typedefs to the list
to make it easier to do future pgindent runs on this file during the
refactor work.

Discussion: https://postgr.es/m/CAAKRu_YSOnhKsDyFcqJsKtBSrd32DP-jjXmv7hL0BPD-z0TGXQ@mail.gmail.com
2023-01-23 23:08:38 +13:00
Heikki Linnakangas 236f1ea84c Fix and clarify function comment on LogicalTapeSetCreate.
Commit c4649cce39 removed the "shared" and "ntapes" arguments, but the
comment still talked about "shared". It also talked about "a shared
file handle", which was technically correct because even before commit
c4649cce39, the "shared file handle" referred to the "fileset"
argument, not "shared". But it was very confusing. Improve the
comment.

Also add a comment on what the "preallocate" argument does.

Backpatch to v15, just to make backpatching other patches easier in
the future.

Discussion: https://www.postgresql.org/message-id/af989685-91d5-aad4-8f60-1d066b5ec309@enterprisedb.com
Reviewed-by: Peter Eisentraut
2023-01-23 11:56:43 +02:00
David Rowley 67c5b8840f Harden new parallel string_agg/array_agg regression test
Per buildfarm member mandrill, it seems that
max_parallel_workers_per_gather may not always be set to the default value
of 2 when the new test added in 16fd03e95 is executed.  Here let's just
explicitly set that to 2 so that the planner never opts to use more than
that many parallel workers.
2023-01-23 21:31:46 +13:00
Michael Paquier c31cf1c03d pg_walinspect: Add pg_get_wal_fpi_info()
This function is able to extract the full page images from a range of
records, specified as of input arguments start_lsn and end_lsn.  Like
the other functions of this module, an error is returned if using LSNs
that do not reflect real system values.  All the FPIs stored in a single
record are extracted.

The module's version is bumped to 1.1.

Author: Bharath Rupireddy
Reviewed-by: Bertrand Drouvot
Discussion: https://postgr.es/m/CALj2ACVCcvzd7WiWvD=6_7NBvVB_r6G0EGSxL4F8vosAi6Se4g@mail.gmail.com
2023-01-23 13:55:18 +09:00
David Rowley 16fd03e956 Allow parallel aggregate on string_agg and array_agg
This adds combine, serial and deserial functions for the array_agg() and
string_agg() aggregate functions, thus allowing these aggregates to
partake in partial aggregations.  This allows both parallel aggregation to
take place when these aggregates are present and also allows additional
partition-wise aggregation plan shapes to include plans that require
additional aggregation once the partially aggregated results from the
partitions have been combined.

Author: David Rowley
Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane
Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 17:35:01 +13:00
Tom Lane 5a3a95385b Track logrep apply workers' last start times to avoid useless waits.
Enforce wal_retrieve_retry_interval on a per-subscription basis,
rather than globally, and arrange to skip that delay in case of
an intentional worker exit.  This probably makes little difference
in the field, where apply workers wouldn't be restarted often;
but it has a significant impact on the runtime of our logical
replication regression tests (even though those tests use
artificially-small wal_retrieve_retry_interval settings already).

Nathan Bossart, with mostly-cosmetic editorialization by me

Discussion: https://postgr.es/m/20221122004119.GA132961@nathanxps13
2023-01-22 14:08:46 -05:00
Tom Lane c9f7f92648 Allow REPLICA IDENTITY to be set on an index that's not (yet) valid.
The motivation for this change is that when pg_dump dumps a
partitioned index that's marked REPLICA IDENTITY, it generates a
command sequence that applies REPLICA IDENTITY before the partitioned
index has been marked valid, causing restore to fail.  We could
perhaps change pg_dump to not do it like that, but that would be
difficult and would not fix existing dump files with the problem.
There seems to be very little reason for the backend to disallow
this anyway --- the code ignores indisreplident when the index
isn't valid --- so instead let's fix it by allowing the case.

Commit 9511fb37a previously expressed a concern that allowing
indisreplident to be set on invalid indexes might allow us to
wind up in a situation where a table could have indisreplident
set on multiple indexes.  I'm not sure I follow that concern
exactly, but in any case the only way that could happen is because
relation_mark_replica_identity is too trusting about the existing set
of markings being valid.  Let's just rip out its early-exit code path
(which sure looks like premature optimization anyway; what are we
doing expending code to make redundant ALTER TABLE ... REPLICA
IDENTITY commands marginally faster and not-redundant ones marginally
slower?) and fix it to positively guarantee that no more than one
index is marked indisreplident.

The pg_dump failure can be demonstrated in all supported branches,
so back-patch all the way.  I chose to back-patch 9511fb37a as well,
just to keep indisreplident handling the same in all branches.

Per bug #17756 from Sergey Belyashov.

Discussion: https://postgr.es/m/17756-dd50e8e0c8dd4a40@postgresql.org
2023-01-21 13:10:29 -05:00
Noah Misch e52daaabf8 Reject CancelRequestPacket having unexpected length.
When the length was too short, the server read outside the allocation.
That yielded the same log noise as sending the correct length with
(backendPID,cancelAuthCode) matching nothing.  Change to a message about
the unexpected length.  Given the attacker's lack of control over the
memory layout and the general lack of diversity in memory layouts at the
code in question, we doubt a would-be attacker could cause a segfault.
Hence, while the report arrived via security@postgresql.org, this is not
a vulnerability.  Back-patch to v11 (all supported versions).

Andrey Borodin, reviewed by Tom Lane.  Reported by Andrey Borodin.
2023-01-21 06:08:00 -08:00
Andres Freund 03023a2664 instr_time: Represent time as an int64 on all platforms
Until now we used struct timespec for instr_time on all platforms but
windows. Using struct timespec causes a fair bit of memory (struct timeval is
16 bytes) and runtime overhead (much more complicated additions). Instead we
can convert the time to nanoseconds in INSTR_TIME_SET_CURRENT(), making the
remaining operations cheaper.

Representing time as int64 nanoseconds provides sufficient range, ~292 years
relative to a starting point (depending on clock source, relative to the unix
epoch or the system's boot time). That'd not be sufficient for calendar time
stored on disk, but is plenty for runtime interval time measurement.

On windows instr_time already is represented as cycles. It might make sense to
represent time as cycles on other platforms as well, as using cycle
acquisition instructions like rdtsc directly can reduce the overhead of time
acquisition substantially. This could be done in a fairly localized manner as
the code stands after this commit.

Because the windows and non-windows paths are now more similar, use a common
set of macros. To make that possible, most of the use of LARGE_INTEGER had to
be removed, which looks nicer anyway.

To avoid users of the API relying on the integer representation, we wrap the
64bit integer inside struct struct instr_time.

Author: Andres Freund <andres@anarazel.de>
Author: Lukas Fittl <lukas@fittl.com>
Author: David Geier <geidav.pg@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20230113195547.k4nlrmawpijqwlsa@awork3.anarazel.de
2023-01-20 21:16:47 -08:00
Andres Freund 25b2aba0c3 Zero initialize uses of instr_time about to trigger compiler warnings
These are all not necessary from a correctness POV. However, in the near
future instr_time will be simplified to an int64, at which point gcc would
otherwise start to warn about the changed places.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/20230116023639.rn36vf6ajqmfciua@awork3.anarazel.de
2023-01-20 21:16:47 -08:00
Michael Paquier 5d29d525ff Rework format of comments in headers for nodes
This is similar to 835d476, except that this one is to add node
attributes related to query jumbling and avoid long lines in the headers
and in the node structures changed by this commit.

Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/Y5BHOUhX3zTH/ig6@paquier.xyz
2023-01-21 12:17:02 +09:00
Michael Paquier 8eba3e3f02 Move queryjumble.c code to src/backend/nodes/
This will ease a follow-up move that will generate automatically this
code.  The C file is renamed, for consistency with the node-related
files whose code are generated by gen_node_support.pl:
- queryjumble.c -> queryjumblefuncs.c
- utils/queryjumble.h -> nodes/queryjumble.h

Per a suggestion from Peter Eisentraut.

Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/Y5BHOUhX3zTH/ig6@paquier.xyz
2023-01-21 11:48:37 +09:00
Robert Haas 557890920d Bump catversion for 6e2775e4d4.
It creates a new predefined role.
2023-01-20 16:37:26 -05:00
Robert Haas 6e2775e4d4 Add new GUC reserved_connections.
This provides a way to reserve connection slots for non-superusers.
The slots reserved via the new GUC are available only to users who
have the new predefined role pg_use_reserved_connections.
superuser_reserved_connections remains as a final reserve in case
reserved_connections has been exhausted.

Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me.

Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20 15:39:13 -05:00
Robert Haas fe00fec1f5 Rename ReservedBackends variable to SuperuserReservedConnections.
This is in preparation for adding a new reserved_connections GUC,
but aligning the GUC name with the variable name is also a good
idea on general principle.

Patch by Nathan Bossart. Reviewed by Tushar Ahuja and by me.

Discussion: http://postgr.es/m/20230119194601.GA4105788@nathanxps13
2023-01-20 15:32:08 -05:00