Commit Graph

55537 Commits

Author SHA1 Message Date
Amit Kapila c981d9145d Improve the code to decide and process the apply action.
The code that decides the apply action missed to handle non-transactional
messages and we didn't catch it in our testing as currently such messages
are simply ignored by the apply worker. This was introduced by changes in
commit 216a784829.

While testing this, I noticed that we forgot to reset stream_xid after
processing the stream stop message which could also result in the wrong
apply action after the fix for non-transactional messages.

In passing, change assert to elog for unexpected apply action in some of
the routines so as to catch the problems in the production environment, if
any.

Reported-by: Tomas Vondra
Author: Amit Kapila
Reviewed-by: Tomas Vondra, Sawada Masahiko, Hou Zhijie
Discussion: https://postgr.es/m/984ff689-adde-9977-affe-cd6029e850be@enterprisedb.com
Discussion: https://postgr.es/m/CAA4eK1+wyN6zpaHUkCLorEWNx75MG0xhMwcFhvjqm2KURZEAGw@mail.gmail.com
2023-01-17 11:28:22 +05:30
Amit Kapila 4f985aba69 Fix typo in comment.
Author: Osumi Takamichi
Discussion: https://postgr.es/m/TYCPR01MB83737EA140C79B7D099F65E8EDC69@TYCPR01MB8373.jpnprd01.prod.outlook.com
2023-01-17 10:39:11 +05:30
Michael Paquier 0b717432ff Track behavior of \1 in pg_ident.conf when quoted
Entries of pg-user in pg_ident.conf that are quoted and include '\1'
allow a replacement from a subexpression in a system user regexp.  This
commit adds a test to track this behavior and a note in the
documentation, as it could be affected by the use of an AuthToken for
the pg-user in the IdentLines parsed.

This subject has come up in the discussion aimed at extending the
support of pg-user in ident entries for more patterns.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQRNow4MwkBjgPxywXdJU_K3a9+Pm78JB7De3yQwwkTDew@mail.gmail.com
2023-01-17 13:41:09 +09:00
David Rowley da5800d5fa Don't presort ORDER BY/DISTINCT Aggrefs with volatile functions
In 1349d2790, we gave the planner the ability to provide ORDER BY/DISTINCT
Aggrefs with presorted input so that nodeAgg would not have to perform
sorts during execution.  That commit failed to properly consider the
implications of if the Aggref had a volatile function in its ORDER
BY/DISTINCT clause.  As it happened, this resulted in an ERROR about the
volatile function being missing from the targetlist.

Here, instead of adding the volatile function to the targetlist, we just
never consider an Aggref with a volatile function in its ORDER BY/DISTINCT
clause when choosing which Aggrefs we should sort by.  We do this as if we
were to choose a plan which provided these aggregates with presorted
input, then if there were many such aggregates which could all share the
same sort order, then it may be surprising if they all shared the same
sort sometimes and didn't at other times when some other set of aggregates
were given presorted results.  We can avoid this inconsistency by just
never providing these volatile function aggregates with presorted input.

Reported-by: Dean Rasheed
Discussion: https://postgr.es/m/CAEZATCWETioXs5kY8vT6BVguY41_wD962VDk=u_Nvd7S1UXzuQ@mail.gmail.com
2023-01-17 16:37:06 +13:00
Tom Lane 52585f8f07 Create common infrastructure for cross-version upgrade testing.
To test pg_upgrade across major PG versions, we have to be able to
modify or drop any old objects with no-longer-supported properties,
and we have to be able to deal with cosmetic changes in pg_dump output.
Up to now, the buildfarm and pg_upgrade's own test infrastructure had
separate implementations of the former, and we had nothing but very
ad-hoc rules for the latter (including an arbitrary threshold on how
many lines of unchecked diff were okay!).  This patch creates a Perl
module that can be shared by both those use-cases, and adds logic
that deals with pg_dump output diffs in a much more tightly defined
fashion.

This largely supersedes previous efforts in commits 0df9641d3,
9814ff550, and 62be9e4cd, which developed a SQL-script-based solution
for the task of dropping old objects.  There was nothing fundamentally
wrong with that work in itself, but it had no basis for solving the
output-formatting problem.  The most plausible way to deal with
formatting is to build a Perl module that can perform editing on the
dump files; and once we commit to that, it makes more sense for the
same module to also embed the knowledge of what has to be done for
dropping old objects.

Back-patch versions of the helper module as far as 9.2, to
support buildfarm animals that still test that far back.
It's also necessary to back-patch PostgreSQL/Version.pm,
because the new code depends on that.  I fixed up pg_upgrade's
002_pg_upgrade.pl in v15, but did not look into back-patching
it further than that.

Tom Lane and Andrew Dunstan

Discussion: https://postgr.es/m/891521.1673657296@sss.pgh.pa.us
2023-01-16 20:35:55 -05:00
Peter Geoghegan 980ae17310 Tighten up VACUUM's approach to setting VM bits.
Tighten up the way that visibilitymap_set() is called: request that both
the all-visible and all-frozen bits get set whenever the all-frozen bit
is set, regardless of what we think we know about the present state of
the all-visible bit.  Also make sure that the page level PD_ALL_VISIBLE
flag is set in the same code path.

In practice there doesn't seem to be a concrete scenario in which the
previous approach could lead to inconsistencies.  It was almost possible
in scenarios involving concurrent HOT updates from transactions that
abort, but (unlike pruning) freezing can never remove XIDs > VACUUM's
OldestXmin, even those from transactions that are known to have aborted.
That was protective here.

These issues have been around since commit a892234f83, which added the
all-frozen bit to the VM fork.  There is no known live bug here, so no
backpatch.

In passing, add some defensive assertions to catch the issue, and stop
reading the existing state of the VM when setting the VM in VACUUM's
final heap pass.  We already know that affected pages must have had at
least one LP_DEAD item before we set it LP_UNUSED, so there is no point
in reading the VM when it is set like this.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAH2-WznuNGSzF8v6OsgjaC5aYsb3cZ6HW6MLm30X0d65cmSH6A@mail.gmail.com
2023-01-16 09:34:37 -08:00
Robert Haas 6fa66ec88f Assorted improvements to SECURITY DEFINER functions documentation.
Add a cross-reference from the part of the page that introdues SECURITY
INVOKER and SECURITY DEFINER to the part of the page that talks about
writing SECURITY DEFINER functions safely, so that users are less likely
to miss it.

Remove discussion of the pre-8.3 behavior on the theory that it's
probably not very relevant any more, that release having gone out of
support nearly a decade ago.

Add a mention of the new createrole_self_grant GUC, which in
certain cases might need to be set to a safe value to avoid
unexpected consequences.

Possibly this section needs major surgery rather than just these
small tweaks, but hopefully this is at least a small step
forward.

Discussion: http://postgr.es/m/CA+Tgmoauqd1cHQjsNEoxL5O-kEO4iC9dAPyCudSvmNqPJGmy9g@mail.gmail.com
2023-01-16 10:49:59 -05:00
Robert Haas 3cdf7502f8 More documentation update for GRANT ... WITH SET OPTION.
Update the reference pages for various ALTER commands that
mentioned that you must be a member of role that will be the
new owner to instead say that you must be able to SET ROLE
to the new owner. Update ddl.sgml's generate statement on this
topic along similar lines.

Likewise, update CREATE SCHEMA and CREATE DATABASE, which
have options to specify who will own the new objects, to say
that you must be able to SET ROLE to the role that will own
them.

Finally, update the documentation for the GRANT statement
itself with some general principles about how the SET option
works and how it can be used.

Patch by me, reviewed (but not fully endorsed) by Noah Misch.

Discussion: http://postgr.es/m/CA+TgmoZk6VB3DQ83+DO5P_HP=M9PQAh1yj-KgeV30uKefVaWDg@mail.gmail.com
2023-01-16 10:35:29 -05:00
Peter Eisentraut 20428d344a Add BufFileRead variants with short read and EOF detection
Most callers of BufFileRead() want to check whether they read the full
specified length.  Checking this at every call site is very tedious.
This patch provides additional variants BufFileReadExact() and
BufFileReadMaybeEOF() that include the length checks.

I considered changing BufFileRead() itself, but this function is also
used in extensions, and so changing the behavior like this would
create a lot of problems there.  The new names are analogous to the
existing LogicalTapeReadExact().

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/f3501945-c591-8cc3-5ef0-b72a2e0eaa9c@enterprisedb.com
2023-01-16 11:01:31 +01:00
Peter Eisentraut 1561612e3b Fix some BufFileRead() error reporting
Remove "%m" from error messages where errno would be bogus.  Add short
read byte counts where appropriate.

This is equivalent to what was done in
7897e3bb90, but some code was apparently
developed concurrently to that and not updated accordingly.

Reviewed-by: Amit Kapila <amit.kapila16@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/f3501945-c591-8cc3-5ef0-b72a2e0eaa9c@enterprisedb.com
2023-01-16 09:44:04 +01:00
Michael Paquier 9a740f81eb Refactor code in charge of running shell-based recovery commands
The code specific to the execution of archive_cleanup_command,
recovery_end_command and restore_command is moved to a new file named
shell_restore.c.  The code is split into three functions:
- shell_restore(), that attempts the execution of a shell-based
restore_command.
- shell_archive_cleanup(), for archive_cleanup_command.
- shell_recovery_end(), for recovery_end_command.

This introduces no functional changes, with failure patterns and logs
generated in consequence being the same as before (one case actually
generates one less DEBUG2 message "could not restore" when a restore
command succeeds but the follow-up stat() to check the size fails, but
that only matters with a elevel high enough).

This is preparatory work for allowing recovery modules, a facility
similar to archive modules, with callbacks shaped similarly to the
functions introduced here.

Author: Nathan Bossart
Reviewed-by: Andres Freund, Michael Paquier
Discussion: https://postgr.es/m/20221227192449.GA3672473@nathanxps13
2023-01-16 16:31:43 +09:00
Michael Paquier 02d3448f4f Store IdentLine->pg_user as an AuthToken
While system_user was stored as an AuthToken in IdentLine, pg_user was
stored as a plain string.  This commit changes the code as we start
storing pg_user as an AuthToken too.

This does not have any functional changes, as all the operations on
pg_user only use the string from the AuthToken.  There is no regexp
compiled and no check based on its quoting, yet.  This is in preparation
of more features that intend to extend its capabilities, like support
for regexps and group membership.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQRNow4MwkBjgPxywXdJU_K3a9+Pm78JB7De3yQwwkTDew@mail.gmail.com
2023-01-16 13:58:07 +09:00
Tom Lane 647fa50054 Remove arbitrary FUNC_MAX_ARGS limit in int2vectorin and oidvectorin.
int2vectorin limited the number of array elements it'd take to
FUNC_MAX_ARGS, which is probably fine for the traditional use-cases.
But now that pg_publication_rel.prattrs is an int2vector, it's not
fine at all: it's easy to construct cases where that can have up to
about MaxTupleAttributeNumber entries.  Trying to replicate such
tables leads to logical-replication failures.

As long as we have to touch this code anyway, let's just remove
the a-priori limit altogether, and let it accept any size that'll
be allowed by repalloc.  (Note that since int2vector isn't toastable,
we cannot store arrays longer than about BLCKSZ/2; but there is no
good excuse for letting int2vectorin depend on that.  Perhaps we
will lift the no-toast restriction someday.)

While at it, also improve the equivalent logic in oidvectorin.
I don't know of any practical use-case for long oidvectors right
now, but doing it right actually makes the code shorter.

Per report from Erik Rijkers.  Back-patch to v15 where
pg_publication_rel.prattrs was added.

Discussion: https://postgr.es/m/668ba539-33c5-8190-ca11-def2913cb94b@xs4all.nl
2023-01-15 17:32:09 -05:00
Tom Lane 3f244d020f Make new GENERATED-expressions code more bulletproof.
In commit 8bf6ec3ba I assumed that no code path could reach
ExecGetExtraUpdatedCols without having gone through
ExecInitStoredGenerated.  That turns out not to be the case in
logical replication: if there's an ON UPDATE trigger on the target
table, trigger.c will call this code before anybody has set up its
generated columns.  Having seen that, I don't have a lot of faith in
there not being other such paths.  ExecGetExtraUpdatedCols can call
ExecInitStoredGenerated for itself, as long as we are willing to
assume that it is only called in CMD_UPDATE operations, which on
the whole seems like a safer leap of faith.

Per report from Vitaly Davydov.

Discussion: https://postgr.es/m/d259d69652b8c2ff50e14cda3c236c7f@postgrespro.ru
2023-01-15 13:14:52 -05:00
Tatsuo Ishii 1334b79a35 Doc: fix typo in backup.sgml.
<varname>archive_command</varname> was unnecessarily repeated.

Author: Tatsuo Ishii
Reviewed-by: Amit Kapila
Backpatch-through: 15
Discussion: https://postgr.es/m/flat/20230114.110234.666053507266410467.t-ishii%40sranhm.sra.co.jp
2023-01-14 18:16:09 +09:00
Jeff Davis ff9618e82a Fix MAINTAIN privileges for toast tables and partitions.
Commit 60684dd8 left loose ends when it came to maintaining toast
tables or partitions.

For toast tables, simply skip the privilege check if the toast table
is an indirect target of the maintenance command, because the main
table privileges have already been checked.

For partitions, allow the maintenance command if the user has the
MAINTAIN privilege on the partition or any parent.

Also make CLUSTER emit "skipping" messages when the user doesn't have
privileges, similar to VACUUM.

Author: Nathan Bossart
Reported-by: Pavel Luzanov
Reviewed-by: Pavel Luzanov, Ted Yu
Discussion: https://postgr.es/m/20230113231339.GA2422750@nathanxps13
2023-01-14 00:16:23 -08:00
Andres Freund ff23b592ad Add 250c8ee07e to git-blame-ignore-revs 2023-01-13 15:33:49 -08:00
Andres Freund 250c8ee07e Manual cleanup and pgindent of pgstat and bufmgr related code
This is in preparation for commiting a larger patch series in the area.

Discussion: https://postgr.es/m/CAAKRu_bHwGEbzNxxy+MQDkrsgog6aO6iUvajJ4d6PD98gFU7+w@mail.gmail.com
2023-01-13 15:23:17 -08:00
Jeff Davis d46a9792a8 Clean up useless "skipping" messages for VACUUM/ANALYZE.
When VACUUM/ANALYZE are run on an entire database, it warns of
skipping relations for which the user doesn't have sufficient
privileges. That only makes sense for tables, so skip such messages
for indexes, etc.

Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/c0a85c2e83158560314b576b6241c8ed0aea1745.camel%40j-davis.com
2023-01-13 14:42:03 -08:00
Jeff Davis c44f6334ca Simplify permissions for LOCK TABLE.
The prior behavior was confusing and hard to document. For instance,
if you had UPDATE privileges, you could lock a table in any lock mode
except ACCESS SHARE mode.

Now, if granted a privilege to lock at a given mode, one also has
privileges to lock at a less-conflicting mode. MAINTAIN, UPDATE,
DELETE, and TRUNCATE privileges allow any lock mode. INSERT privileges
allow ROW EXCLUSIVE (or below). SELECT privileges allow ACCESS SHARE.

Reviewed-by: Nathan Bossart
Discussion: https://postgr.es/m/9550c76535404a83156252b25a11babb4792ea1e.camel%40j-davis.com
2023-01-13 14:33:19 -08:00
Amit Kapila b7ae039536 Ignore dropped and generated columns from the column list.
We don't allow different column lists for the same table in the different
publications of the single subscription. A publication with a column list
except for dropped and generated columns should be considered the same as
a publication with no column list (which implicitly includes all columns
as part of the columns list). However, as we were not excluding the
dropped and generated columns from the column list combining such
publications leads to an error "cannot use different column lists for
table ...".

We decided not to backpatch this fix as there is a risk of users seeing
this as a behavior change and also we didn't see any field report of this
case.

Author: Shi yu
Reviewed-by: Amit Kapila
Discussion: https://postgr.es/m/OSZPR01MB631091CCBC56F195B1B9ACB0FDFE9@OSZPR01MB6310.jpnprd01.prod.outlook.com
2023-01-13 14:49:23 +05:30
Amit Kapila dca8b01f5f Avoid creating parallel apply state hash table unless required.
This hash table is used to cache the state of streaming transactions being
applied by the parallel apply workers. So, this should be created only
when we are successful in launching at least one worker. This avoids rare
case memory leak when we are never able to launch any worker.

Author: Ted Yu
Discussion: https://postgr.es/m/CALte62wg0rBR3Vj2beV=HiWo2qG9L0hzKcX=yULNER0wmf4aEw@mail.gmail.com
2023-01-13 08:28:05 +05:30
Michael Paquier e753ae6397 Add tests for regex replacement with \1 in pg_ident.conf to 0003_peer.pl
Regexp replacement with \1 in pg_ident.conf is tested in one check of
the kerberos test suite, still it requires a dependency on
--with-gssapi to be triggered.  This commit adds to the test suite of
peer authentication two tests to check the replacement of \1 in a
pg-username, coupled with a system-username regexp:
- With a subexpression in system-username, similarly to the kerberos
test suite.
- Without a subexpression in system-username, checking for a failure.
This had no coverage until now, and the error pattern is checked in the
server logs.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQRNow4MwkBjgPxywXdJU_K3a9+Pm78JB7De3yQwwkTDew@mail.gmail.com
2023-01-13 10:35:28 +09:00
Michael Paquier 58597edea1 doc: Simplify description of functions for pg_walinspect
As introduced in 2258e76, the docs were hard to parse:
- The examples used listed a lot of long records, bloating the output.
These are switched to show less records with the expanded format,
similarly to pageinspect.
- The function descriptions listed all the OUT parameters, producing
long lines.  This is updated so as only the input parameters are
documented, clarifying the whole.
- Remove one example on pg_get_wal_stats() when per_record is set to
true, which is not really necessary once we know the output produced,
and the behavior of the parameter is documented.

While on it, fix a few grammar mistakes and simplify a couple of
sentences.

Author: Bharath Rupireddy
Discussion: https://postgr.es/m/CALj2ACVGcUpziGgQrcT-1G3dHWQQfWjYBu1YQ2ypv9y86dgogg@mail.gmail.com
Backpatch-through: 15
2023-01-13 09:29:44 +09:00
Thomas Munro f1821b58fc Fix WaitEventSetWait() buffer overrun.
The WAIT_USE_EPOLL and WAIT_USE_KQUEUE implementations of
WaitEventSetWaitBlock() confused the size of their internal buffer with
the size of the caller's output buffer, and could ask the kernel for too
many events.  In fact the set of events retrieved from the kernel needs
to be able to fit in both buffers, so take the smaller of the two.

The WAIT_USE_POLL and WAIT_USE WIN32 implementations didn't have this
confusion.

This probably didn't come up before because we always used the same
number in both places, but commit 7389aad6 calculates a dynamic size at
construction time, while using MAXLISTEN for its output event buffer on
the stack.  That seems like a reasonable thing to want to do, so
consider this to be a pre-existing bug worth fixing.

As discovered by valgrind on skink.

Back-patch to all supported releases for epoll, and to release 13 for
the kqueue part, which copied the incorrect epoll code.

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/901504.1673504836%40sss.pgh.pa.us
2023-01-13 11:02:12 +13:00
Alexander Korotkov 3161ae86ce Fix jsonpath existense checking of missing variables
The current jsonpath code assumes that the referenced variable always exists.
It could only throw an error at the value valuation time.  At the same time
existence checking assumes variable is present without valuation, and error
suppression doesn't work for missing variables.

This commit makes existense checking trigger an error for missing variables.
This makes the overall behavior consistent.

Backpatch to 12 where jsonpath was introduced.

Reported-by: David G. Johnston
Discussion: https://postgr.es/m/CAKFQuwbeytffJkVnEqDyLZ%3DrQsznoTh1OgDoOF3VmOMkxcTMjA%40mail.gmail.com
Author: Alexander Korotkov, David G. Johnston
Backpatch-through: 12
2023-01-12 18:16:34 +03:00
Peter Eisentraut c8ad4d8166 Constify the arguments of ilist.c/h functions
Const qualifiers ensure that we don't do something stupid in the
function implementation.  Additionally they clarify the interface.  As
an example:

    void
    slist_delete(slist_head *head, const slist_node *node)

Here one can instantly tell that node->next is not going to be set to
NULL.  Finally, const qualifiers potentially allow the compiler to do
more optimizations.  This being said, no benchmarking was done for
this patch.

The functions that return non-const pointers like slist_next_node(),
dclist_next_node() etc. are not affected by the patch intentionally.

Author: Aleksander Alekseev
Reviewed-by: Andres Freund
Discussion: https://postgr.es/m/CAJ7c6TM2%3D08mNKD9aJg8vEY9hd%2BG4L7%2BNvh30UiNT3kShgRgNg%40mail.gmail.com
2023-01-12 08:00:51 +01:00
Peter Eisentraut 881fa869c6 Code cleanup
for commit c96de2ce17

Author: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/20230111185434.GA1912982@nathanxps13
2023-01-12 07:37:39 +01:00
Michael Paquier 8607630d74 Rename some variables related to ident files in hba.{c,h}
The code that handles authentication for user maps was pretty confusing
with its choice of variable names.  It involves two types of users: a
system user and a Postgres user (well, role), and these were not named
consistently throughout the code that processes the user maps loaded
from pg_ident.conf at authentication.

This commit changes the following things to improve the situation:
- Rename "pg_role" to "pg_user" and "token" to "system_user" in
IndetLine.  These choices are more consistent with the pg_ident.conf
example in the docs, as well.  "token" has been introduced recently in
fc579e1, and it is way worse than the choice before that, "ident_user".
- Switch the order of the fields in IdentLine to map with the order of
the items in the ident files, as of map name, system user and PG user.
- In check_ident_usermap(), rename "regexp_pgrole" to "expanded_pg_user"
when processing a regexp for the system user entry in a user map.  This
variable does not store a regular expression at all: it would be either
a string or a substitution to \1 if the Postgres role is specified as
such.

Author: Jelte Fennema
Discussion: https://postgr.es/m/CAGECzQTkwELHUOAKhvdA+m3tWbUQySHHkExJV8GAZ1pwgbEgXg@mail.gmail.com
2023-01-12 14:23:20 +09:00
Michael Paquier bfd2542001 Fix incorrect comment in hba.h
A comment in hba.h mentioned that AuthTokens are used when building the
IdentLines from pg_ident.conf, but since 8fea868 that has added support
of regexps for databases and roles in pg_hba.conf, it is also the case
of HBA files.  This refreshes the comment to refer to both HBA and ident
files.

Issue spotted while going through a different patch.
2023-01-12 13:49:28 +09:00
Michael Paquier 5ad165d2c0 Acquire spinlock when updating 2PC slot data during logical decoding creation
The creation of a logical decoding context in CreateDecodingContext()
updates some data of its slot for two-phase transactions if enabled by
the caller, but the code forgot to acquire a spinlock when updating
these fields like any other code paths.  This could lead to the read of
inconsistent data.

Oversight in a8fd13c.

Author: Sawada Masahiko
Discussion: https://postgr.es/m/CAD21AoAD8_fp47191LKuecjDd3DYhoQ4TaucFco1_TEr_jQ-Zw@mail.gmail.com
Backpatch-through: 15
2023-01-12 13:40:33 +09:00
Tom Lane f0e6d6d3c9 Revert "Get rid of the "new" and "old" entries in a view's rangetable."
This reverts commit 1b4d280ea1.
It's broken the buildfarm members that run cross-version-upgrade tests,
because they're not prepared to deal with cosmetic differences between
CREATE VIEW commands emitted by older servers and HEAD.  Even if we had
a solution to that, which we don't, it'd take some time to roll it out
to the affected animals.  This improvement isn't valuable enough to
justify addressing that problem on an emergency basis, so revert it
for now.
2023-01-11 23:01:22 -05:00
Thomas Munro 5a26c7b310 Refactor DetermineSleepTime() to use milliseconds.
Since we're not using select() anymore, we don't need to bother with
struct timeval.  We can work directly in milliseconds, which the latch
API wants.

Discussion: https://postgr.es/m/CA%2BhUKG%2BZ-HpOj1JsO9eWUP%2Bar7npSVinsC_npxSy%2BjdOMsx%3DGg%40mail.gmail.com
2023-01-12 16:32:30 +13:00
Thomas Munro 7389aad636 Use WaitEventSet API for postmaster's event loop.
Switch to a design similar to regular backends, instead of the previous
arrangement where signal handlers did non-trivial state management and
called fork().  The main changes are:

* The postmaster now has its own local latch to wait on.  (For now, we
  don't want other backends setting its latch directly, but that could
  probably be made to work with more research on robustness.)

* The existing signal handlers are cut in two: a handle_pm_XXX() part
  that just sets pending_pm_XXX flags and the latch, and a
  process_pm_XXX() part that runs later when the latch is seen.

* Signal handlers are now installed with the regular pqsignal()
  function rather than the special pqsignal_pm() function; historical
  portability concerns about the effect of SA_RESTART on select() are no
  longer relevant, and we don't need to block signals anymore.

Reviewed-by: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CA%2BhUKG%2BZ-HpOj1JsO9eWUP%2Bar7npSVinsC_npxSy%2BjdOMsx%3DGg%40mail.gmail.com
2023-01-12 16:32:20 +13:00
Tom Lane d93d68aeea Doc: fix silly thinko in 8bf6ec3ba.
Amit Langote

Discussion: https://postgr.es/m/CA+HiwqG2v-SnWyJuyVM-Z8DEFukY8+qe3XLMwSG4Xp7Yf=RioA@mail.gmail.com
2023-01-11 22:19:55 -05:00
Peter Geoghegan d30b499997 Make lazy_vacuum_heap_rel match lazy_scan_heap.
Make lazy_vacuum_heap_rel variable names match those from lazy_scan_heap
where that makes sense.

Extracted from a larger patch to deal with issues with how vacuumlazy.c
sets pages all-frozen.

Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WznuNGSzF8v6OsgjaC5aYsb3cZ6HW6MLm30X0d65cmSH6A@mail.gmail.com
2023-01-11 18:45:32 -08:00
Peter Geoghegan 8c233cf86b vacuumlazy.c: Tweak local variable name.
Make a local variable name consistent with the name from its WAL record.

Extracted from a larger patch to deal with issues with how vacuumlazy.c
sets pages all-frozen.

Author: Peter Geoghegan <pg@bowt.ie>
Discussion: https://postgr.es/m/CAH2-WznuNGSzF8v6OsgjaC5aYsb3cZ6HW6MLm30X0d65cmSH6A@mail.gmail.com
2023-01-11 17:57:18 -08:00
Peter Geoghegan 50767705ed Rename and relocate freeze plan dedup routines.
Rename the heapam.c freeze plan deduplication routines added by commit
9e540599 to names that follow conventions for functions in heapam.c.
Also relocate the functions so that they're next to their caller, which
runs during original execution, when FREEZE_PAGE WAL records are built.

The routines were initially placed next to (and followed the naming
conventions of) conceptually related REDO routine code, but that scheme
turned out to be kind of jarring when considered in a wider context.

Author: Peter Geoghegan <pg@bowt.ie>
Reported-By: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20230109214308.icz26oqvt3k2274c@awork3.anarazel.de
2023-01-11 17:30:42 -08:00
Tom Lane 1b4d280ea1 Get rid of the "new" and "old" entries in a view's rangetable.
The rule system needs "old" and/or "new" pseudo-RTEs in rule actions
that are ON INSERT/UPDATE/DELETE.  Historically it's put such entries
into the ON SELECT rules of views as well, but those are really quite
vestigial.  The only thing we've used them for is to carry the
view's relid forward to AcquireExecutorLocks (so that we can
re-lock the view to verify it hasn't changed before re-using a plan)
and to carry its relid and permissions data forward to execution-time
permissions checks.  What we can do instead of that is to retain
these fields of the RTE_RELATION RTE for the view even after we
convert it to an RTE_SUBQUERY RTE.  This requires a tiny amount of
extra complication in the planner and AcquireExecutorLocks, but on
the other hand we can get rid of the logic that moves that data from
one place to another.

The principal immediate benefit of doing this, aside from a small
saving in the pg_rewrite data for views, is that these pseudo-RTEs
no longer trigger ruleutils.c's heuristic about qualifying variable
names when the rangetable's length is more than 1.  That results
in quite a number of small simplifications in regression test outputs,
which are all to the good IMO.

Bump catversion because we need to dump a few more fields of
RTE_SUBQUERY RTEs.  While those will always be zeroes anyway in
stored rules (because we'd never populate them until query rewrite)
they are useful for debugging, and it seems like we'd better make
sure to transmit such RTEs accurately in plans sent to parallel
workers.  I don't think the executor actually examines these fields
after startup, but someday it might.

Amit Langote

Discussion: https://postgr.es/m/CA+HiwqEf7gPN4Hn+LoZ4tP2q_Qt7n3vw7-6fJKOf92tSEnX6Gg@mail.gmail.com
2023-01-11 19:41:09 -05:00
Michael Paquier 2ff5ca86e8 Add support for tab completion after ALTER EXTENSION ADD|DROP in psql
This appends the set of object types supported by these commands, and
the objects defined in the cluster are completed after that.  Note that
these may not be in the extension being working on when using DROP, to
keep the code simple, but this is much more useful than the previous
behavior of not knowing the objects that can be touched.

Author: Vignesh C
Discussion: https://postgr.es/m/CALDaNm3LVM2QcUWqgOonKZH80TveT-tUthbw4ZhuE_6pD3yi-A@mail.gmail.com
2023-01-12 08:49:37 +09:00
Peter Geoghegan af3855cb77 Improve TransactionIdDidAbort() documentation.
Document that TransactionIdDidAbort() won't indicate that transactions
that were in-progress during a crash have aborted.  Tie this to existing
discussion of the TransactionIdDidCommit() and TransactionIdDidCommit()
protocol that code in heapam_visibility.c (and a few other places) must
observe.

Follow-up to bugfix commit eb5ad4ff.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/CAH2-Wzn4bEEqgmaUQL3aJ73yM9gAeK-wE4ngi7kjRjLztb+P0w@mail.gmail.com
2023-01-11 15:31:42 -08:00
Tom Lane 8bf6ec3ba3 Improve handling of inherited GENERATED expressions.
In both partitioning and traditional inheritance, require child
columns to be GENERATED if and only if their parent(s) are.
Formerly we allowed the case of an inherited column being
GENERATED when its parent isn't, but that results in inconsistent
behavior: the column can be directly updated through an UPDATE
on the parent table, leading to it containing a user-supplied
value that might not match the generation expression.  This also
fixes an oversight that we enforced partition-key-columns-can't-
be-GENERATED against parent tables, but not against child tables
that were dynamically attached to them.

Also, remove the restriction that the child's generation expression
be equivalent to the parent's.  In the wake of commit 3f7836ff6,
there doesn't seem to be any reason that we need that restriction,
since generation expressions are always computed per-table anyway.
By removing this, we can also allow a child to merge multiple
inheritance parents with inconsistent generation expressions, by
overriding them with its own expression, much as we've long allowed
for DEFAULT expressions.

Since we're rejecting a case that we used to accept, this doesn't
seem like a back-patchable change.  Given the lack of field
complaints about the inconsistent behavior, it's likely that no
one is doing this anyway, but we won't change it in minor releases.

Amit Langote and Tom Lane

Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-11 15:55:02 -05:00
Tom Lane d0d9683287 Don't leave roles behind after core regression tests.
Commits cf5eb37c5 and e5b8a4c09 each created a new role that they
forgot to remove again.  This breaks the use-case of running "make
installcheck" more than once, and it's also against project policy
because it'd be quite unfriendly behavior if one were running
"make installcheck" against a non-throwaway installation.
2023-01-11 12:22:47 -05:00
Peter Eisentraut c971a5b27a Fix MSVC build
for commit c96de2ce17
2023-01-11 14:41:52 +01:00
Peter Eisentraut c96de2ce17 Common function for percent placeholder replacement
There are a number of places where a shell command is constructed with
percent-placeholders (like %x).  It's cumbersome to have to open-code
this several times.  This factors out this logic into a separate
function.  This also allows us to ensure consistency for and document
some subtle behaviors, such as what to do with unrecognized
placeholders.

The unified handling is now that incorrect and unknown placeholders
are an error, where previously in most cases they were skipped or
ignored.  This affects the following settings:

- archive_cleanup_command
- archive_command
- recovery_end_command
- restore_command
- ssl_passphrase_command

The following settings are part of this refactoring but already had
stricter error handling and should be unchanged in their behavior:

- basebackup_to_shell.command

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5238bbed-0b01-83a6-d4b2-7eb0562a054e%40enterprisedb.com
2023-01-11 10:42:35 +01:00
Michael Paquier 5f6401f81c Fix typos in code and comments
Author: Justin Pryzby
Discussion: https://postgr.es/m/20230110045722.GD9837@telsasoft.com
2023-01-11 15:16:38 +09:00
Michael Paquier 69fb29d1af Remove function declarations from headers for some undefined functions
The functions whose declarations are removed here have been removed in
the past, but their respective headers forgot the call.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20230110045722.GD9837@telsasoft.com
2023-01-11 11:54:55 +09:00
David Rowley 3c6fc58209 Have the planner consider Incremental Sort for DISTINCT
Prior to this, we only considered a full sort on the cheapest input path
and uniquifying any path which was already sorted in the required sort
order.  Here we adjust create_final_distinct_paths() so that it also
adds an Incremental Sort path on any path which has presorted keys.

Additionally, this adjusts the parallel distinct code so that we now
consider sorting the cheapest partial path and incrementally sorting any
partial paths with presorted keys.  Previously we didn't consider any
sorting for parallel distinct and only added a unique path atop any path
which had the required pathkeys already.

Author: David Rowley
Reviewed-by: Richard Guo
Discussion: https://postgr.es/m/CAApHDvo8Lz2H=42urBbfP65LTcEUOh288MT7DsG2_EWtW1AXHQ@mail.gmail.com
2023-01-11 10:25:43 +13:00
Robert Haas e5b8a4c098 Add new GUC createrole_self_grant.
Can be set to the empty string, or to either or both of "set" or
"inherit". If set to a non-empty value, a non-superuser who creates
a role (necessarily by relying up the CREATEROLE privilege) will
grant that role back to themselves with the specified options.

This isn't a security feature, because the grant that this feature
triggers can also be performed explicitly. Instead, it's a user experience
feature. A superuser would necessarily inherit the privileges of any
created role and be able to access all such roles via SET ROLE;
with this patch, you can configure createrole_self_grant = 'set, inherit'
to provide a similar experience for a user who has CREATEROLE but not
SUPERUSER.

Discussion: https://postgr.es/m/CA+TgmobN59ct+Emmz6ig1Nua2Q-_o=r6DSD98KfU53kctq_kQw@mail.gmail.com
2023-01-10 12:44:49 -05:00
Robert Haas cf5eb37c5e Restrict the privileges of CREATEROLE users.
Previously, CREATEROLE users were permitted to make nearly arbitrary
changes to roles that they didn't create, with certain exceptions,
particularly superuser roles.  Instead, allow CREATEROLE users to make such
changes to roles for which they possess ADMIN OPTION, and to
grant membership only in roles for which they possess ADMIN OPTION.

When a CREATEROLE user who is not a superuser creates a role, grant
ADMIN OPTION on the newly-created role to the creator, so that they
can administer roles they create or for which they have been given
privileges.

With these changes, CREATEROLE users still have very significant
powers that unprivileged users do not receive: they can alter, rename,
drop, comment on, change the password for, and change security labels
on roles.  However, they can now do these things only for roles for
which they possess appropriate privileges, rather than all
non-superuser roles; moreover, they cannot grant a role such as
pg_execute_server_program unless they themselves possess it.

Patch by me, reviewed by Mark Dilger.

Discussion: https://postgr.es/m/CA+TgmobN59ct+Emmz6ig1Nua2Q-_o=r6DSD98KfU53kctq_kQw@mail.gmail.com
2023-01-10 12:44:30 -05:00