Commit Graph

54191 Commits

Author SHA1 Message Date
Thomas Munro 634a89c708 Remove configure probe for wctype.h.
This header is present in SUSv2 and Windows.

Also remove the inclusion of <wchar.h>, following clues that it was only
included for the benefit of historical systems that didn't have
<wctype.h>.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKGKAmTgbg_hMiGG5T7pkpzOnY1cWFAHYtZXHCpqeC_hCkA%40mail.gmail.com
2022-07-23 16:54:00 +12:00
Thomas Munro a3b8d2a997 Remove configure probe for sys/tas.h.
The last reference to HAVE_SYS_TAS_H disappeared with commit 718aa43a.
2022-07-23 14:59:11 +12:00
Alvaro Herrera 0f39b70a6f
Fix [install]check in interfaces/libpq/Makefile
The common recipe when TAP tests are disabled doesn't work, because the
libpq-specific recipe wants to define the PATH environment variable, so
the starting '@' is misinterpreted as part of the command instead of
silencing said command.

Fix by setting the environment variable in a way that doesn't interfere
with the recipe.

Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://postgr.es/m/20220720172321.GL12702@telsasoft.com
2022-07-22 20:15:11 +02:00
Alvaro Herrera 83011ce7d7
Rework grammar for REINDEX
The part of grammar have grown needlessly duplicative and more complex
that necessary.  Rewrite.

Reviewed-by: Michaël Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/20220721174212.cmitjpuimx6ssyyj@alvherre.pgsql
2022-07-22 19:23:39 +02:00
Tom Lane 0b292bed92 Close old gap in dependency checks for functions returning composite.
The dependency logic failed to register a column-level dependency
when a view or rule contains a reference to a specific column of
the result of a function-returning-composite.  That meant you could
drop the column from the composite type, causing trouble for future
executions of the view.  We've known about this for years, but never
summoned the energy to actually fix it, instead installing various
low-level defenses to prevent crashing on references to dropped columns.
We had to do that to plug the hole in stable branches, where there might
be pre-existing broken references; but let's fix the root cause today.

To do that, add some logic (borrowed from get_rte_attribute_is_dropped)
to find_expr_references_walker, to check whether a Var referencing an
RTE_FUNCTION RTE is referencing a column of a composite type, and if
so add the proper dependency.

However ... it seems mighty unwise to remove said low-level defenses,
since there could be other bugs now or in the future that allow
reaching them.  By the same token, letting those defenses go untested
seems unwise.  Hence, rather than just dropping the associated test
cases, hack them to continue working by the expedient of manually
dropping the pg_depend entries that this fix installs.

Back-patch into v15.  I don't want to risk changing this behavior
in stable branches, but it seems not too late for v15.  (Since
we have already forced initdb for beta3, we can be sure that all
production v15 installations will have these added dependencies.)

Discussion: https://postgr.es/m/182492.1658431155@sss.pgh.pa.us
2022-07-22 12:46:42 -04:00
Tom Lane 90474c16a7 Fix minor memory leaks in psql's tab completion.
Tang Haiying and Tom Lane

Discussion: https://postgr.es/m/OS0PR01MB6113EA19F05E217C823B4CCAFB909@OS0PR01MB6113.jpnprd01.prod.outlook.com
2022-07-22 10:53:26 -04:00
Alvaro Herrera 7d158e8cb4
parser: centralize common auxiliary productions
Things like "opt_name" can well be shared by various commands rather
than there being multiple definitions of the same thing.  Rename these
productions and move them to appear together in gram.y, which may
improve chances of reuse in the future.

Discussion: https://postgr.es/m/20220721174212.cmitjpuimx6ssyyj@alvherre.pgsql
2022-07-22 13:13:20 +02:00
Alvaro Herrera 9853bf6ab0
Update src/backend/parser/README
New files have been added to this directory, but not listed here.
Repair.
2022-07-22 12:56:21 +02:00
Thomas Munro 5344723755 Remove unnecessary Windows-specific basebackup code.
Commit c6f2f016 added an explicit check for a Windows "junction point".
That turned out to be needed only because get_dirent_type() was busted
on Windows.  It's been fixed by commit 9d3444dc, so remove it.

Add a TAP-test to demonstrate that in-place tablespaces are copied by
pg_basebackup.  This exercises the codepath that would fail before
c6f2f016 on Windows, and shows that it still doesn't fail now that we're
using get_dirent_type() on both Windows and Unix.

Back-patch to 15, where in-place tablespaces arrived and caused this
problem (ie directories where previously only symlinks were expected).

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/CA%2BhUKGLzLK4PUPx0_AwXEWXOYAejU%3D7XpxnYE55Y%2Be7hB2N3FA%40mail.gmail.com
2022-07-22 17:41:47 +12:00
Thomas Munro 9d3444dcce Fix get_dirent_type() for Windows junction points.
Commit 87e6ed7c8 added code that intended to report Windows "junction
points" as DT_LNK (the same way we report symlinks on Unix).  Windows
junction points are *also* directories according to the Windows
attributes API, and we were reporting them as as DT_DIR.  Change the
order we check the attribute flags, to prioritize DT_LNK.

If at some point we start using Windows' recently added real symlinks
and need to distinguish them from junction points, we may need to
rethink this, but for now this continues the tradition of wrapper
functions that treat junction points as symlinks.

Back-patch to 14, where get_dirent_type() landed.

Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Alvaro Herrera <alvherre@alvh.no-ip.org>
Discussion: https://postgr.es/m/CA%2BhUKGLzLK4PUPx0_AwXEWXOYAejU%3D7XpxnYE55Y%2Be7hB2N3FA%40mail.gmail.com
Discussion: https://postgr.es/m/20220721111751.x7hod2xgrd76xr5c%40alvherre.pgsql
2022-07-22 16:57:12 +12:00
Fujii Masao 44ccdce514 postgres_fdw: Fix bug in checking of return value of PQsendQuery().
When postgres_fdw begins an asynchronous data fetch, it submits FETCH query
by using PQsendQuery(). If PQsendQuery() fails and returns 0, postgres_fdw
should report an error. But, previously, postgres_fdw reported an error
only when the return value is less than 0, though PQsendQuery() never return
the values other than 0 and 1. Therefore postgres_fdw could not handle
the failure to send FETCH query in an asynchronous data fetch.

This commit fixes postgres_fdw so that it reports an error
when PQsendQuery() returns 0.

Back-patch to v14 where asynchronous execution was supported in postgres_fdw.

Author: Fujii Masao
Reviewed-by: Japin Li, Tom Lane
Discussion: https://postgr.es/m/b187a7cf-d4e3-5a32-4d01-8383677797f3@oss.nttdata.com
2022-07-22 11:59:38 +09:00
Thomas Munro a1b56090eb Remove O_FSYNC and associated macros.
O_FSYNC was a pre-POSIX way of spelling O_SYNC, supported since commit
9d645fd84c for non-conforming operating systems of the time.  It's not
needed on any modern system.  We can just use standard O_SYNC directly
if it exists (= all targeted systems except Windows), and get rid of our
OPEN_SYNC_FLAG macro.

Similarly for standard O_DSYNC, we can just use that directly if it
exists (= all targeted systems except DragonFlyBSD), and get rid of our
OPEN_DATASYNC_FLAG macro.

We still avoid choosing open_datasync as a default value for
wal_sync_method if O_DSYNC has the same value as O_SYNC (= only
OpenBSD), so there is no change in default behavior.

Discussion: https://postgr.es/m/CA%2BhUKGJE7y92NY7FG2ftUbZUaqohBU65_Ys_7xF5mUHo4wirTQ%40mail.gmail.com
2022-07-22 12:41:17 +12:00
Thomas Munro 4f1f5a7f85 Remove fls(), use pg_leftmost_one_pos32() instead.
Commit 4f658dc8 provided the traditional BSD fls() function in
src/port/fls.c so it could be used in several places.  Later we added a
bunch of similar facilities in pg_bitutils.h, based on compiler
builtins that map to hardware instructions.  It's a bit confusing to
have both 1-based and 0-based variants of this operation in use in
different parts of the tree, and neither is blessed by a standard.
Let's drop fls.c and the configure probe, and reuse the newer code.

Reviewed-by: David Rowley <dgrowleyml@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKG%2B7dSX1XF8yFGmYk-%3D48dbjH2kmzZj16XvhbrWP-9BzRg%40mail.gmail.com
2022-07-22 10:41:50 +12:00
Thomas Munro 3225399021 Extend size_t support in pg_bitutils.h.
Use a more compact notation that allows us to add more size_t variants
as required.  This will be used by a later commit.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CA%2BhUKG%2B7dSX1XF8yFGmYk-%3D48dbjH2kmzZj16XvhbrWP-9BzRg%40mail.gmail.com
2022-07-22 09:40:39 +12:00
Bruce Momjian a4f09ef229 doc: use wording "restore" instead of "reload" of dumps
Reported-by: axel.kluener@gmail.com

Discussion: https://postgr.es/m/164736074430.660.3645615289283943146@wrigleys.postgresql.org

Backpatch-through: 11
2022-07-21 14:55:23 -04:00
Dean Rasheed 624aa2a13b Make the name optional in CREATE STATISTICS.
This allows users to omit the statistics name in a CREATE STATISTICS
command, letting the system auto-generate a sensible, unique name,
putting the statistics object in the same schema as the table.

Simon Riggs, reviewed by Matthias van de Meent.

Discussion: https://postgr.es/m/CANbhV-FGD2d_C3zFTfT2aRfX_TaPSgOeKES58RLZx5XzQp5NhA@mail.gmail.com
2022-07-21 19:23:13 +01:00
Bruce Momjian fa6c230ef2 doc: clarify that auth. names are lower case and case-sensitive
This is true even for acronyms that are usually upper case, like LDAP.

Reported-by: Alvaro Herrera

Discussion: https://postgr.es/m/202205141521.2nodjabmsour@alvherre.pgsql

Backpatch-through: 10
2022-07-21 13:58:45 -04:00
Tom Lane b9654cecea Fix ruleutils issues with dropped cols in functions-returning-composite.
Due to lack of concern for the case in the dependency code, it's
possible to drop a column of a composite type even though stored
queries have references to the dropped column via functions-in-FROM
that return the composite type.  There are "soft" references,
namely FROM-clause aliases for such columns, and "hard" references,
that is actual Vars referring to them.  The right fix for hard
references is to add dependencies preventing the drop; something
we've known for many years and not done (and this commit still doesn't
address it).  A "soft" reference shouldn't prevent a drop though.
We've been around on this before (cf. 9b35ddce9, 2c4debbd0), but
nobody had noticed that the current behavior can result in dump/reload
failures, because ruleutils.c can print more column aliases than the
underlying composite type now has.  So we need to rejigger the
column-alias-handling code to treat such columns as dropped and not
print aliases for them.

Rather than writing new code for this, I used expandRTE() which already
knows how to figure out which function result columns are dropped.
I'd initially thought maybe we could use expandRTE() in all cases, but
that fails for EXPLAIN's purposes, because the planner strips a lot of
RTE infrastructure that expandRTE() needs.  So this patch just uses it
for unplanned function RTEs and otherwise does things the old way.

If there is a hard reference (Var), then removing the column alias
causes us to fail to print the Var, since there's no longer a name
to print.  Failing seems less desirable than printing a made-up
name, so I made it print "?dropped?column?" instead.

Per report from Timo Stolz.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/5c91267e-3b6d-5795-189c-d15a55d61dbb@nullachtvierzehn.de
2022-07-21 13:56:02 -04:00
Amit Kapila 2a51a066dd Add missing space in comments.
Author: Junwang Zhao
Discussion: https://postgr.es/m/CAEG8a3++YQ6A-y5-w6KxP8QH6qxDJDk4dEtZw0cLcW9bsQFydg@mail.gmail.com
2022-07-21 16:25:07 +05:30
Amit Kapila 366283961a Allow users to skip logical replication of data having origin.
This patch adds a new SUBSCRIPTION parameter "origin". It specifies
whether the subscription will request the publisher to only send changes
that don't have an origin or send changes regardless of origin. Setting it
to "none" means that the subscription will request the publisher to only
send changes that have no origin associated. Setting it to "any" means
that the publisher sends changes regardless of their origin. The default
is "any".
Usage:
CREATE SUBSCRIPTION sub1 CONNECTION 'dbname=postgres port=9999'
PUBLICATION pub1 WITH (origin = none);

This can be used to avoid loops (infinite replication of the same data)
among replication nodes.

This feature allows filtering only the replication data originating from
WAL but for initial sync (initial copy of table data) we don't have such a
facility as we can only distinguish the data based on origin from WAL. As
a follow-up patch, we are planning to forbid the initial sync if the
origin is specified as none and we notice that the publication tables were
also replicated from other publishers to avoid duplicate data or loops.

We forbid to allow creating origin with names 'none' and 'any' to avoid
confusion with the same name options.

Author: Vignesh C, Amit Kapila
Reviewed-By: Peter Smith, Amit Kapila, Dilip Kumar, Shi yu, Ashutosh Bapat, Hayato Kuroda
Discussion: https://postgr.es/m/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
2022-07-21 08:47:38 +05:30
Fujii Masao f2d0c7f18b docs: Improve pg_settings_get_flags docs.
In the docs, the GUC flags that pg_settings_get_flags() reported were
listed using <simplelist>. But the list was treated as separate lines
in the existing function table and didn't look good. For better view,
this commit separates the list from the table entry for
pg_settings_get_flags() and adds the table for it at the bottom of
the existing function table.

Author: Fujii Masao
Reviewed-by: Alvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/f093edf9-6e5a-b119-ee50-6a2c97c79ee8@oss.nttdata.com
2022-07-21 11:58:04 +09:00
Michael Paquier 2b99ce10c2 Tweak a bit the new TAP tests of REINDEX DATABASE/SYSTEM
This renames the relation storing the relfilenode state into something
more generic as it also stores data for non-toast relations.  A
restriction on the number of digits used for the OID number when
filtering toast relation names is removed, while on it, as there is no
need for it.

Reported-by: Justin Pryzby
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/20220719022652.GE12702@telsasoft.com
2022-07-21 11:00:48 +09:00
Michael Paquier 171ab95f64 Fix various memory leaks in psql's describe commands \d*
Most of these have been introduced in d2d3547 with the new pattern
validation logic, and would leak memory worth an amount of one
PQExpBuffer each time (as of 256 bytes at minimum, possibly more).

Most of the patch has been written by Tang Haiying, with a few tweaks
coming from Álvaro Herrera.

Reported-by: Tang Haiying
Author: Tang Haiying, Álvaro Herrera
Reviewed-by: Mark Dilger, Andres Freund, Álvaro Herrera, Tom Lane, Japin
Li, Michael Paquier, Junwang Zhao
Backpatch-through: 15
2022-07-21 10:41:33 +09:00
Tom Lane af119e08fd Dump more fields when dumping planner internal data structures.
Commit 964d01ae9 marked a lot of fields as read_write_ignore
to stay consistent with what was dumped by the manually-maintained
outfuncs.c code.  However, it seems that a pretty fair number
of those omissions were either flat-out oversights, or a shortcut
taken because hand-written code seemed like it'd be too much trouble.
Let's upgrade things where it seems to make sense to dump.

To do this, we need to add support to gen_node_support.pl and
outfuncs.c for variable-length arrays of Node pointers.  That's
pretty straightforward given the model of the existing code
for arrays of scalars, but I found I needed to tighten the
type-recognizing regexes in gen_node_support.pl.  (As they
stood, they mistook "foo **" for "foo *".  Make sure they're
all fully anchored to prevent additional problems.)

The main thing left un-done here is that a lot of partitioning-related
structs are still not dumped, because they are bare structs not Nodes.
I'm not sure about the wisdom of that choice ... but changing it would
be fairly invasive, so it probably requires more justification than
just making planner node dumps more complete.

Discussion: https://postgr.es/m/1295668.1658258637@sss.pgh.pa.us
2022-07-20 13:54:30 -04:00
Jeff Davis 6c31ac091f Process shared_preload_libraries in single-user mode.
Without processing shared_preload_libraries, it's impossible to
recover if custom WAL resource managers are needed. It may also pose a
problem running VACUUM on a table with a custom AM, if the module
implementing the AM is expecting to be loaded by
shared_preload_libraries.

The reason this wasn't done before was just the general principle to
do fewer things in single-user mode. But it's easy enough to just set
shared_preload_libraries to empty, for the same effect.

Discussion: https://postgr.es/m/9decc18a42634f8a2f15c97a385a0f51a752f396.camel%40j-davis.com
Reviewed-by: Tom Lane, Andres Freund
Backpatch-through: 15
2022-07-20 10:15:47 -07:00
Tom Lane 2d04277121 Make serialization of Nodes' scalar-array fields more robust.
When the ability to print variable-length-array fields was first
added to outfuncs.c, there was no corresponding read capability,
as it was used only for debug dumps of planner-internal Nodes.
Not a lot of thought seems to have been put into the output format:
it's just the space-separated array elements and nothing else.
Later such fields appeared in Plan nodes, and still later we grew
read support so that Plans could be transferred to parallel workers,
but the original text format wasn't rethought.  It seems inadequate
to me because (a) no cross-check is possible that we got the right
number of array entries, (b) we can't tell the difference between
a NULL pointer and a zero-length array, and (c) except for
WRITE_INDEX_ARRAY, we'd crash if a non-zero length is specified
when the pointer is NULL, a situation that can arise in some fields
that we currently conveniently avoid printing.

Since we're currently in a campaign to make the Node infrastructure
generally more it-just-works-without-thinking-about-it, now seems
like a good time to improve this.

Let's adopt a format similar to that used for Lists, that is "<>"
for a NULL pointer or "(item item item)" otherwise.  Also retool
the code to not have so many copies of the identical logic.

I bumped catversion out of an abundance of caution, although I think
that we don't use any such array fields in Nodes that can get into
the catalogs.

Discussion: https://postgr.es/m/1528424.1658272135@sss.pgh.pa.us
2022-07-20 13:04:33 -04:00
Alexander Korotkov f77ff08335 Document the ability to specify TableAM for pgbench
Upcoming custom Table Access Methods (TableAM) need benchmarking.  Despite
pgbench doesn't have an explicit option for TableAM specification, one can
specify it using PGOPTION environmental variable.  The present commit documents
this way to specify TableAM for pgbench.

Discussion: https://postgr.es/m/CAC77N6ih%3DLbhZQXV76grEsaVQkBL464Y2Foqq9o%3Df4UBfEOfEQ%40mail.gmail.com
Author: Michel Pelletier, Alexander Korotkov
Reviewed-by: Justin Pryzby, Mason Sharp, Michael Paquier
2022-07-20 15:49:37 +03:00
Dean Rasheed bcedd8f5fc Make subquery aliases optional in the FROM clause.
This allows aliases for sub-SELECTs and VALUES clauses in the FROM
clause to be omitted.

This is an extension of the SQL standard, supported by some other
database systems, and so eases the transition from such systems, as
well as removing the minor inconvenience caused by requiring these
aliases.

Patch by me, reviewed by Tom Lane.

Discussion: https://postgr.es/m/CAEZATCUCGCf82=hxd9N5n6xGHPyYpQnxW8HneeH+uP7yNALkWA@mail.gmail.com
2022-07-20 09:29:42 +01:00
Alvaro Herrera 1caf915ff3
Add PGDLLEXPORTS to some plpgsql function declarations
After -fvisibility=hidden was added by 089480c077, plpgsql_check no
longer works; this quick hack fixes it.  It would be better to
restructure the plpgsql.h header so that this doesn't look as random,
but we can leave that for another day.

Reported-by: Pavel Stehule
Discussion: https://postgr.es/m/CAFj8pRAFxc3-SHMD3URU09JZXEKY3W-RwXKp8xPEnEq8rrka7w@mail.gmail.com
2022-07-20 10:24:50 +02:00
Thomas Munro bde60daa0e Fix warnings on Windows.
Avoid macro redefinition warnings.

Reported-by: David Rowley <dgrowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvocHp4SXcPeMTwFiCQGaf9JypjTJ3Bh90jcPuGwxyDjjQ%40mail.gmail.com
2022-07-20 16:09:50 +12:00
Thomas Munro 9430fb407b Add wal_sync_method=fdatasync for Windows.
Windows 10 gained support for flushing NTFS files with fdatasync()
semantics.  The main advantage over open_datasync (in Windows API terms
FILE_FLAG_WRITE_THROUGH) is that the latter does not flush SATA drive
caches.  The default setting is not changed, so users have to opt in to
this.

Discussion: https://postgr.es/m/CA%2BhUKGJZJVO%3DiX%2Beb-PXi2_XS9ZRqnn_4URh0NUQOwt6-_51xQ%40mail.gmail.com
2022-07-20 13:55:52 +12:00
Fujii Masao b24b2be119 Fix assertion failure and segmentation fault in backup code.
When a non-exclusive backup is canceled, do_pg_abort_backup() is called
and resets some variables set by pg_backup_start (pg_start_backup in v14
or before). But previously it forgot to reset the session state indicating
whether a non-exclusive backup is in progress or not in this session.

This issue could cause an assertion failure when the session running
BASE_BACKUP is terminated after it executed pg_backup_start and
pg_backup_stop (pg_stop_backup in v14 or before). Also it could cause
a segmentation fault when pg_backup_stop is called after BASE_BACKUP
in the same session is canceled.

This commit fixes the issue by making do_pg_abort_backup reset
that session state.

Back-patch to all supported branches.

Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Michael Paquier, Robert Haas
Discussion: https://postgr.es/m/3374718f-9fbf-a950-6d66-d973e027f44c@oss.nttdata.com
2022-07-20 09:57:01 +09:00
Fujii Masao ee79647769 Prevent BASE_BACKUP in the middle of another backup in the same session.
Multiple non-exclusive backups are able to be run conrrently in different
sessions. But, in the same session, only one non-exclusive backup can be
run at the same moment. If pg_backup_start (pg_start_backup in v14 or before)
is called in the middle of another non-exclusive backup in the same session,
an error is thrown.

However, previously, in logical replication walsender mode, even if that
walsender session had already called pg_backup_start and started
a non-exclusive backup, it could execute BASE_BACKUP command and
start another non-exclusive backup. Which caused subsequent pg_backup_stop
to throw an error because BASE_BACKUP unexpectedly reset the session state
marked by pg_backup_start.

This commit prevents BASE_BACKUP command in the middle of another
non-exclusive backup in the same session.

Back-patch to all supported branches.

Author: Fujii Masao
Reviewed-by: Kyotaro Horiguchi, Masahiko Sawada, Michael Paquier, Robert Haas
Discussion: https://postgr.es/m/3374718f-9fbf-a950-6d66-d973e027f44c@oss.nttdata.com
2022-07-20 09:56:42 +09:00
Michael Paquier 12c254c99f Tweak detail and hint messages to be consistent with project policy
Detail and hint messages should be full sentences and should end with a
period, but some of the messages newly-introduced in v15 did not follow
that.

Author: Justin Pryzby
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/20220719120948.GF12702@telsasoft.com
Backpatch-through: 15
2022-07-20 09:50:12 +09:00
Fujii Masao ecc84b916f Add regression test for TRUNCATE on foreign table not supporting TRUNCATE.
file_fdw doesn't support INSERT, UPDATE, DELETE and TRUNCATE.
It has the regression test that confirms that INSERT, UPDATE and DELETE
fail on its foreign table, but not TRUNCATE yet. It's better to
also test TRUNCATE fails on a foreign table not allowing TRUNCATE,
for test coverage. This commit adds that regression test using file_fdw.

Author: Yugo Nagata
Reviewed-by: Fujii Masao
Discussion: https://postgr.es/m/20220630104812.ec1556481452c019874f4ac9@sraoss.co.jp
2022-07-20 09:35:14 +09:00
Tom Lane 13d8388151 Fix missed corner cases for grantable permissions on GUCs.
We allow users to set the values of not-yet-loaded extension GUCs,
remembering those values in "placeholder" GUC entries.  When/if
the extension is loaded later in the session, we need to verify that
the user had permissions to set the GUC.  That was done correctly
before commit a0ffa885e, but as of that commit, we'd check the
permissions of the active role when the LOAD happens, not the role
that had set the value.  (This'd be a security bug if it had made it
into a released version.)

In principle this is simple enough to fix: we just need to remember
the exact role OID that set each GUC value, and use that not
GetUserID() when verifying permissions.  Maintaining that data in
the guc.c data structures is slightly tedious, but fortunately it's
all basically just copy-n-paste of the logic for tracking the
GucSource of each setting, as we were already doing.

Another oversight is that validate_option_array_item() hadn't
been taught to check for granted GUC privileges.  This appears
to manifest only in that ALTER ROLE/DATABASE RESET ALL will
fail to reset settings that the user should be allowed to reset.

Patch by myself and Nathan Bossart, per report from Nathan Bossart.
Back-patch to v15 where the faulty code came in.

Discussion: https://postgr.es/m/20220706224727.GA2158260@nathanxps13
2022-07-19 17:21:55 -04:00
Tom Lane d6a3aeb9a3 Convert planner's AggInfo and AggTransInfo structs to proper Nodes.
This is mostly just to get outfuncs.c support for them, so that
the agginfos and aggtransinfos lists can be dumped when dumping
the contents of PlannerInfo.

While here, improve some related comments; notably, clean up
obsolete comments left over from when preprocess_minmax_aggregates
had to make its own scan of the query tree.

Discussion: https://postgr.es/m/742479.1658160504@sss.pgh.pa.us
2022-07-19 12:29:37 -04:00
Tom Lane e2f6c307c0 Estimate cost of elided SubqueryScan, Append, MergeAppend nodes better.
setrefs.c contains logic to discard no-op SubqueryScan nodes, that is,
ones that have no qual to check and copy the input targetlist unchanged.
(Formally it's not very nice to be applying such optimizations so late
in the planner, but there are practical reasons for it; mostly that we
can't unify relids between the subquery and the parent query until we
flatten the rangetable during setrefs.c.)  This behavior falsifies our
previous cost estimates, since we would've charged cpu_tuple_cost per
row just to pass data through the node.  Most of the time that's little
enough to not matter, but there are cases where this effect visibly
changes the plan compared to what you would've gotten with no
sub-select.

To improve the situation, make the callers of cost_subqueryscan tell
it whether they think the targetlist is trivial.  cost_subqueryscan
already has the qual list, so it can check the other half of the
condition easily.  It could make its own determination of tlist
triviality too, but doing so would be repetitive (for callers that
may call it several times) or unnecessarily expensive (for callers
that can determine this more cheaply than a general test would do).

This isn't a 100% solution, because createplan.c also does things
that can falsify any earlier estimate of whether the tlist is
trivial.  However, it fixes nearly all cases in practice, if results
for the regression tests are anything to go by.

setrefs.c also contains logic to discard no-op Append and MergeAppend
nodes.  We did have knowledge of that behavior at costing time, but
somebody failed to update it when a check on parallel-awareness was
added to the setrefs.c logic.  Fix that while we're here.

These changes result in two minor changes in query plans shown in
our regression tests.  Neither is relevant to the purposes of its
test case AFAICT.

Patch by me; thanks to Richard Guo for review.

Discussion: https://postgr.es/m/2581077.1651703520@sss.pgh.pa.us
2022-07-19 11:18:19 -04:00
Alvaro Herrera 1679d57a55
Wrap overly long lines
Reported by Richard Guo.

Reviewed-by: Richard Guo <guofenglinux@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAMbWs4-3ywL_tPHJKk-Vvzr-tBaR--b6XxGGm8Xe7vsG38AWog@mail.gmail.com
2022-07-19 09:54:03 +02:00
Peter Eisentraut 4371d34f29 Clean up temp file from refactored dtrace rule
related to eb6569fd0e
2022-07-19 07:31:58 +02:00
Peter Eisentraut 14a8bd9827 Convert macros to static inline functions (itup.h)
Reviewed-by: Amul Sul <sulamul@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/5b558da8-99fb-0a99-83dd-f72f05388517%40enterprisedb.com
2022-07-19 07:24:55 +02:00
Michael Paquier 2cbc3c17a5 Rework logic and simplify syntax of REINDEX DATABASE/SYSTEM
Per discussion, this commit includes a couple of changes to these two
flavors of REINDEX:
* The grammar is changed to make the name of the object optional, hence
one can rebuild all the indexes of the wanted area by specifying only
"REINDEX DATABASE;" or "REINDEX SYSTEM;".  Previously, the object name
was mandatory and had to match the name of the database on which the
command is issued.
* REINDEX DATABASE is changed to ignore catalogs, making this task only
possible with REINDEX SYSTEM.  This is a historical change, but there
was no way to work only on the indexes of a database without touching
the catalogs.  We have discussed more approaches here, like the addition
of an option to skip the catalogs without changing the original
behavior, but concluded that what we have here is for the best.

This builds on top of the TAP tests introduced in 5fb5b6c, showing the
change in behavior for REINDEX SYSTEM.  reindexdb is updated so as we do
not issue an extra REINDEX SYSTEM when working on a database in the
non-concurrent case, something that was confusing when --concurrently
got introduced, so this simplifies the code.

Author: Simon Riggs
Reviewed-by: Ashutosh Bapat, Bernd Helmle, Álvaro Herrera, Cary Huang,
Michael Paquier
Discussion: https://postgr.es/m/CANbhV-H=NH6Om4-X6cRjDWfH_Mu1usqwkuYVp-hwdB_PSHWRfg@mail.gmail.com
2022-07-19 11:45:06 +09:00
Michael Paquier 5fb5b6c4c1 Add more tests for REINDEX DATABASE/SYSTEM with relfilenode changes
Adding such commands in the main regression test suite is not a good
approach performance-wise as it impacts all the objects in the
regression database, so this additional coverage is added in the TAP
tests of reindexdb where we already run a few REINDEX commands with
SYSTEM and DATABASE so there is no runtime difference for the test.
This additional coverage checks which relations are rewritten with
relfilenode changes, as of:
- a toast index in user table.
- a toast index in catalog table.
- a catalog index.
- a user index.

This test suite is something I have implemented for a separate patch
that reworks a bit the way we handle these two REINDEX behaviors, but it
has enough value in itself to be in a separate commit.  This also makes
easier to follow what actually changes once the REINDEX logic is
reworked (currently, DABATASE rewrites both catalog and user tables, and
SYSTEM works only on catalogs).

Discussion: https://postgr.es/m/YtOqA7ldcJQADEE8@paquier.xyz
2022-07-19 10:51:27 +09:00
Andres Freund 950e64fa46 Use STDOUT/STDERR_FILENO in most of syslogger.
This fixes problems on windows when logging collector is used in a service,
failing with:
FATAL:  could not redirect stderr: Bad file descriptor

This is triggered by 76e38b37a5. The problem is that STDOUT/STDERR_FILENO
aren't defined on windows, which lead us to use _fileno(stdout) etc, but that
doesn't work if stdout/stderr are closed.

Author: Andres Freund <andres@anarazel.de>
Reported-By: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
Message-Id: 20220520164558.ozb7lm6unakqzezi@alap3.anarazel.de (on pgsql-packagers)
Backpatch: 15-, where 76e38b37a5 came in
2022-07-18 17:22:11 -07:00
Andres Freund c290e79cf0 windows: msvc: Define STDIN/OUT/ERR_FILENO.
Because they are not available we've used _fileno(stdin) in some places, but
that doesn't reliably work, because stdin might be closed. This is the
prerequisite of the subsequent commit, fixing a failure introduced in
76e38b37a5.

Author: Andres Freund <andres@anarazel.de>
Reported-By: Sandeep Thakkar <sandeep.thakkar@enterprisedb.com>
Message-Id: 20220520164558.ozb7lm6unakqzezi@alap3.anarazel.de (on pgsql-packagers)
Backpatch: 15-, where 76e38b37a5 came in
2022-07-18 17:21:54 -07:00
Tom Lane 0778eb79b1 Improve perl style in ecpg's parser-construction scripts.
parse.pl and check_rules.pl used "no warnings 'uninitialized'",
which doesn't seem like it measures up to current project standards.
Removing that shows that it was hiding various places that accessed
off the end of an array, which are easily protected by minor logic
adjustments.  There's no change in the script results.

While here, improve the Makefile rule that invokes these scripts.
It neglected to depend on check_rules.pl, so that editing that file
didn't result in re-running the check; and it ran check_rules.pl
after building preproc.y, so that if check_rules.pl did fail the
next "make" attempt would just bypass it.  check_rules.pl failures
are sufficiently un-heard-of that I don't feel a need to back-patch
this.

Discussion: https://postgr.es/m/838180.1658181982@sss.pgh.pa.us
2022-07-18 19:43:16 -04:00
Andres Freund d268d0f7a1 ecpg: use our instead of my in parse.pl to fix perlcritic complaint
In db0a272d12 I used open(our $something, ...), which perlcritic doesn't
like. It looks like the warning is due to perlcritic knowing about 'my' but
not 'our' when checking for bareword file handles.

However, it's clearly unnecessary to use "our" here, change it to "my".

Via buildfarm member crake and discussion with Tom Lane.

Discussion: https://postgr.es/m/20220718215042.sl3hivoupdb7lkwv@awork3.anarazel.de
2022-07-18 14:53:02 -07:00
Andres Freund eb6569fd0e Refactor dtrace postprocessing make rules
This is in preparation for building postgres with meson / ninja.

Move the dtrace postprocessing sed commands into a separate file so
that it can be shared by meson.  Also split the rule into two for
proper dependency declaration.

Reviewed-by: Andres Freund <andres@anarazel.de>
Author: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/5e216522-ba3c-f0e6-7f97-5276d0270029@enterprisedb.com
2022-07-18 12:33:02 -07:00
Andres Freund adba4b7471 Add output directory option to gen_node_support.pl
This is in preparation for building postgres with meson / ninja.

When building with meson, commands are run at the root of the build tree. Add
an option to put build output into the appropriate place. This can be utilized
by src/tools/msvc/ for a minor simplification, which also provides some
coverage for the new option.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/5e216522-ba3c-f0e6-7f97-5276d0270029@enterprisedb.com
2022-07-18 12:32:53 -07:00
Andres Freund c8a9246e09 Add output directory argument to generate-unicode_norm_table.pl
This is in preparation for building postgres with meson / ninja.

When building with meson, commands are run at the root of the build tree. Add
an option to put build output into the appropriate place.

Author: Andres Freund <andres@anarazel.de>
Author: Peter Eisentraut <peter@eisentraut.org>
Author: Nazir Bilal Yavuz <byavuz81@gmail.com>
Discussion: https://postgr.es/m/5e216522-ba3c-f0e6-7f97-5276d0270029@enterprisedb.com
2022-07-18 12:24:39 -07:00