Commit Graph

39805 Commits

Author SHA1 Message Date
Peter Eisentraut 2016055a92 Expand palloc/pg_malloc API for more type safety
This adds additional variants of palloc, pg_malloc, etc. that
encapsulate common usage patterns and provide more type safety.

Specifically, this adds palloc_object(), palloc_array(), and
repalloc_array(), which take the type name of the object to be
allocated as its first argument and cast the return as a pointer to
that type.  There are also palloc0_object() and palloc0_array()
variants for initializing with zero, and pg_malloc_*() variants of all
of the above.

Inspired by the talloc library.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/bb755632-2a43-d523-36f8-a1e7a389a907@enterprisedb.com
2022-09-12 08:45:03 +02:00
John Naylor b060f57791 Make eval statement naturally proof against perltidy
This is a bit less verbose than adding an exception. Rewrite
the other eval statement in Catalog.pm as well, just for
the sake of consistency.

Idea from Andrew Dunstan
Discussion: https://www.postgresql.org/message-id/CAD5tBc%2BzRhuWn_S4ayH2sWKe60FQu1guTTokDFS3YMOeSrsozA%40mail.gmail.com
2022-09-12 11:59:43 +07:00
Michael Paquier c3fb5809ea Replace loading of ldap_start_tls_sA() by direct function call
This change impacts the backend-side code in charge of starting a LDAP
TLS session.  It is a bit sad that it is not possible to unify the WIN32
and non-WIN32 code paths, but the different number of arguments for both
discard this possibility.

This is similar to 47bd0b3, where this replaces the last function
loading that seems worth it, any others being either environment or
version-dependent.

Reported-by: Thomas Munro
Reviewed-by: Thomas Munro
Discussion: https://postgr.es/m/Yx0rxpNgDh8tN4XA@paquier.xyz
2022-09-12 09:07:10 +09:00
Michael Paquier 6afcab6ac1 Add psql tab compression for ALTER TABLE .. { OF | NOT OF }
ALTER TABLE .. OF is now able to complete with the list of available
composite types that can be used with the query.

Author: Aleksander Alekseev
Reviewed-by: Shinya Kato
Discussion: https://postgr.es/m/47b71e0c523b30357208e79786161281@oss.nttdata.com
2022-09-10 17:22:29 +09:00
Michael Paquier 799437e0bd Free correctly LDAPMessage returned by ldap_search_s() in auth.c
The LDAP wiki states that the search message should be freed regardless
of the return value of ldap_search_s(), but we failed to do so in one
backend code path when searching LDAP with a filter.  This is not
critical in an authentication code path failing in the backend as this
causes such the process to exit promptly, but let's be clean and free
the search message appropriately, as documented by upstream.

All the other code paths failing a LDAP operation do that already, and
somebody looking at this code in the future may miss what LDAP expects
with the search message.

Author: Zhihong Yu
Discussion: https://postgr.es/m/CALNJ-vTf5Y+8RtzZ4GjOGE9qWVHZ8awfhnFYc_qGm8fMLUNRAg@mail.gmail.com
2022-09-10 16:56:07 +09:00
Andres Freund fe6a64a58a aix: No need to use mkldexport when we want to export all symbols
When building a shared library with exports.txt there's no need to build an
intermediary static library, we can just pass -Wl,-bE:... when generating the
.so.

When building a shared library without exports.txt, there's no need to call
mkldexport.sh to export all symbols, because all symbols are exported anyway,
and we don't need the export file on the import side (like we do for
postgres.imp).

This makes building .so's on aix a lot more similar to building on other
platforms. In particular, we don't create and remove a .a of the same name but
different contents anymore.

Discussion: https://postgr.es/m/20220820174213.d574qde4ptwdzoqz@awork3.anarazel.de
2022-09-09 19:11:49 -07:00
Tom Lane b7050e2584 Fix possible omission of variable storage markers in ECPG.
The ECPG preprocessor converted code such as

static varchar str1[10], str2[20], str3[30];

into

static  struct varchar_1  { int len; char arr[ 10 ]; }  str1 ;
        struct varchar_2  { int len; char arr[ 20 ]; }  str2 ;
        struct varchar_3  { int len; char arr[ 30 ]; }  str3 ;

thus losing the storage attribute for the later variables.
Repeat the declaration for each such variable.

(Note that this occurred only for variables declared "varchar"
or "bytea", which may help explain how it escaped detection
for so long.)

Andrey Sokolov

Discussion: https://postgr.es/m/942241662288242@mail.yandex.ru
2022-09-09 15:34:04 -04:00
Alvaro Herrera 8c848cd4b8
Fix GetForeignKey*Triggers for self-referential FKs
Because of inadequate filtering, the check triggers were confusing the
search for action triggers in GetForeignKeyActionTriggers and vice-versa
in GetForeignKeyCheckTriggers; this confusion results in seemingly
random assertion failures, and can have real impact in non-asserting
builds depending on catalog order.  Change these functions so that they
correctly ignore triggers that are not relevant to each side.

To reduce the odds of further problems, do not break out of the
searching loop in assertion builds.  This break is likely to hide bugs;
without it, we would have detected this bug immediately.

This problem was introduced by f4566345cf, so backpatch to 15 where
that commit first appeared.

Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/20220908172029.sejft2ppckbo6oh5@awork3.anarazel.de
Discussion: https://postgr.es/m/4104619.1662663056@sss.pgh.pa.us
2022-09-09 12:22:20 +02:00
John Naylor 8b878bffa8 Bump minimum version of Flex to 2.5.35
Since the retirement of some older buildfarm members, the oldest Flex
that gets regular testing is 2.5.35.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/1097762.1662145681@sss.pgh.pa.us
2022-09-09 12:55:23 +07:00
John Naylor b086a47a27 Bump minimum version of Bison to 2.3
Since the retirement of some older buildfarm members, the oldest Bison
that gets regular testing is 2.3. MacOS ships that version, and will
continue doing so for the forseeable future because of Apple's policy
regarding GPLv3. While Mac users could use a package manager to install
a newer version, there is no compelling reason to force them do so at
this time.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/1097762.1662145681@sss.pgh.pa.us
2022-09-09 12:31:41 +07:00
John Naylor 96683db880 Add jsonpath_gram.h to list of distprep targets
Oversight in dac048f71e
2022-09-09 11:46:19 +07:00
Michael Paquier 47bd0b3fa6 Replace load of functions by direct calls for some WIN32
This commit changes the following code paths to do direct system calls
to some WIN32 functions rather than loading them from an external
library, shaving some code in the process:
- Creation of restricted tokens in pg_ctl.c, introduced by a25cd81.
- QuerySecurityContextToken() in auth.c for SSPI authentication in the
backend, introduced in d602592.
- CreateRestrictedToken() in src/common/.  This change is similar to the
case of pg_ctl.c.

Most of these functions were loaded rather than directly called because,
as mentioned in the code comments, MinGW headers were not declaring
them.  I have double-checked the recent MinGW code, and all the
functions changed here are declared in its headers, so this change
should be safe.  Note that I do not have a MinGW environment at hand so
I have not tested it directly, but that MSVC was fine with the change.
The buildfarm will tell soon enough if this change is appropriate or not
for a much broader set of environments.

A few code paths still use GetProcAddress() to load some functions:
- LDAP authentication for ldap_start_tls_sA(), where I am not confident
that this change would work.
- win32env.c and win32ntdll.c where we have a per-MSVC version
dependency for the name of the library loaded.
- crashdump.c for MiniDumpWriteDump() and EnumDirTree(), where direct
calls were not able to work after testing.

Reported-by: Thomas Munro
Reviewed-by: Justin Prysby
Discussion: https://postgr.es/m/CA+hUKG+BMdcaCe=P-EjMoLTCr3zrrzqbcVE=8h5LyNsSVHKXZA@mail.gmail.com
2022-09-09 10:52:17 +09:00
Michael Paquier df4a056619 Add more error context to RestoreBlockImage() and consume it
On failure in restoring a block image, no details were provided, while
it is possible to see failure with an inconsistent record state, a
failure in processing decompression or a failure in decompression
because a build does not support this option.

RestoreBlockImage() is used in two code paths in the backend code,
during recovery and when checking a page consistency after applying
masking, and both places are changed to consume the error message
produced by the internal routine when it returns a false status.  All
the error messages are reported under ERRCODE_INTERNAL_ERROR, that gets
used also when attempting to access a page compressed by a method
not supported by the build attempting the decompression.  This is
something that can happen in core when doing physical replication with
primary and standby using inconsistent build options, for example.

This routine is available since 2c03216d and it has never provided any
context about the error happening when it failed.  This change is
justified even more after 57aa5b2, that introduced compression of FPWs
in WAL.

Reported-by: Justin Prysby
Author: Michael Paquier
Discussion: https://postgr.es/m/20220905002320.GD31833@telsasoft.com
Backpatch-through: 15
2022-09-09 10:00:40 +09:00
Peter Geoghegan d977ffd923 Instrument freezing in autovacuum log reports.
Add a new line to log reports from autovacuum (as well as VACUUM VERBOSE
output) that shows information about freezing.  Emphasis is placed on
the total number of heap pages that had one or more tuples frozen by
VACUUM.  The total number of tuples frozen is also shown.

Author: Peter Geoghegan <pg@bowt.ie>
Reviewed-By: Jeff Janes <jeff.janes@gmail.com>
Discussion: https://postgr.es/m/CAH2-WznTY6D0zyE8VLrC6Gd4kh_HGAXxnTPtcOQOOsxzLx9zog@mail.gmail.com
2022-09-08 10:29:39 -07:00
David Rowley b76fb6c2a9 Temporarily make MemoryContextContains return false
5265e91fd changed MemoryContextContains to update it so that it works
correctly with the new MemoryChunk code added in c6e0fe1f2.  However,
5265e91fd was done with the assumption that MemoryContextContains would
only ever be given pointers to memory that had been returned by one of our
MemoryContext allocators.  It seems that's not true and many of our 32-bit
buildfarm animals are clearly showing that.

There are some code paths that call MemoryContextContains with a pointer
pointing part way into an allocated chunk.  The example of this found by
the 32-bit buildfarm animals is the int2int4_sum() function.  This
function returns transdata->sum, which is not a pointer to memory that was
allocated directly.  This return value is then subsequently passed to
MemoryContextContains which causes it to crash due to it thinking the
memory directly prior to that pointer is a MemoryChunk.  What's actually
in that memory is the field in the struct that comes prior to the "sum"
field.  This problem didn't occur in 64-bit world because BIGINT is a
byval type and the code which was calling MemoryContextContains with the
bad pointer only does so with non-byval types.

Here, instead of reverting 5265e91fd and making MemoryContextContains
completely broken again, let's just make it always return false for now.
Effectively prior to 5265e91fd it was doing that anyway, this at least
makes that more explicit.  The only repercussions of this with the current
MemoryContextContains calls are that we perform a datumCopy() when we
might not need to.  This should make the 32-bit buildfarm animals happy
again and give us more time to consider a long-term fix.

Discussion: https://postgr.es/m/20220907130552.sfjri7jublfxyyi4%40jrouhaud
2022-09-09 00:28:38 +12:00
Alvaro Herrera e7936f8b3e
Choose FK name correctly during partition attachment
During ALTER TABLE ATTACH PARTITION, if the name of a parent's foreign
key constraint is already used on the partition, the code tries to
choose another one before the FK attributes list has been populated,
so the resulting constraint name was "<relname>__fkey" instead of
"<relname>_<attrs>_fkey".  Repair, and add a test case.

Backpatch to 12.  In 11, the code to attach a partition was not smart
enough to cope with conflicting constraint names, so the problem doesn't
exist there.

Author: Jehan-Guillaume de Rorthais <jgdr@dalibo.com>
Discussion: https://postgr.es/m/20220901184156.738ebee5@karst
2022-09-08 13:17:02 +02:00
Thomas Munro adb466150b Fix recovery_prefetch with low maintenance_io_concurrency.
We should process completed IOs *before* trying to start more, so that
it is always possible to decode one more record when the decoded record
queue is empty, even if maintenance_io_concurrency is set so low that a
single earlier WAL record might have saturated the IO queue.

That bug was hidden because the effect of maintenance_io_concurrency was
arbitrarily clamped to be at least 2.  Fix the ordering, and also remove
that clamp.  We need a special case for 0, which is now treated the same
as recovery_prefetch=off, but otherwise the number is used directly.
This allows for testing with 1, which would have made the problem
obvious in simple test scenarios.

Also add an explicit error message for missing contrecords.  It was a
bit strange that we didn't report an error already, and became a latent
bug with prefetching, since the internal state that tracks aborted
contrecords would not survive retrying, as revealed by
026_overwrite_contrecord.pl with this adjustment.  Reporting an error
prevents that.

Back-patch to 15.

Reported-by: Justin Pryzby <pryzby@telsasoft.com>
Reviewed-by: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Discussion: https://postgr.es/m/20220831140128.GS31833%40telsasoft.com
2022-09-08 21:44:55 +12:00
Alvaro Herrera 12d40d4a8d
Fix perltidy breaking perlcritic
perltidying a "##no critic" line moves the marker to where it becomes
useless.  Put the line back to how it was, and protect it from further
malfeasance.

Per buildfarm member crake.
2022-09-08 11:20:29 +02:00
John Naylor b2e6e76823 Run perltidy over Catalog.pm
Commit 69eb643b2 deliberately left indentation unchanged to make the changes
more legible. Rather than waiting until next year's perltidy run, do it now
to avoid confusion

Per suggestion from Álvaro Herrera
Discussion: https://www.postgresql.org/message-id/20220907083558.vfvb5hcauaictgum%40alvherre.pgsql
2022-09-08 14:01:13 +07:00
John Naylor 69eb643b25 Parse catalog .dat files as a whole when compiling the backend
Previously Catalog.pm eval'd each individual hash reference
so that comments and whitespace can be preserved when running
reformat-dat-files. This is unnecessary when building, and we can save
~15% off the run time of genbki.pl by simply slurping and eval'-ing
the whole file at once. This saves a bit of time, especially in highly
parallel builds, since most build targets depend on this script's outputs.

Report and review by Andres Freund
Discussion: https://www.postgresql.org/message-id/CAFBsxsGW%3DWRbnxXrc8UqqR479XuxtukSFWV-hnmtgsbuNAUO6w%40mail.gmail.com
2022-09-08 13:55:41 +07:00
Amit Kapila 0324651573 Fix the test case introduced by commit 8756930190.
Before dropping a relation, ensure that it has reached a 'ready' state
after initial synchronization.

Author: Vignesh C
Reviewed-By: Amit Kapila
Discussion: https://www.postgresql.org/message-id/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
2022-09-08 09:47:43 +05:30
Amit Kapila 8756930190 Raise a warning if there is a possibility of data from multiple origins.
This commit raises a warning message for a combination of options
('copy_data = true' and 'origin = none') during CREATE/ALTER subscription
operations if the publication tables were also replicated from other
publishers.

During replication, we can skip the data from other origins as we have that
information in WAL but that is not possible during initial sync so we raise
a warning if there is such a possibility.

Author: Vignesh C
Reviewed-By: Peter Smith, Amit Kapila, Jonathan Katz, Shi yu, Wang wei
Discussion: https://www.postgresql.org/message-id/CALDaNm0gwjY_4HFxvvty01BOT01q_fJLKQ3pWP9=9orqubhjcQ@mail.gmail.com
2022-09-08 06:54:13 +05:30
Alvaro Herrera 4b4663fb4a
Message style fixes 2022-09-07 17:33:49 +02:00
David Rowley 5265e91fd1 Make MemoryContextContains work correctly again
c6e0fe1f2 recently changed the way we store headers for allocated chunks
of memory.  Prior to that commit, we stored a pointer to the owning
MemoryContext directly prior to the pointer to the allocated memory.
That's no longer true and c6e0fe1f2 neglected to update
MemoryContextContains() so that it correctly obtains the owning context
with the new method.

A side effect of this change and c6e0fe1f2, in general, is that it's even
less safe than it was previously to pass MemoryContextContains() an
arbitrary pointer which was not allocated by one of our MemoryContexts.
Previously some comments in MemoryContextContains() seemed to indicate
that the worst that could happen by passing an arbitrary pointer would be
a false positive return value.  It seems to me that this was a rather
wishful outlook as we subsequently proceeded to subtract sizeof(void *)
from the given pointer and then dereferenced that memory.  So it seems
quite likely that we could have segfaulted instead of returning a false
positive.  However, it's not impossible that the memory sizeof(void *)
bytes before the pointer could have been owned by the process, but it's
far less likely to work now as obtaining a pointer to the owning
MemoryContext is less direct than before c6e0fe1f2 and will access memory
that's possibly much further away to obtain the owning MemoryContext.
Because of this, I took the liberty of updating the comment to warn
against any future usages of the function and checked the existing core
usages to ensure that we only ever pass in a pointer to memory allocated
by a MemoryContext.

Extension authors updating their code for PG16 who are using
MemoryContextContains should check to ensure that only NULL pointers and
pointers to chunks allocated with a MemoryContext will ever be passed to
MemoryContextContains.

Reported-by: Andres Freund
Discussion: https://postgr.es/m/20220905230949.kb3x2fkpfwtngz43@awork3.anarazel.de
2022-09-08 00:20:20 +12:00
Peter Eisentraut 3fe76ab972 Renumber confusing value for GUC_UNIT_BYTE
It had a power-of-two value, which looks right, and causes the other
values which aren't powers-of-two to look wrong.  But this is tested
for equality and not a bitwise test.

See also:
6e7baa3227
https://www.postgresql.org/message-id/CAOG9ApEu8bXVwBxkOO9J7ZpM76TASK_vFMEEiCEjwhMmSLiaqQ%40mail.gmail.com

Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20220720145220.GJ12702@telsasoft.com
2022-09-07 11:03:53 +02:00
David Rowley 0e480385ec Make more effort to put a sentinel at the end of allocated memory
Traditionally, in MEMORY_CONTEXT_CHECKING builds, we only ever marked a
sentinel byte just beyond the requested size if there happened to be
enough space on the chunk to do so.  For Slab and Generation context
types, we only rounded the size of the chunk up to the next maxalign
boundary, so it was often not that likely that those would ever have space
for the sentinel given that the majority of allocation requests are going
to be for sizes which are maxaligned.  For AllocSet, it was a little
different as smaller allocations are rounded up to the next power-of-2
value rather than the next maxalign boundary, so we're a bit more likely
to have space for the sentinel byte, especially when we get away from tiny
sized allocations such as 8 or 16 bytes.

Here we make more of an effort to allow space so that there is enough room
for the sentinel byte in more cases.  This makes it more likely that we'll
detect when buggy code accidentally writes beyond the end of any of its
memory allocations.

Each of the 3 MemoryContext types has been changed as follows:

The Slab allocator will now always set a sentinel byte.  Both the current
usages of this MemoryContext type happen to use chunk sizes which were on
the maxalign boundary, so these never used sentinel bytes previously.

For the Generation allocator, we now always ensure there's enough space in
the allocation for a sentinel byte.

For AllocSet, this commit makes an adjustment for allocation sizes which
are greater than allocChunkLimit.  We now ensure there is always space for
a sentinel byte.  We don't alter the sentinel behavior for request sizes
<= allocChunkLimit.  Making way for the sentinel byte for power-of-2
request sizes would require doubling up to the next power of 2.  Some
analysis done on the request sizes made during installcheck shows that a
fairly large portion of allocation requests are for power-of-2 sizes.  The
amount of additional memory for the sentinel there seems prohibitive, so
we do nothing for those here.

Author: David Rowley
Discussion: https://postgr.es/m/3478405.1661824539@sss.pgh.pa.us
2022-09-07 15:46:57 +12:00
Tom Lane 20b6847176 Fix new pg_publication_tables query.
The addition of published column names forgot to filter on attisdropped,
leading to cases where you could see "........pg.dropped.1........"
or the like as a reportedly-published column.

While we're here, rewrite the new subquery to get a more efficient plan
for it.

Hou Zhijie, per report from Jaime Casanova.  Back-patch to v15 where
the bug was introduced.  (Sadly, this means we need a post-beta4
catversion bump before beta4 has even hit the streets.  I see no
good alternative though.)

Discussion: https://postgr.es/m/Yxa1SU4nH2HfN3/i@ahch-to
2022-09-06 18:00:32 -04:00
John Naylor cec2754fbe Fix cplusplusscheck in vpath builds
Same solution as 829906fb6.
2022-09-06 13:51:15 +07:00
Michael Paquier 4cbe579746 Add psql tab compression for SET COMPRESSION with ALTER TABLE
Author: Aleksander Alekseev
Reviewed-by: Shinya Kato
Discussion: https://postgr.es/m/CAJ7c6TMuT+=P7uDepjVpdqSEp2xOmXET3Y2K-xWAO=sCz-28gg@mail.gmail.com
2022-09-06 15:36:42 +09:00
John Naylor 829906fb6c Fix headerscheck in vpath builds
Oversight in dac048f71e per buildfarm animal crake.
Fix per suggestion from Andrew Dunstan.

Discussion: https://www.postgresql.org/message-id/e3f4a3d0-dfcc-41cc-1ed2-acc15700ddef%40dunslane.net
2022-09-06 12:49:26 +07:00
John Naylor eac76cc012 Fix failure to maintainer-clean jsonpath_gram.h
Oversight in dac048f71e
2022-09-06 12:37:33 +07:00
David Rowley c89b44a68d Fix typo in 16d69ec29
As noted by Justin Pryzby, just I forgot to commit locally before creating
a patch file.

Discussion: https://postgr.es/m/20220901053146.GI31833@telsasoft.com
2022-09-06 15:59:15 +12:00
David Rowley 16d69ec29b Remove buggy and dead code from CreateTriggerFiringOn
Here we remove some dead code from CreateTriggerFiringOn() which was
attempting to find the relevant child partition index corresponding to the
given indexOid.  As it turned out, thanks to -Wshadow=compatible-local,
this code was buggy as the code which was finding the child indexes
assigned those to a shadowed variable that directly went out of scope.
The code which thought it was looking at the List of child indexes was
always referencing an empty List.

On further investigation, this code is dead.  We never call
CreateTriggerFiringOn() passing a valid indexOid in a way that the
function would actually ever execute the code in question.  So, for lack
of a way to test if a fix actually works, let's just remove the dead code
instead.

As a reminder, if there is ever a need to resurrect this code, an Assert()
has been added to remind future feature developers that they might need to
write some code to find the corresponding child index.

Reported-by: Justin Pryzby
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/20220819211824.GX26426@telsasoft.com
2022-09-06 15:51:44 +12:00
John Naylor 8c06a282fe Add missing exceptions to cpluspluscheck
dac048f71 added exceptions to headerscheck but failed to do the
same for cpluspluscheck

Per report from Andres Freund regarding CI
Discussion:https://www.postgresql.org/message-id/20220904205743.y3ntq6ij3aibmxvy%40awork3.anarazel.de
2022-09-06 10:31:22 +07:00
David Rowley 8b26769bc4 Fix an assortment of improper usages of string functions
In a similar effort to f736e188c and 110d81728, fixup various usages of
string functions where a more appropriate function is available and more
fit for purpose.

These changes include:

1. Use cstring_to_text_with_len() instead of cstring_to_text() when
   working with a StringInfoData and the length can easily be obtained.
2. Use appendStringInfoString() instead of appendStringInfo() when no
   formatting is required.
3. Use pstrdup(...) instead of psprintf("%s", ...)
4. Use pstrdup(...) instead of psprintf(...) (with no formatting)
5. Use appendPQExpBufferChar() instead of appendPQExpBufferStr() when the
   length of the string being appended is 1.
6. appendStringInfoChar() instead of appendStringInfo() when no formatting
   is required and string is 1 char long.
7. Use appendPQExpBufferStr(b, .) instead of appendPQExpBuffer(b, "%s", .)
8. Don't use pstrdup when it's fine to just point to the string constant.

I (David) did find other cases of #8 but opted to use #4 instead as I
wasn't certain enough that applying #8 was ok (e.g in hba.c)

Author: Ranier Vilela, David Rowley
Discussion: https://postgr.es/m/CAApHDvo2j2+RJBGhNtUz6BxabWWh2Jx16wMUMWKUjv70Ver1vg@mail.gmail.com
2022-09-06 13:19:44 +12:00
Peter Eisentraut 6bcda4a721 Fix incorrect uses of Datum conversion macros
Since these macros just cast whatever you give them to the designated
output type, and many normal uses also cast the output type further, a
number of incorrect uses go undiscovered.  The fixes in this patch
have been discovered by changing these macros to inline functions,
which is the subject of a future patch.

Reviewed-by: Aleksander Alekseev <aleksander@timescale.com>
Discussion: https://www.postgresql.org/message-id/flat/8528fb7e-0aa2-6b54-85fb-0c0886dbd6ed%40enterprisedb.com
2022-09-05 13:30:44 +02:00
Tomas Vondra 2fe6b2a806 Force parallelism in partition_aggregate
Commit db0d67db2 tweaked sort costing, which however resulted in a
couple plan changes in our regression tests. Most of the new plans were
fine, but partition_aggregate were meant to test parallel plans and the
new plans were serial.

Fix that by lowering parallel_setup_cost to 0, which is enough to switch
to the parallel plan again.

Commit 1349d2790 already made the plans parallel again, but do this
anyway to keep the tests in sync with 15, to make backpatching simpler.

Report and patch by David Rowley.

Author: David Rowley
Reviewed-by: Tomas Vondra
Discussion: https://postgr.es/m/CAApHDvpVFgWzXdtUQkjyOPhNrNvumRi_=ftgS79KeAZ92tnHKQ@mail.gmail.com
2022-09-05 00:09:17 +02:00
John Naylor 92e7b7722d Fix MSVC linker error for specparse.obj
Per buildfarm animals drongo
2022-09-04 18:01:04 +07:00
John Naylor dac048f71e Build all Flex files standalone
The proposed Meson build system will need a way to ignore certain
generated files in order to coexist with the autoconf build system,
and C files generated by Flex which are #include'd into .y files make
this more difficult. In similar vein to 72b1e3a21, arrange for all Flex
C files to compile to their own .o targets.

Reviewed by Andres Freund

Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
Discussion: https://www.postgresql.org/message-id/CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com
2022-09-04 12:09:01 +07:00
John Naylor 80e8450a74 Move private declarations shared between guc.c and guc-file.l to new header
Further preparatory refactoring for compiling guc-file.c standalone.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
Discussion: https://www.postgresql.org/message-id/CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com
2022-09-04 10:45:56 +07:00
John Naylor 1b188ea792 Preparatory refactoring for compiling guc-file.c standalone
Mostly this involves moving ProcessConfigFileInternal() to guc.c
and fixing the shared API to match.

Reviewed by Andres Freund
Discussion: https://www.postgresql.org/message-id/20220810171935.7k5zgnjwqzalzmtm%40awork3.anarazel.de
Discussion: https://www.postgresql.org/message-id/CAFBsxsF8Gc2StS3haXofshHCzqNMRXiSxvQEYGwnFsTmsdwNeg@mail.gmail.com
2022-09-04 10:12:56 +07:00
John Naylor 73b9d051c6 Fix sign-compare warnings arising from port/simd.h
Noted while building an extension using -Wsign-compare.

Per gripe from Pavel Stehule
Discussion: https://www.postgresql.org/message-id/CAFj8pRAagKQHfw71aQbL8PbL0S_360M61V0_vPqJXbpUFvqnRA%40mail.gmail.com
2022-09-04 09:23:57 +07:00
Thomas Munro 932b016300 Fix cache invalidation bug in recovery_prefetch.
XLogPageRead() can retry internally after a pread() system call has
succeeded, in the case of short reads, and page validation failures
while in standby mode (see commit 0668719801).  Due to an oversight in
commit 3f1ce973, these cases could leave stale data in the internal
cache of xlogreader.c without marking it invalid.  The main defense
against stale cached data on failure to read a page was in the error
handling path of the calling function ReadPageInternal(), but that
wasn't quite enough for errors handled internally by XLogPageRead()'s
retry loop if we then exited with XLREAD_WOULDBLOCK.

1.  ReadPageInternal() now marks the cache invalid before calling the
    page_read callback, by setting state->readLen to 0.  It'll be set to
    a non-zero value only after a successful read.  It'll stay valid as
    long as the caller requests data in the cached range.

2.  XLogPageRead() no long performs internal retries while reading
    ahead.  While such retries should work, the general philosophy is
    that we should give up prefetching if anything unusual happens so we
    can handle it when recovery catches up, to reduce the complexity of
    the system.  Let's do that here too.

3.  While here, a new function XLogReaderResetError() improves the
    separation between xlogrecovery.c and xlogreader.c, where the former
    previously clobbered the latter's internal error buffer directly.
    The new function makes this more explicit, and also clears a related
    flag, without which a standby would needlessly retry in the outer
    function.

Thanks to Noah Misch for tracking down the conditions required for a
rare build farm failure in src/bin/pg_ctl/t/003_promote.pl, and
providing a reproducer.

Back-patch to 15.

Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/20220807003627.GA4168930%40rfd.leadboat.com
2022-09-03 13:28:43 +12:00
Tom Lane ff720a597c Fix planner to consider matches to boolean columns in extension indexes.
The planner has to special-case indexes on boolean columns, because
what we need for an indexscan on such a column is a qual of the shape
of "boolvar = pseudoconstant".  For plain bool constants, previous
simplification will have reduced this to "boolvar" or "NOT boolvar",
and we have to reverse that if we want to make an indexqual.  There is
existing code to do so, but it only fires when the index's opfamily
is BOOL_BTREE_FAM_OID or BOOL_HASH_FAM_OID.  Thus extension AMs, or
extension opclasses such as contrib/btree_gin, are out in the cold.

The reason for hard-wiring the set of relevant opfamilies was mostly
to avoid a catalog lookup in a hot code path.  We can improve matters
while not taking much of a performance hit by relying on the
hard-wired set when the opfamily OID is visibly built-in, and only
checking the catalogs when dealing with an extension opfamily.

While here, rename IsBooleanOpfamily to IsBuiltinBooleanOpfamily
to remind future users of that macro of its limitations.  At some
point we might want to make indxpath.c's improved version of the
test globally accessible, but it's not presently needed elsewhere.

Zongliang Quan and Tom Lane

Discussion: https://postgr.es/m/f293b91d-1d46-d386-b6bb-4b06ff5c667b@yeah.net
2022-09-02 17:01:51 -04:00
Peter Eisentraut 1c3aa54502 Fix PL/Perl build on Cygwin
This was broken by b4e936859d.  The
reason why this fixes it are not entirely clear, but it seemed the
best way to get it working again.

Discussion: https://www.postgresql.org/message-id/flat/8c4fcb72-2574-ff7c-4c25-1f032d4a2a57%40enterprisedb.com
2022-09-02 17:56:14 +02:00
Michael Paquier bfb9dfd937 Expand the use of get_dirent_type(), shaving a few calls to stat()/lstat()
Several backend-side loops scanning one or more directories with
ReadDir() (WAL segment recycle/removal in xlog.c, backend-side directory
copy, temporary file removal, configuration file parsing, some logical
decoding logic and some pgtz stuff) already know the type of the entry
being scanned thanks to the dirent structure associated to the entry, on
platforms where we know about DT_REG, DT_DIR and DT_LNK to make the
difference between a regular file, a directory and a symbolic link.

Relying on the direct structure of an entry saves a few system calls to
stat() and lstat() in the loops updated here, shaving some code while on
it.  The logic of the code remains the same, calling stat() or lstat()
depending on if it is necessary to look through symlinks.

Authors: Nathan Bossart, Bharath Rupireddy
Reviewed-by: Andres Freund, Thomas Munro, Michael Paquier
Discussion: https://postgr.es/m/CALj2ACV8n-J-f=yiLUOx2=HrQGPSOZM3nWzyQQvLPcccPXxEdg@mail.gmail.com
2022-09-02 16:58:06 +09:00
John Naylor 0a8de93a48 Speed up lexing of long JSON strings
Use optimized linear search when looking ahead for end quotes,
backslashes, and non-printable characters. This results in nearly 40%
faster JSON parsing on x86-64 when most values are long strings, and
all platforms should see some improvement.

Reviewed by Andres Freund and Nathan Bossart
Discussion: https://www.postgresql.org/message-id/CAFBsxsGhaR2KQ5eisaK%3D6Vm60t%3DaxhD8Ckj1qFoCH1pktZi%2B2w%40mail.gmail.com
Discussion: https://www.postgresql.org/message-id/CAFBsxsESLUyJ5spfOSyPrOvKUEYYNqsBosue9SV1j8ecgNXSKA%40mail.gmail.com
2022-09-02 09:36:22 +07:00
Andres Freund 05519126a0 Move darwin sysroot determination into separate file
The sysroot determination is fairly complex and will soon also be needed when
building with meson. Instead of duplicating the logic, move it to a dedicated
shell script invoked both by configure and meson.

Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com>
Discussion: https://postgr.es/m/2180a97c-c026-1b6c-cec8-d6e499f97017@enterprisedb.com
2022-09-01 16:54:19 -07:00
Andrew Dunstan 2f2b18bd3f Revert SQL/JSON features
The reverts the following and makes some associated cleanups:

    commit f79b803dc: Common SQL/JSON clauses
    commit f4fb45d15: SQL/JSON constructors
    commit 5f0adec25: Make STRING an unreserved_keyword.
    commit 33a377608: IS JSON predicate
    commit 1a36bc9db: SQL/JSON query functions
    commit 606948b05: SQL JSON functions
    commit 49082c2cc: RETURNING clause for JSON() and JSON_SCALAR()
    commit 4e34747c8: JSON_TABLE
    commit fadb48b00: PLAN clauses for JSON_TABLE
    commit 2ef6f11b0: Reduce running time of jsonb_sqljson test
    commit 14d3f24fa: Further improve jsonb_sqljson parallel test
    commit a6baa4bad: Documentation for SQL/JSON features
    commit b46bcf7a4: Improve readability of SQL/JSON documentation.
    commit 112fdb352: Fix finalization for json_objectagg and friends
    commit fcdb35c32: Fix transformJsonBehavior
    commit 4cd8717af: Improve a couple of sql/json error messages
    commit f7a605f63: Small cleanups in SQL/JSON code
    commit 9c3d25e17: Fix JSON_OBJECTAGG uniquefying bug
    commit a79153b7a: Claim SQL standard compliance for SQL/JSON features
    commit a1e7616d6: Rework SQL/JSON documentation
    commit 8d9f9634e: Fix errors in copyfuncs/equalfuncs support for JSON node types.
    commit 3c633f32b: Only allow returning string types or bytea from json_serialize
    commit 67b26703b: expression eval: Fix EEOP_JSON_CONSTRUCTOR and EEOP_JSONEXPR size.

The release notes are also adjusted.

Backpatch to release 15.

Discussion: https://postgr.es/m/40d2c882-bcac-19a9-754d-4299e1d87ac7@postgresql.org
2022-09-01 17:07:14 -04:00
Andres Freund e5484554ba aix: when building with gcc, tell gcc we're building a shared library
Not passing -shared to gcc when building a shared library triggers linking to
the wrong libgcc (libgcc.a instead of libgcc_s.a) and prevents emitting
correct unwind information. It's somewhat surprising that this hasn't caused
known problems so far.

Doing so requires adding path to libgcc to libpath, or linking statically to
libgcc - as the latter increases .so size substantially (for not entirely
obvious reasons), shared linking seems preferrable.  It likely is worth
building executables with -shared-libgcc too, but I've not done that here.

Discussion: https://postgr.es/m/20220820174213.d574qde4ptwdzoqz@awork3.anarazel.de
2022-09-01 11:49:36 -07:00