Commit Graph

56467 Commits

Author SHA1 Message Date
Thomas Munro 8d9a9f034e All supported systems have locale_t.
locale_t is defined by POSIX.1-2008 and SUSv4, and available on all
targeted systems.  For Windows, win32_port.h redirects to a partial
implementation called _locale_t.  We can now remove a lot of
compile-time tests for HAVE_LOCALE_T, and associated comments and dead
code branches that were needed for older computers.

Since configure + MinGW builds didn't detect locale_t but now we assume
that all systems have it, further inconsistencies among the 3 Windows build
systems were revealed.  With this commit, we no longer define
HAVE_WCSTOMBS_L and HAVE_MBSTOWCS_L on any Windows build system, but
we have logic to deal with that so that replacements are available where
appropriate.

Reviewed-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Tristan Partin <tristan@neon.tech>
Reviewed-by: Peter Eisentraut <peter@eisentraut.org>
Discussion: https://postgr.es/m/CA%2BhUKGLg7_T2GKwZFAkEf0V7vbnur-NfCjZPKZb%3DZfAXSV1ORw%40mail.gmail.com
2023-07-09 11:55:18 +12:00
Andrew Dunstan e9f15bc9db Fix tmpdir issues with commit e213de8e78
Commit e213de8e78 fixed a problem with path lengths to a tempdir on
Windows, but caused problems on at least some Unix systems where the
system tempdir is on a different file system. To work around this, only
used the system temdir for the destination of pg_replslot on Windows,
and otherwise restore the old behaviour.

Backpatch to relase 14 like the previous patch.

Problem exposed by a myriad of buildfarm animals.
2023-07-08 12:34:25 -04:00
Andrew Dunstan e213de8e78 Use shorter location for pg_replslot in pg_basebackup test
The symlink to a longer location tripped up some Windows limit on
buildfarm animal fairywren when running with meson, which uses slightly
longer paths.

Backpatch to release 14 to keep the script in sync. Before that the
script skipped all symlink related tests on Windows.
2023-07-08 11:48:06 -04:00
Peter Eisentraut 6ee01e25b7 Fix Perl warning
Use of uninitialized value $content in concatenation (.) or string
2023-07-08 17:27:18 +02:00
Peter Eisentraut 5edf438eeb Make some indentation in gram.y consistent
Reviewed-by: Amit Langote <amitlangote09@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CA+HiwqE4XTdfb1nW=Ojoy_tQSRhYt-q_kb6i5d4xcKyrLC1Nbg@mail.gmail.com
2023-07-08 15:56:01 +02:00
Jeff Davis 9f6253613e test_extensions: sync meson.build with Makefile.
Makefile does not specify ENCODING, meson.build should not,
either. Oversight in commit 877bf52cff.

Discussion: https://postgr.es/m/ZKYpvvNQdbQuRDGx@paquier.xyz
Reported-by: Michael Paquier
2023-07-07 11:39:19 -07:00
Nathan Bossart 151c22deee Revert MAINTAIN privilege and pg_maintain predefined role.
This reverts the following commits: 4dbdb82513, c2122aae63,
5b1a879943, 9e1e9d6560, ff9618e82a, 60684dd834, 4441fc704d,
and b5d6382496.  A role with the MAINTAIN privilege may be able to
use search_path tricks to escalate privileges to the table owner.
Unfortunately, it is too late in the v16 development cycle to apply
the proposed fix, i.e., restricting search_path when running
maintenance commands.

Bumps catversion.

Reviewed-by: Jeff Davis
Discussion: https://postgr.es/m/E1q7j7Y-000z1H-Hr%40gemulon.postgresql.org
Backpatch-through: 16
2023-07-07 11:25:13 -07:00
Tomas Vondra ec99d6e9c8 Document relaxed HOT for summarizing indexes
Commit 19d8e2308b allowed a weaker check for HOT with summarizing
indexes, but it did not update README.HOT. So do that now.

Patch by Matthias van de Meent, minor changes by me. Backpatch to 16,
where the optimization was introduced.

Author: Matthias van de Meent
Reviewed-by: Tomas Vondra
Backpatch-through: 16
Discussion: https://postgr.es/m/CAEze2WiEOm8V+c9kUeYp2BPhbEc5s473fUf51xNeqvSFGv44Ew@mail.gmail.com
2023-07-07 19:04:53 +02:00
Michael Paquier da98d005cd Fix some comments in ./configure and meson setup files
The comments referring to SSL_CTX_set_cert_cb() in ./configure and
./configure.ac were inconsistent, as `autoreconf -i` would show.  While
on it, fix a typo on a comment related to the same check in
meson.build.

Issue introduced in 8e278b6, that removed support for OpenSSL 1.0.1.
Per offlist report from Thomas Munro.
2023-07-07 13:59:41 +09:00
Daniel Gustafsson f863d82b2f pg_regress: Remove unused variable
The restrictedToken handle was set but never read, so remove the
variable and change to a boolean style check to match other uses
of CreateRestrictedProcess().

Reviewed-by: Nathan Bossart <nathandbossart@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Discussion: https://postgr.es/m/62A63C81-3893-4E3F-A34E-2081DF67074E@yesql.se
2023-07-06 21:21:37 +02:00
Andres Freund f0a94d81e4 Fix type of iterator variable in SH_START_ITERATE
Also add comment to make the reasoning behind the Assert() more explicit (per
Tom).

Reported-by: Ranier Vilela
Discussion: https://postgr.es/m/CAEudQAocXNJ6s1VLz+hMamLAQAiewRoW17OJ6-+9GACKfj6iPQ@mail.gmail.com
Backpatch: 11-
2023-07-06 09:57:28 -07:00
Andrew Dunstan 1fa25ce70f Skip pg_baseback long filename test if path too long on Windows
On Windows, it's sometimes difficult to create a file with a path longer
than 255 chars, and if it can be created it might not be seen by the
archiver. This can be triggered by the test for tar backups with
filenames greater than 100 bytes. So we skip that test if the path would
exceed 255.

Backpatch to all live branches.

Reviewed by Daniel Gustafsson

Discussion: https://postgr.es/m/666ac55b-3400-fb2c-2cea-0281bf36a53c@dunslane.net
2023-07-06 12:27:40 -04:00
Heikki Linnakangas 3142a8845b WAL-log the creation of the init fork of unlogged indexes.
We create a file, so we better WAL-log it. In practice, all the
built-in index AMs and all extensions that I'm aware of write a
metapage to the init fork, which is WAL-logged, and replay of the
metapage implicitly creates the fork too. But if ambuildempty() didn't
write any page, we would miss it.

This can be seen with dummy_index_am. Set up replication, create a
'dummy_index_am' index on an unlogged table, and look at the files
created in the replica: the init fork is not created on the
replica. Dummy_index_am doesn't do anything with the relation files,
however, so it doesn't lead to any user-visible errors.

Backpatch to all supported versions.

Reviewed-by: Robert Haas
Discussion: https://www.postgresql.org/message-id/6e5bbc08-cdfc-b2b3-9e23-1a914b9850a9%40iki.fi
2023-07-06 17:25:29 +03:00
Amit Kapila 69a674a170 Fix code indentation vioaltion introduced in commit cc32ec24fd.
Per buildfarm member koel

Discussion: https://postgr.es/m/CAA4eK1K2Y_iegdXRUNbsghY9+b-4cSOrxYt9T8TtwXkkdWMVJA@mail.gmail.com
2023-07-06 11:49:18 +05:30
Michael Paquier a14354cac0 Add GUC parameter "huge_pages_status"
This is useful to show the allocation state of huge pages when setting
up a server with "huge_pages = try", where allocating huge pages would
be attempted but the server would continue its startup sequence even if
the allocation fails.  The effective status of huge pages is not easily
visible without OS-level tools (or for instance, a lookup at
/proc/N/smaps), and the environments where Postgres runs may not
authorize that.  Like the other GUCs related to huge pages, this works
for Linux and Windows.

This GUC can report as values:
- "on", if huge pages were allocated.
- "off", if huge pages were not allocated.
- "unknown", a special state that could only be seen when using for
example postgres -C because it is only possible to know if the shared
memory allocation worked after we can check for the GUC values, even if
checking a runtime-computed GUC.  This value should never be seen when
querying for the GUC on a running server.  An assertion is added to
check that.

The discussion has also turned around having a new function to grab this
status, but this would have required more tricks for -DEXEC_BACKEND,
something that GUCs already handle.

Noriyoshi Shinoda has initiated the thread that has led to the result of
this commit.

Author: Justin Pryzby
Reviewed-by: Nathan Bossart, Kyotaro Horiguchi, Michael Paquier
Discussion: https://postgr.es/m/TU4PR8401MB1152EBB0D271F827E2E37A01EECC9@TU4PR8401MB1152.NAMPRD84.PROD.OUTLOOK.COM
2023-07-06 14:42:36 +09:00
Michael Paquier cf05113eb0 Add newline at the end of header generated by generate-wait_event_types.pl
The header file wait_event_types.h was generated without a newline at
its end, which was inconsistent with all the other things generated
automatically.

Per offline gripe from Nathan Bossart.
2023-07-06 13:35:50 +09:00
Amit Kapila cc32ec24fd Revert the commits related to allowing page lock to conflict among parallel group members.
This commit reverts the work done by commits 3ba59ccc89 and 72e78d831a.
Those commits were incorrect in asserting that we never acquire any other
heavy-weight lock after acquring page lock other than relation extension
lock. We can acquire a lock on catalogs while doing catalog look up after
acquring page lock.

This won't impact any existing feature but we need to think some other way
to achieve this before parallelizing other write operations or even
improving the parallelism in vacuum (like allowing multiple workers
for an index).

Reported-by: Jaime Casanova
Author: Amit Kapila
Backpatch-through: 13
Discussion: https://postgr.es/m/CAJKUy5jffnRKNvRHKQ0LynRb0RJC-o4P8Ku3x9vGAVLwDBWumQ@mail.gmail.com
2023-07-06 08:52:10 +05:30
Michael Paquier ae6d06f096 Handle \v as a whitespace character in parsers
This commit comes as a continuation of the discussion that has led to
d522b05, as \v was handled inconsistently when parsing array values or
anything going through the parsers, and changing a parser behavior in
stable branches is a scary thing to do.  The parsing of array values now
uses the more central scanner_isspace() and array_isspace() is removed.

As pointing out by Peter Eisentraut, fix a confusing reference to
horizontal space in the parsers with the term "horiz_space".  \f was
included in this set since 3cfdd8f from 2000, but it is not horizontal.
"horiz_space" is renamed to "non_newline_space", to refer to all
whitespace characters except newlines.

The changes impact the parsers for the backend, psql, seg, cube, ecpg
and replication commands.  Note that JSON should not escape \v, as per
RFC 7159, so these are not touched.

Reviewed-by: Peter Eisentraut, Tom Lane
Discussion: https://postgr.es/m/ZJKcjNwWHHvw9ksQ@paquier.xyz
2023-07-06 08:16:24 +09:00
Heikki Linnakangas 4f4d73466d Fix leak of LLVM "fatal-on-oom" section counter.
llvm_release_context() called llvm_enter_fatal_on_oom(), but was missing
the corresponding llvm_leave_fatal_on_oom() call. As a result, if JIT was
used at all, we were almost always in the "fatal-on-oom" state.

It only makes a difference if you use an extension written in C++, and
run out of memory in a C++ 'new' call. In that case, you would get a
PostgreSQL FATAL error, instead of the default behavior of throwing a
C++ exception.

Back-patch to all supported versions.

Reviewed-by: Daniel Gustafsson
Discussion: https://www.postgresql.org/message-id/54b78cca-bc84-dad8-4a7e-5b56f764fab5@iki.fi
2023-07-05 13:13:13 +03:00
Peter Eisentraut 0885390151 pg_waldump: Add test case for notice message
Add a test case for the "first record is after" message.  Also, this
message should really go to stderr, so change to use pg_log_info.

Reviewed-by: Tristen Raab <tristen.raab@highgo.ca>
Discussion: https://www.postgresql.org/message-id/flat/CAAcByaKM7zyJSDkPWv6_%2BapupY4fXXM3q3SRXas9MMNVPUGcsQ%40mail.gmail.com
2023-07-05 10:50:16 +02:00
Peter Eisentraut 96063e2836 pg_waldump: Add more tests
This adds tests for most command-line options and tests for most
rmgrdesc routines.

Based on patch by Dong Wook Lee <sh95119@gmail.com>.

Reviewed-by: Tristen Raab <tristen.raab@highgo.ca>
Discussion: https://www.postgresql.org/message-id/flat/CAAcByaKM7zyJSDkPWv6_%2BapupY4fXXM3q3SRXas9MMNVPUGcsQ%40mail.gmail.com
2023-07-05 10:50:08 +02:00
Heikki Linnakangas 5e8068f04e Change example in pgindent README on "/*-----" comments.
Most, but not all, of our "/*----" style comments have an end-guard
line with dashes at the end of the comment. However, pgindent doesn't
care about the end-guards, so they mostly just waste screen
space. Going forward, let's not require end-guards.

Remove a broken end-guard in a comment in walsender.c that led me to
think about this. Remove the end guard from another comment in
walsender.c for consistency, so that we use the same style in all
comments in the file.

However, we have thousands of existing "/*----" comments the repository,
so it's not worth the code churn to change them all.

Discussion: https://www.postgresql.org/message-id/fb083c91-d490-3b65-25f3-05e9118b6b0d%40iki.fi
2023-07-05 10:02:15 +03:00
Daniel Gustafsson a77d39d5f4 Rename EVT cache hash to make context name unique
BuildEventTriggerCache sets up a context "EventTriggerCache" which
house a hash named "Event Trigger Cache", which in turn creates a
context with the table name.  This generated log output for memory
context dumps like below:

LOG:  level: 2; EventTriggerCache: 8192 total in 1 blocks; 7928 free (4 chunks); 264 used
LOG:  level: 3; Event Trigger Cache: 8192 total in 1 blocks; 2616 free (0 chunks); 5576 used

This renames the hash to ensure that the hash context has a unique
name for easier log reading and debugging.

Discussion: https://postgr.es/m/5EDC969E-CAE3-4CBD-965E-3B8A1294CFA4@yesql.se
2023-07-05 08:53:09 +02:00
Masahiko Sawada 68a59f9e99 pgstat: fix subscription stats entry leak.
Commit 7b64e4b3 taught DropSubscription() to drop stats entry of
subscription that is not associated with a replication slot for apply
worker at DROP SUBSCRIPTION but missed covering the case where the
subscription is not associated with replication slots for both apply
worker and tablesync worker.

Also add a test to verify that the stats for slot-less subscription is
removed at DROP SUBSCRIPTION time.

Backpatch down to 15.

Author: Masahiko Sawada
Reviewed-by: Nathan Bossart, Hayato Kuroda, Melih Mutlu, Amit Kapila
Discussion: https://postgr.es/m/CAD21AoB71zkP7uPT7JDPsZcvp0749ExEQnOJxeNKPDFisHar+w@mail.gmail.com
Backpatch-through: 15
2023-07-05 14:49:46 +09:00
Peter Eisentraut cccdbc5d95 Clean up command argument assembly
Several commands internally assemble command lines to call other
commands.  This includes initdb, pg_dumpall, and pg_regress.  (Also
pg_ctl, but that is different enough that I didn't consider it here.)
This has all evolved a bit organically, with fixed-size buffers, and
various optional command-line arguments being injected with
confusing-looking code, and the spacing between options handled in
inconsistent ways.  Clean all this up a bit to look clearer and be
more easily extensible with new arguments and options.  We start each
command with printfPQExpBuffer(), and then append arguments as
necessary with appendPQExpBuffer().  Also standardize on using
initPQExpBuffer() over createPQExpBuffer() where possible.  pg_regress
uses StringInfo instead of PQExpBuffer, but many of the same ideas
apply.

Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://www.postgresql.org/message-id/flat/16d0beac-a141-e5d3-60e9-323da75f49bf@eisentraut.org
2023-07-05 07:15:23 +02:00
Michael Paquier fa88928470 Generate automatically code and documentation related to wait events
The documentation and the code is generated automatically from a new
file called wait_event_names.txt, formatted in sections dedicated to
each wait event class (Timeout, Lock, IO, etc.) with three tab-separated
fields:
- C symbol in enums
- Format in the system views
- Description in the docs

Using this approach has several advantages, as we have proved to be
rather bad in maintaining this area of the tree across the years:
- The order of each item in the documentation and the code, which should
be alphabetical, has become incorrect multiple times, and the script
generating the code and documentation has a few rules to enforce that,
making the maintenance a no-brainer.
- Some wait events were added to the code, but not documented, so this
cannot be missed now.
- The order of the tables for each wait event class is enforced in the
documentation (the input .txt file does so as well for clarity, though
this is not mandatory).
- Less code, shaving 1.2k lines from the tree, with 1/3 of the savings
coming from the code, the rest from the documentation.

The wait event types "Lock" and "LWLock" still have their own code path
for their code, hence only the documentation is created for them.  These
classes are listed with a special marker called WAIT_EVENT_DOCONLY in
the input file.

Adding a new wait event now requires only an update of
wait_event_names.txt, with "Lock" and "LWLock" treated as exceptions.

This commit has been tested with configure/Makefile, the CI and VPATH
build.  clean, distclean and maintainer-clean were working fine.

Author: Bertrand Drouvot, Michael Paquier
Discussion: https://postgr.es/m/77a86b3a-c4a8-5f5d-69b9-d70bbf2e9b98@gmail.com
2023-07-05 10:53:11 +09:00
Daniel Gustafsson 48efb2302b Fix assertion failure in snapshot building
Clear any potential stale next_phase_at value from the snapshot
builder which otherwise may trip an assertion check ensuring
that there is no next_phase_at value.

This can be reproduced by running 80 concurrent sessions like
the below where $c is a loop counter (assumes there has been
1..$c databases created) :

  echo "
    CREATE TABLE replication_example(id SERIAL PRIMARY KEY,
                                     somedata int,
                                     text varchar(120));
    SELECT 'init' FROM
      pg_create_logical_replication_slot('regression_slot_$c',
                                         'test_decoding');
    SELECT data FROM
      pg_logical_slot_get_changes('regression_slot_$c', NULL,
                                  NULL, 'include-xids', '0',
                                  'skip-empty-xacts', '1');
  " | psql -d regress_$c >>psql.log &

Backpatch down to v16.

Bug: #17695
Author: Masahiko Sawada <sawada.mshk@gmail.com>
Reviewed-by: Alexander Lakhin <exclusion@gmail.com>
Reported-by: bowenshi <zxwsbg@qq.com>
Discussion: https://postgr.es/m/17695-6be9277c9295985f@postgresql.org
Backpatch-through: v16
2023-07-04 17:36:13 +02:00
Heikki Linnakangas 4b4798e138 Ensure that creation of an empty relfile is fsync'd at checkpoint.
If you create a table and don't insert any data into it, the relation file
is never fsync'd. You don't lose data, because an empty table doesn't have
any data to begin with, but if you crash and lose the file, subsequent
operations on the table will fail with "could not open file" error.

To fix, register an fsync request in mdcreate(), like we do for mdwrite().

Per discussion, we probably should also fsync the containing directory
after creating a new file. But that's a separate and much wider issue.

Backpatch to all supported versions.

Reviewed-by: Andres Freund, Thomas Munro
Discussion: https://www.postgresql.org/message-id/d47d8122-415e-425c-d0a2-e0160829702d%40iki.fi
2023-07-04 17:57:03 +03:00
Peter Eisentraut 657f5f223e Remove incidental md5() function uses from several tests
This removes md5() function calls from these test suites:

- bloom
- test_decoding
- isolation
- recovery
- subscription

This covers all remaining test suites where md5() calls were just used
to generate some random data and can be replaced by appropriately
adapted sha256() calls.  This will eventually allow these tests to
pass in OpenSSL FIPS mode (which does not allow MD5 use).  See also
208bf364a9.  Unlike for the main regression tests, I didn't write a
fipshash() wrapper here, because that would have been too repetitive
and wouldn't really save much here.  In some cases it was easier to
remove one layer of indirection by changing column types from text to
bytea.

Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://www.postgresql.org/message-id/flat/f9b480b5-e473-d2d1-223a-4b9db30a229a@eisentraut.org
2023-07-04 14:31:57 +02:00
David Rowley 625d5b3ca0 Allow Incremental Sorts on GiST and SP-GiST indexes
Previously an "amcanorderbyop" index would only be used when the index
could provide sorted results which satisfied all query_pathkeys.  Here
we relax this so that we also allow these indexes to be considered by the
planner when they only provide partially sorted results.  This allows the
planner to later consider making use of an Incremental Sort to satisfy the
remaining pathkeys.  This change is particularly useful for KNN-type
queries which contain a LIMIT clause and an additional ORDER BY clause for
a non-indexed column.

Author: Miroslav Bendik
Reviewed-by: Richard Guo, David Rowley
Discussion: https://postgr.es/m/CAPoEpV0QYDtzjwamwWUBqyWpaCVbJV2d6qOD7Uy09bWn47PJtw%40mail.gmail.com
2023-07-04 23:08:52 +12:00
Michael Paquier 28b5726561 libpq: Add support for Close on portals and statements
The following routines are added to libpq:
PGresult *PQclosePrepared(PGconn *conn, const char *stmt);
PGresult *PQclosePortal(PGconn *conn, const char *portal);
int PQsendClosePrepared(PGconn *conn, const char *stmt);
int PQsendClosePortal(PGconn *conn, const char *portal);

The "send" routines are non-blocking versions of the two others.

Close messages are part of the protocol but they did not have a libpq
implementation.  And, having these routines is for instance useful with
connection poolers as these can detect more easily Close messages
than DEALLOCATE queries.

The implementation takes advantage of what the Describe routines rely on
for portals and statements.  Some regression tests are added in
libpq_pipeline, for the four new routines, by closing portals and
statements created already by the tests.

Author: Jelte Fennema
Reviewed-by: Jian He, Michael Paquier
Discussion: https://postgr.es/m/CAGECzQTb4xFAopAVokudB+L62Kt44mNAL4Z9zZ7UTrs1TRFvWA@mail.gmail.com
2023-07-04 14:48:10 +09:00
Thomas Munro 03f80daac8 Re-bin segment when memory pages are freed.
It's OK to be lazy about re-binning memory segments when allocating,
because that can only leave segments in a bin that's too high.  We'll
search higher bins if necessary while allocating next time, and
also eventually re-bin, so no memory can become unreachable that way.

However, when freeing memory, the largest contiguous range of free pages
might go up, so we should re-bin eagerly to make sure we don't leave the
segment in a bin that is too low for get_best_segment() to find.

The re-binning code is moved into a function of its own, so it can be
called whenever free pages are returned to the segment's free page map.

Back-patch to all supported releases.

Author: Dongming Liu <ldming101@gmail.com>
Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier version)
Reviewed-by: Thomas Munro <thomas.munro@gmail.com>
Discussion: https://postgr.es/m/CAL1p7e8LzB2LSeAXo2pXCW4%2BRya9s0sJ3G_ReKOU%3DAjSUWjHWQ%40mail.gmail.com
2023-07-04 15:16:47 +12:00
David Rowley a8c09daa8b Remove trailing zero words from Bitmapsets
Prior to this, Bitmapsets could contain trailing words which had no
members set.  Many places in bitmapset.c had to loop over trailing words
to check for empty words.  If we ensure we always remove these trailing
zero words then we can perform various optimizations such as fast
pathing bms_is_subset to return false when 'a' has more words than 'b'.
A similar optimization is possible in bms_equal.  Both of these together
can yield quite significant performance increases in the query planner
when querying a partitioned table with around 100 or more partitions.

While we're at it, since the minimum number of words a Bitmapset can
contain is 1, we can make use of do/while loops instead of for loops
when looping over all words in a set.  This means checking the loop
condition 1 less time, which for single-word sets cuts the loop
condition checks in half.

Author: David Rowley
Reviewed-by: Yuya Watari
Discussion: https://postgr.es/m/CAApHDvr5O41MuUjw0DQKqmAnv7QqvmLqXReEd5o4nXTzWp8-+w@mail.gmail.com
2023-07-04 12:34:48 +12:00
Michael Paquier 44e73a498c Fix regression tests of unaccent to work without UTF8 support
The tests of unaccent rely on UTF8 characters, and unlike any other test
suite in the tree (fuzzystrmatch, citext, hstore, etc.), they would fail
if run on a database that does not support UTF8 encoding.

This commit fixes the tests of unaccent so as these are skipped when run
on a database without UTF8 support, using the same method as the other
test suits based on \if, getdatabaseencoding() and an alternate output
file.

This has been broken for a long time, but nobody has complained about
that either, so no backpatch is done.  This can be reproduced with
something like REGRESS_OPTS="--no-locale --encoding=sql_ascii", for
instance.  To defend against that, this module's Makefile and
meson.build enforced a UTF8 encoding without locales, but it did not
offer protection for options given by REGRESS_OPTS.  This switch makes
this regression test suite more consistent with all the others, as
well.

Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/ZIq1HUnIV2ksW85x@paquier.xyz
2023-07-04 08:05:00 +09:00
Michael Paquier 72df1d3f1b Show information about flex (version and command path) in meson output
Like bison, this is useful to know which version and command of flex are
detected by meson.  For example, this was useful for me when using
win_flex.exe on Windows.

Reviewed-by: Peter Eisentraut
Discussion: https://postgr.es/m/ZDT3FbmrjBuBthmR@paquier.xyz
2023-07-04 07:25:34 +09:00
Nathan Bossart 957845789b Increase size of bgw_library_name.
This commit increases the size of the bgw_library_name member of
the BackgroundWorker struct from BGW_MAXLEN (96) bytes to MAXPGPATH
(default of 1024) bytes so that it can store longer file names
(e.g., absolute paths).

Author: Yurii Rashkovskii
Reviewed-by: Daniel Gustafsson, Aleksander Alekseev
Discussion: https://postgr.es/m/CA%2BRLCQyjFV5Y8tG5QgUb6gjteL4S3p%2B1gcyqWTqigyM93WZ9Pg%40mail.gmail.com
2023-07-03 15:02:16 -07:00
Thomas Munro 126552c85c Fix race in SSI interaction with gin fast path.
The ginfast.c code previously checked for conflicts in before locking
the relevant buffer, leaving a window where a RW conflict could be
missed.  Re-order.

There was also a place where buffer ID and block number were confused
while trying to predicate-lock a page, noted by visual inspection.

Back-patch to all supported releases.  Fixes one more problem discovered
with the reproducer from bug #17949, in this case when Dmitry tried
other index types.

Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com>
Reported-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-04 09:07:31 +12:00
Thomas Munro bcc93a389c Fix race in SSI interaction with bitmap heap scan.
When performing a bitmap heap scan, we don't want to miss concurrent
writes that occurred after we observed the heap's rs_nblocks, but before
we took predicate locks on index pages.  Therefore, we can't skip
fetching any heap tuples that are referenced by the index, because we
need to test them all with CheckForSerializableConflictOut().  The
old optimization that would ignore any references to blocks >=
rs_nblocks gets in the way of that requirement, because it means that
concurrent writes in that window are ignored.

Removing that optimization shouldn't affect correctness at any isolation
level, because any new tuples shouldn't be visible to an MVCC snapshot.
There also shouldn't be any error-causing references to heap blocks past
the end, because we should have held at least an AccessShareLock on the
table before the index scan.  It can't get smaller while our transaction
is running.  For now, though, we'll keep the optimization at lower
levels to avoid making unnecessary changes in a bug fix.

Back-patch to all supported releases.  In release 11, the code is in a
different place but not fundamentally different.  Fixes one aspect of
bug #17949.

Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-04 09:07:31 +12:00
Thomas Munro f9b7fc651a Fix race in SSI interaction with empty btrees.
When predicate-locking btrees, we have a special case for completely
empty btrees, since there is no page to lock.  This was racy, because,
without buffer lock held, a matching key could be inserted between the
_bt_search() and the PredicateLockRelation() calls.

Fix, by rechecking _bt_search() after taking the relation-level SIREAD
lock, if using SERIALIZABLE isolation and an empty btree is discovered.

Back-patch to all supported releases.  Fixes one aspect of bug #17949.

Reported-by: Artem Anisimov <artem.anisimov.255@gmail.com>
Reviewed-by: Dmitry Dolgov <9erthalion6@gmail.com>
Reviewed-by: Heikki Linnakangas <hlinnaka@iki.fi>
Discussion: https://postgr.es/m/17949-a0f17035294a55e2%40postgresql.org
2023-07-04 09:07:31 +12:00
Nathan Bossart 562bee0fc1 Don't truncate database and user names in startup packets.
Unlike commands such as CREATE DATABASE, ProcessStartupPacket()
does not perform multibyte-aware truncation of overlength names.
This means that connection attempts might fail even if the user
provides the same overlength names that were used in CREATE
DATABASE, CREATE ROLE, etc.  Ideally, we'd do the same multibyte-
aware truncation in both code paths, but it doesn't seem worth the
added complexity of trying to discover the encoding of the names.
Instead, let's simply skip truncating the names in the startup
packet and let the user/database lookup fail later on.  With this
change, users must provide the exact names stored in the catalogs,
even if the names were truncated.

This reverts commit d18c1d1f51.

Author: Bertrand Drouvot
Reviewed-by: Kyotaro Horiguchi, Tom Lane
Discussion: https://postgr.es/m/07436793-1426-29b2-f924-db7422a05fb7%40gmail.com
2023-07-03 13:18:05 -07:00
Tomas Vondra 29cf61ade3 Consider fillfactor when estimating relation size
When table_block_relation_estimate_size() estimated the number of tuples
in a relation without statistics (e.g. right after load), it did not
consider fillfactor when calculating density. With non-default
fillfactor values, this may result in significant overestimate of the
number of tuples - up to 10x with the minimum 10% fillfactor. This may
have unexpected consequences, e.g. when creating hash indexes.

This considers the current fillfactor value in the "no statistics" code
path.  If the fillfactor changes after loading data into the table, the
estimate may be off. But that seems much less likely than changing the
fillfactor before the data load.

Reviewed-by: Corey Huinker, Peter Eisentraut
Discussion: https://postgr.es/m/cf154ef9-6bac-d268-b735-67a3443debba@enterprisedb.com
2023-07-03 18:55:31 +02:00
Tomas Vondra 087a933b21 Remove expensive test of postgres_fdw batch inserts
The test inserted 70k rows into a foreign table, in order to verify
correct behavior with more than 65535 parameters, and was added in
response to a bug report.

However, this is rather expensive, especially when running the tests
under valgrind, CLOBBER_CACHE_ALWAYS etc. It doesn't seem worth it to
keep running the test, so remove it from all branches (14+).

Backpatch-through: 14
Discussion: https://postgr.es/m/2131017.1623451468@sss.pgh.pa.us
2023-07-03 18:16:58 +02:00
Andrew Dunstan 9436535b08 Improve pg_basebackup long file name test Windows robustness
Creation of a file with a very long name can create problems on Windows
due to its file path limits. Work around that by creating the file via a
symlink with a shorter name.

Error displayed by buildfarm animal fairywren.o

Backpatch to all live branches
2023-07-03 10:06:26 -04:00
Tomas Vondra a4cfeeca5a Fix code indentation violations
Commits ce5aaea8cd, 2b8b2852bb and 28d03feac3 violated the expected code
indentation rules, upsetting the new buildfarm member "koel."

Discussion: https://postgr.es/m/ZKIU4mhWpgJOM0W0%40paquier.xyz
2023-07-03 12:47:49 +02:00
Peter Eisentraut 6d56c501a7 A minor simplification for List manipulation
Fix one place that was using lfirst(list_head(list)) by using linitial(list)
instead.  They are equivalent but the latter is simpler.  We did the same in
9d299a49.

Author: Richard Guo <guofenglinux@gmail.com>
Discussion: https://www.postgresql.org/message-id/flat/CAMbWs49dJnpezDQDDxCPKq7+=_3NyqLqGqnhqCjd+dYe4MS15w@mail.gmail.com
2023-07-03 11:39:03 +02:00
Heikki Linnakangas dc6070bf5d Update PG_CACHE_LINE_SIZE description.
PG_CACHE_LINE_SIZE was originally only used in xlog.c, but this hasn't
been true for a very long time and is now wildly used, so modify its
description to not mention any explicit source code file.

Author: Julien Rouhaud
Discussion: https://www.postgresql.org/message-id/20230701074936.p3qcssl4t7murt2q@jrouhaud
2023-07-03 11:56:30 +03:00
Peter Eisentraut 9945750ae0 cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS
See also: 54100f5c6052404f68de9ce7310ceb61f1c2918f

Author: Justin Pryzby <pryzby@telsasoft.com>
Discussion: https://www.postgresql.org/message-id/flat/20220528153741.GK19626@telsasoft.com
2023-07-03 10:35:48 +02:00
Peter Eisentraut c69bdf837f Take pg_attribute out of VacAttrStats
The VacAttrStats structure contained the whole Form_pg_attribute for a
column, but it actually only needs attstattarget from there.  So
remove the Form_pg_attribute field and make a separate field for
attstattarget.  This simplifies some code for extended statistics that
doesn't deal with a column but an expression, which had to fake up
pg_attribute rows to satisfy internal APIs.  Also, we can remove some
comments that essentially said "don't look at pg_attribute directly".

Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/d6069765-5971-04d3-c10d-e4f7b2e9c459%40eisentraut.org
2023-07-03 07:18:57 +02:00
Peter Eisentraut 7a7f60aef8 Add macro for maximum statistics target
The number of places where 10000 was hardcoded had grown a bit beyond
the comfort level.  Introduce a macro MAX_STATISTICS_TARGET instead.

Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/d6069765-5971-04d3-c10d-e4f7b2e9c459%40eisentraut.org
2023-07-03 07:18:57 +02:00
Peter Eisentraut 3ee2f25d21 Change type of pg_statistic_ext.stxstattarget
Change from int32 to int16, to match attstattarget (changed in
90189eefc1).

Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://www.postgresql.org/message-id/flat/d6069765-5971-04d3-c10d-e4f7b2e9c459%40eisentraut.org
2023-07-03 07:18:57 +02:00