Commit Graph

48333 Commits

Author SHA1 Message Date
Magnus Hagander 5ba4987a40 Add missing comma in list of SSL versions 2020-11-02 15:20:45 +01:00
Michael Paquier bebad33420 Fix some grammar and typos in comments and docs
The documentation fixes are backpatched down to where they apply.

Author: Justin Pryzby
Discussion: https://postgr.es/m/20201031020801.GD3080@telsasoft.com
Backpatch-through: 9.6
2020-11-02 15:15:25 +09:00
Michael Paquier a8795445bc Extend PageIsVerified() to handle more custom options
This is useful for checks of relation pages without having to load the
pages into the shared buffers, and two cases can make use of that: page
verification in base backups and the online, lock-safe, flavor.

Compatibility is kept with past versions using a routine that calls the
new extended routine with the set of options compatible with the
original version.  Contrary to d401c576, a macro cannot be used as there
may be external code relying on the presence of the original routine.

This is applied down to 11, where this will be used by a follow-up
commit addressing a set of issues with page verification in base
backups.

Extracted from a larger patch by the same author.

Author: Anastasia Lubennikova
Reviewed-by: Michael Paquier, Julien Rouhaud
Discussion: https://postgr.es/m/608f3476-0598-2514-2c03-e05c7d2b0cbd@postgrespro.ru
Backpatch-through: 11
2020-11-02 10:41:30 +09:00
Tom Lane 7d72fd9e6d Avoid null pointer dereference if error result lacks SQLSTATE.
Although error results received from the backend should always have
a SQLSTATE field, ones generated by libpq won't, making this code
vulnerable to a crash after, say, untimely loss of connection.
Noted by Coverity.

Oversight in commit 403a3d91c.  Back-patch to 9.5, as that was.
2020-11-01 11:26:31 -05:00
Michael Paquier 41a033b505 Preserve index data in pg_statistic across REINDEX CONCURRENTLY
Statistics associated to an index got lost after running REINDEX
CONCURRENTLY, while the non-concurrent case preserves these correctly.
The concurrent and non-concurrent operations need to be consistent for
the end-user, and missing statistics would force to wait for a new
analyze to happen, which could take some time depending on the activity
of the existing autovacuum workers.  This issue is fixed by copying any
existing entries in pg_statistic associated to the old index to the new
one.  Note that this copy is already done with the data of the index in
the stats collector.

Reported-by: Fabrízio de Royes Mello
Author: Michael Paquier, Fabrízio de Royes Mello
Reviewed-by: Justin Pryzby
Discussion: https://postgr.es/m/CAFcNs+qpFPmiHd1oTXvcPdvAHicJDA9qBUSujgAhUMJyUMb+SA@mail.gmail.com
Backpatch-through: 12
2020-11-01 21:24:15 +09:00
Noah Misch 741b84e9f7 Reproduce debug_query_string==NULL on parallel workers.
Certain background workers initiate parallel queries while
debug_query_string==NULL, at which point they attempted strlen(NULL) and
died to SIGSEGV.  Older debug_query_string observers allow NULL, so do
likewise in these newer ones.  Back-patch to v11, where commit
7de4a1bcc5 introduced the first of these.

Discussion: https://postgr.es/m/20201014022636.GA1962668@rfd.leadboat.com
2020-10-31 08:44:13 -07:00
Tom Lane 25b587f03a Stabilize timetz test across DST transitions.
The timetz test cases I added in commit a9632830b were unintentionally
sensitive to whether or not DST is active in the PST8PDT time zone.
Thus, they'll start failing this coming weekend, as reported by
Bernhard M. Wiedemann in bug #16689.  Fortunately, DST-awareness is
not significant to the purpose of these test cases, so we can just
force them all to PDT (DST hours) to preserve stability of the
results.

Back-patch to v10, as the prior patch was.

Discussion: https://postgr.es/m/16689-57701daa23b377bf@postgresql.org
2020-10-29 15:28:35 -04:00
Tom Lane cb0982ba97 Use mode "r" for popen() in psql's evaluate_backtick().
In almost all other places, we use plain "r" or "w" mode in popen()
calls (the exceptions being for COPY data).  This one has been
overlooked (possibly because it's buried in a ".l" flex file?),
but it's using PG_BINARY_R.

Kensuke Okamura complained in bug #16688 that we fail to strip \r
when stripping the trailing newline from a backtick result string.
That's true enough, but we'd also fail to convert embedded \r\n
cleanly, which also seems undesirable.  Fixing the popen() mode
seems like the best way to deal with this.

It's been like this for a long time, so back-patch to all supported
branches.

Discussion: https://postgr.es/m/16688-c649c7b69cd7e6f8@postgresql.org
2020-10-28 14:35:53 -04:00
Tom Lane 43330cdd40 Calculate extraUpdatedCols in query rewriter, not parser.
It's unsafe to do this at parse time because addition of generated
columns to a table would not invalidate stored rules containing
UPDATEs on the table ... but there might now be dependent generated
columns that were not there when the rule was made.  This also fixes
an oversight that rewriteTargetView failed to update extraUpdatedCols
when transforming an UPDATE on an updatable view.  (Since the new
calculation is downstream of that, rewriteTargetView doesn't actually
need to do anything; but before, there was a demonstrable bug there.)

In v13 and HEAD, this leads to easily-visible bugs because (since
commit c6679e4fc) we won't recalculate generated columns that aren't
listed in extraUpdatedCols.  In v12 this bitmap is mostly just used
for trigger-firing decisions, so you'd only notice a problem if a
trigger cared whether a generated column had been updated.

I'd complained about this back in May, but then forgot about it
until bug #16671 from Michael Paul Killian revived the issue.

Back-patch to v12 where this field was introduced.  If existing
stored rules contain any extraUpdatedCols values, they'll be
ignored because the rewriter will overwrite them, so the bug will
be fixed even for existing rules.  (But note that if someone were
to update to 13.1 or 12.5, store some rules with UPDATEs on tables
having generated columns, and then downgrade to a prior minor version,
they might observe issues similar to what this patch fixes.  That
seems unlikely enough to not be worth going to a lot of effort to fix.)

Discussion: https://postgr.es/m/10206.1588964727@sss.pgh.pa.us
Discussion: https://postgr.es/m/16671-2fa55851859fb166@postgresql.org
2020-10-28 13:47:02 -04:00
Tom Lane 93f726c04f Fix use-after-free bug with event triggers and ALTER TABLE.
EventTriggerAlterTableEnd neglected to make sure that it built its
output list in the right context.  In simple cases this was masked
because the function is called in PortalContext which will be
sufficiently long-lived anyway; but that doesn't make it not a bug.
Commit ced138e8c fixed this in HEAD and v13, but mistakenly chose
not to back-patch further.  Back-patch the same code change all
the way (I didn't bother with the test case though, as it would
prove nothing in pre-v13 branches).

Per report from Arseny Sher.
Original fix by Jehan-Guillaume de Rorthais.

Discussion: https://postgr.es/m/877drcyprb.fsf@ars-thinkpad
Discussion: https://postgr.es/m/20200902193715.6e0269d4@firost
2020-10-27 15:37:13 -04:00
Bruce Momjian 197014baa4 Makefile comment: remove reference to tools/thread/thread_test
You can't compile thread_test alone anymore, and the location moved too.

Reported-by: Tom Lane

Discussion: https://postgr.es/m/1062278.1603819969@sss.pgh.pa.us

Backpatch-through: 9.5
2020-10-27 14:00:48 -04:00
Alvaro Herrera 9f829aeb36
pg_dump: Lock all relations, not just plain tables
Now that LOCK TABLE can take any relation type, acquire lock on all
relations that are to be dumped.  This prevents schema changes or
deadlock errors that could cause a dump to fail after expending much
effort.  The server is tested to have the capability and the feature
disabled if it doesn't, so that a patched pg_dump doesn't fail when
connecting to an unpatched server.

Backpatch to 9.5.

Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
2020-10-27 14:31:37 -03:00
Alvaro Herrera 7ffead21a4
Accept relations of any kind in LOCK TABLE
The restriction that only tables and views can be locked by LOCK TABLE
is quite arbitrary, since the underlying mechanism can lock any relation
type.  Drop the restriction so that programs such as pg_dump can lock
all relations they're interested in, preventing schema changes that
could cause a dump to fail after expending much effort.

Backpatch to 9.5.

Author: Álvaro Herrera <alvherre@alvh.no-ip.org>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reported-by: Wells Oliver <wells.oliver@gmail.com>
Discussion: https://postgr.es/m/20201021200659.GA32358@alvherre.pgsql
2020-10-27 13:49:19 -03:00
Bruce Momjian 340bbe1fef docs: remove reference to src/tools/thread
This directory and the ability to build the thread test independently
were removed in commit 8a2121185b.

Reported-by: e.indrupskaya@postgrespro.ru

Discussion: https://postgr.es/m/160379609706.24746.7506163279454026608@wrigleys.postgresql.org

Backpatch-through: 9.5
2020-10-27 12:43:11 -04:00
Bruce Momjian 209e8e4c87 doc: simplify wording of function error affects
Reported-by: bob.henkel@gmail.com

Discussion: https://postgr.es/m/160324449781.693.8298142858847611071@wrigleys.postgresql.org

Backpatch-through: 9.5
2020-10-26 22:38:11 -04:00
Bruce Momjian baad9adfa9 doc: make blooms docs match reality
Parallel execution changed the way bloom queries are executed, so update
the EXPLAIN output, and restructure the docs to be clearer and more
accurate.

Reported-by: Daniel Westermann

Discussion: https://postgr.es/m/ZR0P278MB0122119FAE78721A694C30C8D2340@ZR0P278MB0122.CHEP278.PROD.OUTLOOK.COM

Author: Daniel Westermann and me

Backpatch-through: 9.6
2020-10-26 19:17:05 -04:00
Tom Lane de78c10072 Fix corner case for a BEFORE ROW UPDATE trigger returning OLD.
If the old row has any "missing" attributes that are supposed to
be retrieved from an associated tuple descriptor, the wrong things
happened because the trigger result is shoved directly into an
executor slot that lacks the missing-attribute data.  Notably,
CHECK-constraint verification would incorrectly see those columns
as NULL, and so would RETURNING-list evaluation.

Band-aid around this by forcibly expanding the tuple before passing
it to the trigger function.  (IMO it was a fundamental misdesign to
put the missing-attribute data into tuple constraints, which so
much of the system considers to be optional.  But we're probably
stuck with that now, and will have to continue to apply band-aids
as we find other places with similar issues.)

Back-patch to v12.  v11 would also have the issue, except that
commit 920311ab1 already applied a similar band-aid.  That forced
expansion in more cases than seem really necessary, though, so
this isn't a directly equivalent fix.

Amit Langote, with some cosmetic changes by me

Discussion: https://postgr.es/m/16644-5da7ef98a7ac4545@postgresql.org
2020-10-25 13:57:46 -04:00
David Rowley 2a15eeeddf Fix incorrect parameter name in a function header comment
Author: Zhijie Hou
Discussion: https://postgr.es/m/14cd74ea00204cc8a7ea5d738ac82cd1@G08CNEXMBPEKD05.g08.fujitsu.local
Backpatch-through: 12, where the mistake was introduced
2020-10-25 22:41:29 +13:00
Tom Lane bdc79ddd10 Fix ancient bug in ecpg's pthread_once() emulation for Windows.
We must not set the "done" flag until after we've executed the
initialization function.  Otherwise, other threads can fall through
the initial unlocked test before initialization is really complete.

This has been seen to cause rare failures of ecpg's thread/descriptor
test, and it could presumably cause other sorts of misbehavior in
threaded ECPG-using applications, since ecpglib relies on
pthread_once() in several places.

Diagnosis and patch by me, based on investigation by Alexander Lakhin.
Back-patch to all supported branches (the bug dates to 2007).

Discussion: https://postgr.es/m/16685-d6cd241872c101d3@postgresql.org
2020-10-24 13:12:26 -04:00
Tom Lane 78ccf7f42b Update time zone data files to tzdata release 2020d.
DST law changes in Palestine, with a whopping 120 hours' notice.
Also some historical corrections for Palestine.
2020-10-22 21:24:07 -04:00
Tom Lane f56c42e503 Sync our copy of the timezone library with IANA release tzcode2020d.
There's no functional change at all here, but I'm curious to see
whether this change successfully shuts up Coverity's warning about
a useless strcmp(), which appeared with the previous update.

Discussion: http://mm.icann.org/pipermail/tz/2020-October/029370.html
2020-10-22 21:16:04 -04:00
Tom Lane f656517ecd Fix connection string handling in psql's \connect command.
psql's \connect claims to be able to re-use previous connection
parameters, but in fact it only re-uses the database name, user name,
host name (and possibly hostaddr, depending on version), and port.
This is problematic for assorted use cases.  Notably, pg_dump[all]
emits "\connect databasename" commands which we would like to have
re-use all other parameters.  If such a script is loaded in a psql run
that initially had "-d connstring" with some non-default parameters,
those other parameters would be lost, potentially causing connection
failure.  (Thus, this is the same kind of bug addressed in commits
a45bc8a4f and 8e5793ab6, although the details are much different.)

To fix, redesign do_connect() so that it pulls out all properties
of the old PGconn using PQconninfo(), and then replaces individual
properties in that array.  In the case where we don't wish to re-use
anything, get libpq's default settings using PQconndefaults() and
replace entries in that, so that we don't need different code paths
for the two cases.

This does result in an additional behavioral change for cases where
the original connection parameters allowed multiple hosts, say
"psql -h host1,host2", and the \connect request allows re-use of the
host setting.  Because the previous coding relied on PQhost(), it
would only permit reconnection to the same host originally selected.
Although one can think of scenarios where that's a good thing, there
are others where it is not.  Moreover, that behavior doesn't seem to
meet the principle of least surprise, nor was it documented; nor is
it even clear it was intended, since that coding long pre-dates the
addition of multi-host support to libpq.  Hence, this patch is content
to drop it and re-use the host list as given.

Per Peter Eisentraut's comments on bug #16604.  Back-patch to all
supported branches.

Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-10-21 16:19:01 -04:00
Alvaro Herrera a818000f6a
Use fast checkpoint in PostgresNode::backup()
Should cause tests to be a bit faster
2020-10-21 14:37:25 -03:00
Alvaro Herrera 0e6b6f8c71
Fix ALTER TABLE .. ENABLE/DISABLE TRIGGER recursion
More precisely, correctly handle the ONLY flag indicating not to
recurse.  This was implemented in 86f575948c by recursing in
trigger.c, but that's the wrong place; use ATSimpleRecursion instead,
which behaves properly.  However, because legacy inheritance has never
recursed in that situation, make sure to do that only for new-style
partitioning.

I noticed this problem while testing a fix for another bug in the
vicinity.

This has been wrong all along, so backpatch to 11.

Discussion: https://postgr.es/m/20201016235925.GA29829@alvherre.pgsql
2020-10-20 19:22:09 -03:00
Peter Eisentraut bd0677bb8a Avoid invalid alloc size error in shm_mq
In shm_mq_receive(), a huge payload could trigger an unjustified
"invalid memory alloc request size" error due to the way the buffer
size is increased.

Add error checks (documenting the upper limit) and avoid the error by
limiting the allocation size to MaxAllocSize.

Author: Markus Wanner <markus.wanner@2ndquadrant.com>
Discussion: https://www.postgresql.org/message-id/flat/3bb363e7-ac04-0ac4-9fe8-db1148755bfa%402ndquadrant.com
2020-10-20 14:31:52 +02:00
Tom Lane c6d0b9b166 Fix connection string handling in src/bin/scripts/ programs.
When told to process all databases, clusterdb, reindexdb, and vacuumdb
would reconnect by replacing their --maintenance-db parameter with the
name of the target database.  If that parameter is a connstring (which
has been allowed for a long time, though we failed to document that
before this patch), we'd lose any other options it might specify, for
example SSL or GSS parameters, possibly resulting in failure to connect.
Thus, this is the same bug as commit a45bc8a4f fixed in pg_dump and
pg_restore.  We can fix it in the same way, by using libpq's rules for
handling multiple "dbname" parameters to add the target database name
separately.  I chose to apply the same refactoring approach as in that
patch, with a struct to handle the command line parameters that need to
be passed through to connectDatabase.  (Maybe someday we can unify the
very similar functions here and in pg_dump/pg_restore.)

Per Peter Eisentraut's comments on bug #16604.  Back-patch to all
supported branches.

Discussion: https://postgr.es/m/16604-933f4b8791227b15@postgresql.org
2020-10-19 19:03:46 -04:00
Heikki Linnakangas 7e175c53d9 Misc documentation fixes.
- Misc grammar and punctuation fixes.

- Stylistic cleanup: use spaces between function arguments and JSON fields
  in examples. For example "foo(a,b)" -> "foo(a, b)". Add semicolon after
  last END in a few PL/pgSQL examples that were missing them.

- Make sentence that talked about "..." and ".." operators more clear,
  by avoiding to end the sentence with "..". That makes it look the same
  as "..."

- Fix syntax description for HAVING: HAVING conditions cannot be repeated

Patch by Justin Pryzby, per Yaroslav Schekin's report. Backpatch to all
supported versions, to the extent that the patch applies easily.

Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
2020-10-19 19:29:54 +03:00
Heikki Linnakangas c092b2ee23 Fix TRUNCATE doc: ALTER SEQUENCE RESTART is now transactional.
ALTER SEQUENCE RESTART was made transactional in commit 3d79013b97.
Backpatch to v10, where that was introduced.

Patch by Justin Pryzby, per Yaroslav Schekin's report.

Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
2020-10-19 19:02:57 +03:00
Heikki Linnakangas 660cbe8075 Fix output of tsquery example in docs.
The output for this query changed in commit 4e2477b7b8. Backport to 9.6
like that commit.

Patch by Justin Pryzby, per Yaroslav Schekin's report.

Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
2020-10-19 18:50:56 +03:00
Tom Lane 407580aabb In libpq for Windows, call WSAStartup once and WSACleanup not at all.
The Windows documentation insists that every WSAStartup call should
have a matching WSACleanup call.  However, if that ever had actual
relevance, it wasn't in this century.  Every remotely-modern Windows
kernel is capable of cleaning up when a process exits without doing
that, and must be so to avoid resource leaks in case of a process
crash.  Moreover, Postgres backends have done WSAStartup without
WSACleanup since commit 4cdf51e64 in 2004, and we've never seen any
indication of a problem with that.

libpq's habit of doing WSAStartup during connection start and
WSACleanup during shutdown is also rather inefficient, since a
series of non-overlapping connection requests leads to repeated,
quite expensive DLL unload/reload cycles.  We document a workaround
for that (having the application call WSAStartup for itself), but
that's just a kluge.  It's also worth noting that it's far from
uncommon for applications to exit without doing PQfinish, and
we've not heard reports of trouble from that either.

However, the real reason for acting on this is that recent
experiments by Alexander Lakhin show that calling WSACleanup
during PQfinish is triggering the symptom we occasionally see
that a process using libpq fails to emit expected stdio output.

Therefore, let's change libpq so that it calls WSAStartup only
once per process, during the first connection attempt, and never
calls WSACleanup at all.

While at it, get rid of the only other WSACleanup call in our code
tree, in pg_dump/parallel.c; that presumably is equally useless.

Back-patch of HEAD commit 7d00a6b2d.

Discussion: https://postgr.es/m/ac976d8c-03df-d6b8-025c-15a2de8d9af1@postgrespro.ru
2020-10-19 11:23:51 -04:00
Heikki Linnakangas f0e92bc4b0 Fix doc for full text search distance operator.
Commit 028350f619 changed its behavior from "at most" to "exactly", but
forgot to update the documentation. Backpatch to 9.6.

Patch by Justin Pryzby, per Yaroslav Schekin's report.

Discussion: https://www.postgresql.org/message-id/20201005191922.GE17626%40telsasoft.com
2020-10-19 17:58:59 +03:00
Magnus Hagander 9b27176c3f Update link for pllua
Author: Daniel Gustafsson <daniel@yesql.se>
Discussion: https://postgr.es/m/A05874AE-8771-4C61-A24E-0B6249B8F3C2@yesql.se
2020-10-19 13:48:17 +02:00
David Rowley 77ca44b764 Relax some asserts in merge join costing code
In the planner, it was possible, given an extreme enough case containing a
large number of joins for the number of estimated rows to become infinite.
This could cause problems in initial_cost_mergejoin() where we perform
some calculations based on those row estimates.

A problem case, presented by Onder Kalaci showed an Assert failure from
an Assert checking outerstartsel <= outerendsel.  In his test case this
was effectively NaN <= Inf, which is false.  The NaN outerstartsel came
from multiplying the infinite outer_path_rows by 0.0.

In master, this problem was fixed by a90c950fc, however, that fix was too
invasive for the backbranches.  Here we just relax the Asserts to allow
them to pass.  The worst that appears to happen from this is that we show
NaN cost values and infinite row estimates in EXPLAIN.  add_path() would
have had a hard time doing anything useful with such costs, but that does
not really matter as if the row estimates were even close to accurate,
such plan would not complete this side of the heat death of the universe.

Reported-by: Onder Kalaci
Backpatch: 9.5 to 13
Discussion: https://postgr.es/m/DM6PR21MB1211FF360183BCA901B27F04D80B0@DM6PR21MB1211.namprd21.prod.outlook.com
2020-10-20 00:05:41 +13:00
Michael Paquier 57bdf29dd5 Fix potential memory leak in pgcrypto
When allocating a EVP context, it would have been possible to leak some
memory allocated directly by OpenSSL, that PostgreSQL lost track of if
the initialization of the context allocated failed.  The cleanup can be
done with EVP_MD_CTX_destroy().

Note that EVP APIs exist since OpenSSL 0.9.7 and we have in the tree
equivalent implementations for older versions since ce9b75d (code
removed with 9b7cd59a as of 10~).  However, in 9.5 and 9.6, the existing
code makes use of EVP_MD_CTX_destroy() and EVP_MD_CTX_create() without
an equivalent implementation when building the tree with OpenSSL 0.9.6
or older, meaning that this code is in reality broken with such versions
since it got introduced in e2838c5.  As we have heard no complains about
that, it does not seem worth bothering with in 9.5 and 9.6, so I have
left that out for simplicity.

Author: Michael Paquier
Discussion: https://postgr.es/m/20201015072212.GC2305@paquier.xyz
Backpatch-through: 9.5
2020-10-19 09:37:55 +09:00
Tom Lane 3753e2720a Doc: caution against misuse of 'now' and related datetime literals.
Section 8.5.1.4, which defines these literals, made only a vague
reference to the fact that they might be evaluated too soon to be
safe in non-interactive contexts.  Provide a more explicit caution
against misuse.  Also, generalize the wording in the related tip in
section 9.9.4: while it clearly described this problem, it implied
(or really, stated outright) that the problem only applies to table
DEFAULT clauses.

Per gripe from Tijs van Dam.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/c2LuRv9BiRT3bqIo5mMQiVraEXey_25B4vUn0kDqVqilwOEu_iVF1tbtvLnyQK7yDG3PFaz_GxLLPil2SDkj1MCObNRVaac-7j1dVdFERk8=@thalex.com
2020-10-17 16:02:47 -04:00
Tom Lane b39c94097d Update time zone data files to tzdata release 2020c.
DST law changes in Morocco, Canadian Yukon, Fiji, Macquarie Island,
Casey Station (Antarctica).  Historical corrections for France,
Hungary, Monaco.
2020-10-16 21:53:50 -04:00
Tom Lane 3d13a83079 Sync our copy of the timezone library with IANA release tzcode2020c.
This changes zic's default output format from "-b fat" to "-b slim".
We were already using "slim" in v13/HEAD, so those branches drop
the explicit -b switch in the Makefiles.  Instead, add an explicit
"-b fat" in v12 and before, so that we don't change the output file
format in those branches.  (This is perhaps excessively conservative,
but we decided not to do so in a12079109, and I'll stick with that.)

Other non-cosmetic changes are to drop support for zic's long-obsolete
"-y" switch, and to ensure that strftime() does not change errno
unless it fails.

As usual with tzcode changes, back-patch to all supported branches.
2020-10-16 21:40:16 -04:00
Tom Lane 7004ce7589 Add missing error check in pgcrypto/crypt-md5.c.
In theory, the second px_find_digest call in px_crypt_md5 could fail
even though the first one succeeded, since resource allocation is
required.  Don't skip testing for a failure.  (If one did happen,
the likely result would be a crash rather than clean recovery from
an OOM failure.)

The code's been like this all along, so back-patch to all supported
branches.

Daniel Gustafsson

Discussion: https://postgr.es/m/AA8D6FE9-4AB2-41B4-98CB-AE64BA668C03@yesql.se
2020-10-16 11:59:31 -04:00
Tom Lane 05e6fa8b1b Doc: tweak column widths in synchronous-commit-matrix table.
Commit a97e85f2b caused "exceed the available area" warnings in PDF
builds.  Fine-tune colwidth values to avoid that.

Back-patch to 9.6, like the prior patch.  (This is of dubious value
before v13, since we were far from free of such warnings in older
branches.  But we might as well keep the SGML looking the same in all
branches.)

Per buildfarm.
2020-10-16 11:36:34 -04:00
Andres Freund c835c7ffe2 llvmjit: Work around bug in LLVM 3.9 causing crashes after 72559438f9.
Unfortunately in LLVM 3.9 LLVMGetAttributeCountAtIndex(func, index)
crashes when called with an index that has 0 attributes. Since there's
no way to work around this in the C API, add a small C++ wrapper doing
so.

The only reason this didn't fail before 72559438f9 is that there
always are function attributes...

Author: Andres Freund <andres@anarazel.de>
Discussion: https://postgr.es/m/20201016001254.w2nfj7gd74jmb5in@alap3.anarazel.de
Backpatch: 11-, like 72559438f9
2020-10-15 18:17:01 -07:00
Bruce Momjian 0ab7ca98a1 pg_upgrade: remove C99 compiler req. from commit 3c0471b5fd
This commit required support for inline variable definition, which is
not a requirement.

RELEASE NOTE AUTHOR:  the author of commit 3c0471b5fd
(pg_upgrade/tablespaces) was Justin Pryzby, not me.

Reported-by: Andres Freund

Discussion: https://postgr.es/m/20201016001959.h24fkywfubkv2pc5@alap3.anarazel.de

Backpatch-through: 9.5
2020-10-15 20:37:20 -04:00
Bruce Momjian a106236d82 pg_upgrade: generate check error for left-over new tablespace
Previously, if pg_upgrade failed, and the user recreated the cluster but
did not remove the new cluster tablespace directory, a later pg_upgrade
would fail since the new tablespace directory would already exists.
This adds error reporting for this during check.

Reported-by: Justin Pryzby

Discussion: https://postgr.es/m/20200925005531.GJ23631@telsasoft.com

Backpatch-through: 9.5
2020-10-15 19:33:43 -04:00
Andres Freund c8a2bb0f1a llvmjit: Also copy parameter / return value attributes from template functions.
Previously we only copied the function attributes. That caused problems at
least on s390x: Because we didn't copy the 'zeroext' attribute for
ExecAggTransReparent()'s *IsNull parameters, expressions invoking it didn't
ensure that the upper bytes of the registers were zeroed. In the - relatively
rare - cases where not, ExecAggTransReparent() wrongly ended up in the
newValueIsNull branch due to the register not being zero. Subsequently causing
a crash.

It's quite possible that this would cause problems on other platforms, and in
other places than just ExecAggTransReparent() on s390x.

Thanks to Christoph (and the Debian project) for providing me with access to a
s390x machine, allowing me to debug this.

Reported-By: Christoph Berg
Author: Andres Freund
Discussion: https://postgr.es/m/20201015083246.kie5726xerdt3ael@alap3.anarazel.de
Backpatch: 11-, where JIT was added
2020-10-15 14:31:32 -07:00
Bruce Momjian f915453c5a doc: improve description of synchronous_commit modes
Previously it wasn't clear exactly what each of the synchronous_commit
modes accomplished.  This clarifies that, and adds a table describing it.
Only backpatched through 9.6 since 9.5 doesn't have all the options.

Reported-by: kghost0@gmail.com

Discussion: https://postgr.es/m/159741195522.14321.13812604195366728976@wrigleys.postgresql.org

Backpatch-through: 9.6
2020-10-15 15:15:29 -04:00
Tom Lane 8b53dbada4 In the postmaster, rely on the signal infrastructure to block signals.
POSIX sigaction(2) can be told to block a set of signals while a
signal handler executes.  Make use of that instead of manually
blocking and unblocking signals in the postmaster's signal handlers.
This should save a few cycles, but more importantly it prevents
recursive invocation of signal handlers when many signals arrive in
close succession.  (Assuming that the platform's signal infrastructure
is designed to avoid consuming stack space in that case, but this is
demonstrably true at least on Linux.)  The existing code has been seen
to recurse to the point of stack overflow, either in the postmaster
or in a forked-off child.

Back-patch of commit 9abb2bfc0.  At the time, we'd only seen excess
postmaster stack consumption in the buildfarm; but we now have a
user report of it, and that commit has aged enough to have a fair
amount of confidence that it doesn't break anything.

This still doesn't change anything about the way that it works on
Windows.  Perhaps someone else would like to fix that?

Per bug #16673 from David Geier.  Back-patch to 9.6.  Although
the problem exists in principle before that, we've only seen it
actually materialize in connection with heavy use of parallel
workers, so it doesn't seem necessary to do anything in 9.5;
and the relevant code is different there, too.

Discussion: https://postgr.es/m/16673-d278c604f8e34ec0@postgresql.org
Discussion: https://postgr.es/m/14878.1570820201@sss.pgh.pa.us
2020-10-15 12:50:57 -04:00
Fujii Masao 72b1574090 doc: Mention that toast_tuple_target affects also column marked as Main.
Previously it was documented that toast_tuple_target affected column
marked as only External or Extended. But this description is not correct
and toast_tuple_target affects also column marked as Main.

Back-patch to v11 where toast_tuple_target reloption was introduced.

Author: Shinya Okano
Reviewed-by: Tatsuhito Kasahara, Fujii Masao
Discussion: https://postgr.es/m/93f46e311a67422e89e770d236059817@oss.nttdata.com
2020-10-15 11:07:43 +09:00
Tom Lane c7e2364a5f Paper over regression failures in infinite_recurse() on PPC64 Linux.
Our infinite_recurse() test to verify sane stack-overrun behavior
is affected by a bug of the Linux kernel on PPC64: it will get SIGSEGV
if it receives a signal when the stack depth is (a) over 1MB and
(b) within a few kB of filling the current physical stack allocation.
See https://bugzilla.kernel.org/show_bug.cgi?id=205183.

Since this test is a bit time-consuming and we run it in parallel with
test scripts that do a lot of DDL, it can be expected to get an sinval
catchup interrupt at some point, leading to failure if the timing is
wrong.  This has caused more than 100 buildfarm failures over the
past year or so.

While a fix exists for the kernel bug, it might be years before that
propagates into all production kernels, particularly in some of the
older distros we have in the buildfarm.  For now, let's just back off
and not run this test on Linux PPC64; that loses nothing in test
coverage so far as our own code is concerned.

To do that, split this test into a new script infinite_recurse.sql
and skip the test when the platform name is powerpc64...-linux-gnu.

Back-patch to v12.  Branches before that have not been seen to get
this failure.  No doubt that's because the "errors" test was not
run in parallel with other tests before commit 798070ec0, greatly
reducing the odds of an sinval catchup being necessary.

I also back-patched 3c8553547 into v12, just so the new regression
script would look the same in all branches having it.

Discussion: https://postgr.es/m/3479046.1602607848@sss.pgh.pa.us
Discussion: https://postgr.es/m/20190723162703.GM22387%40telsasoft.com
2020-10-13 17:44:56 -04:00
Tom Lane 12945874eb Fix GiST buffering build to work when there are included columns.
gistRelocateBuildBuffersOnSplit did not get the memo about which
attribute count to use.  This could lead to a crash if there were
included columns and buffering build was chosen.  (Because there
are random page-split decisions elsewhere in GiST index build,
the crashes are not entirely deterministic.)

Back-patch to v12 where GiST gained support for included columns.

Pavel Borisov

Discussion: https://postgr.es/m/CALT9ZEECCV5m7wvxg46PC-7x-EybUmnpupBGhSFMoAAay+r6HQ@mail.gmail.com
2020-10-12 18:01:34 -04:00
Tom Lane f35c117700 Fix memory leak when guc.c decides a setting can't be applied now.
The prohibitValueChange code paths in set_config_option(), which
are executed whenever we re-read a PGC_POSTMASTER variable from
postgresql.conf, neglected to free anything before exiting.  Thus
we'd leak the proposed new value of a PGC_STRING variable, as noted
by BoChen in bug #16666.  For all variable types, if the check hook
creates an "extra" chunk, we'd also leak that.

These are malloc not palloc chunks, so there is no mechanism for
recovering the leaks before process exit.  Fortunately, the values
are typically not very large, meaning you'd have to go through an
awful lot of SIGHUP configuration-reload cycles to make the leakage
amount to anything.  Still, for a long-lived postmaster process it
could potentially be a problem.

Oversight in commit 2594cf0e8.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/16666-2c41a4eec61b03e1@postgresql.org
2020-10-12 13:31:24 -04:00
Tom Lane 8b231d9753 Fix optimization hazard in gram.y's makeOrderedSetArgs(), redux.
It appears that commit cf63c641c, which intended to prevent
misoptimization of the result-building step in makeOrderedSetArgs,
didn't go far enough: buildfarm member hornet's version of xlc
is now optimizing back to the old, broken behavior in which
list_length(directargs) is fetched only after list_concat() has
changed that value.  I'm not entirely convinced whether that's
an undeniable compiler bug or whether it can be justified by a
sufficiently aggressive interpretation of C sequence points.
So let's just change the code to make it harder to misinterpret.

Back-patch to all supported versions, just in case.

Discussion: https://postgr.es/m/1830491.1601944935@sss.pgh.pa.us
2020-10-07 18:42:12 -04:00