Commit Graph

47620 Commits

Author SHA1 Message Date
Andres Freund 6a04d345fd Don't include utils/array.h from acl.h.
For most uses of acl.h the details of how "Acl" internally looks like
are irrelevant. It might make sense to move a lot of the
implementation details into a separate header at a later point.

The main motivation of this change is to avoid including fmgr.h (via
array.h, which needs it for exposed structs) in a lot of files that
otherwise don't need it. A subsequent commit will remove the fmgr.h
include from a lot of files.

Directly include utils/array.h and utils/expandeddatum.h from the
files that need them, but previously included them indirectly, via
acl.h.

Author: Andres Freund
Discussion: https://postgr.es/m/20190803193733.g3l3x3o42uv4qj7l@alap3.anarazel.de
2019-08-16 10:33:30 -07:00
Andres Freund 0ae2dc4db2 Remove redundant prototypes for SQL callable functions.
These aren't needed after 352a24a1f9. The remaining prototypes are
not defined on the SQL level.

Author: Andres Freund
Discussion: https://postgr.es/m/20190803193733.g3l3x3o42uv4qj7l@alap3.anarazel.de
2019-08-16 10:17:32 -07:00
Etsuro Fujita 076e9d4209 Remove useless bms_free() calls in build_child_join_rel().
These seem to be leftovers from the original partitionwise-join patch,
perhaps.

Discussion: https://postgr.es/m/CAPmGK145YiMTPRnvev1dLz8na_-0aZ=Xyqn8f2QsJFBUTObNow@mail.gmail.com
2019-08-16 14:35:55 +09:00
Tom Lane 1ced082b95 Prevent possible double-free when update trigger returns old tuple.
This is a variant of the problem fixed in commit 25b692568, which
unfortunately we failed to detect at the time.  If an update trigger
returns the "old" tuple, as it's entitled to do, then a subsequent
iteration of the loop in ExecBRUpdateTriggers would have "oldtuple"
equal to "trigtuple" and would fail to notice that it shouldn't
free that.

In addition to fixing the code, extend the test case added by
25b692568 so that it covers multiple-trigger-iterations cases.

This problem does not manifest in v12/HEAD, as a result of the
relevant code having been largely rewritten for slotification.
However, include the test case into v12/HEAD anyway, since this
is clearly an area that someone could break again in future.

Per report from Piotr Gabriel Kosinski.  Back-patch into all
supported branches, since the bug seems quite old.

Diagnosis and code fix by Thomas Munro, test case by me.

Discussion: https://postgr.es/m/CAFMLSdP0rd7LqC3j-H6Fh51FYSt5A10DDh-3=W4PPc4LLUQ8YQ@mail.gmail.com
2019-08-15 20:04:19 -04:00
Tom Lane fe9b7b2fe5 Fix plpgsql to re-look-up composite type names at need.
Commit 4b93f5799 rearranged things in plpgsql to make it cope better with
composite types changing underneath it intra-session.  However, I failed to
consider the case of a composite type being dropped and recreated entirely.
In my defense, the previous coding didn't consider that possibility at all
either --- but it would accidentally work so long as you didn't change the
type's field list, because the built-at-compile-time list of component
variables would then still match the type's new definition.  The new
coding, however, occasionally tries to re-look-up the type by OID, and
then fails to find the dropped type.

To fix this, we need to save the TypeName struct, and then redo the type
OID lookup from that.  Of course that's expensive, so we don't want to do
it every time we need the type OID.  This can be fixed in the same way that
4b93f5799 dealt with changes to composite types' definitions: keep an eye
on the type's typcache entry to see if its tupledesc has been invalidated.
(Perhaps, at some point, this mechanism should be generalized so it can
work for non-composite types too; but for now, plpgsql only tries to
cope with intra-session redefinitions of composites.)

I'm slightly hesitant to back-patch this into v11, because it changes
the contents of struct PLpgSQL_type as well as the signature of
plpgsql_build_datatype(), so in principle it could break code that is
poking into the innards of plpgsql.  However, the only popular extension
of that ilk is pldebugger, and it doesn't seem to be affected.  Since
this is a regression for people who were relying on the old behavior,
it seems worth taking the small risk of causing compatibility issues.

Per bug #15913 from Daniel Fiori.  Back-patch to v11 where 4b93f5799
came in.

Discussion: https://postgr.es/m/15913-a7e112e16dedcffc@postgresql.org
2019-08-15 15:21:47 -04:00
Tom Lane bb5ae8f6c4 Use a hash table to de-duplicate NOTIFY events faster.
Previously, async.c got rid of duplicate notifications by scanning
the list of pending events to compare each one to the proposed new
event.  This works okay for very small numbers of distinct events,
but degrades as O(N^2) for many events.  We can improve matters by
using a hash table to probe for duplicates.  So as not to add a
lot of overhead for the simple cases that the code did handle well
before, create the hash table only once a (sub)transaction has
queued more than 16 distinct notify events.

A downside is that we now have to do per-event work to propagate
a successful subtransaction's notify events up to its parent.
(But this isn't significant unless the subtransaction had many
events, in which case the O(N^2) behavior would have been in
play already, so we still come out ahead.)

We can make some lemonade out of this lemon, though: since we must
examine each event anyway, it's now possible to de-duplicate events
fully, rather than skipping that for events merged up from
subtransactions.  Hence, remove the old weasel wording in notify.sgml
about whether de-duplication happens or not, and adjust the test
case in async-notify.spec that exhibited the old behavior.

While at it, rearrange the definition of struct Notification to make
it more compact and require just one palloc per event, rather than
two or three.  This saves space when there are a lot of events,
in fact more than enough to buy back the space needed for the hash
table.

Patch by me, based on discussions around a different patch
submitted by Filip Rembiałkowski.

Discussion: https://postgr.es/m/17822.1564186806@sss.pgh.pa.us
2019-08-15 12:22:12 -04:00
Tom Lane 45aaaa42fe Doc: improve documentation about postgresql.auto.conf.
Clarify what external tools can do to this file, and add a bit
of detail about what ALTER SYSTEM itself does.

Discussion: https://postgr.es/m/aed6cc9f-98f3-2693-ac81-52bb0052307e@2ndquadrant.com
2019-08-15 11:14:26 -04:00
Tom Lane f1bf619acd Fix ALTER SYSTEM to cope with duplicate entries in postgresql.auto.conf.
ALTER SYSTEM itself normally won't make duplicate entries (although
up till this patch, it was possible to confuse it by writing case
variants of a GUC's name).  However, if some external tool has appended
entries to the file, that could result in duplicate entries for a single
GUC name.  In such a situation, ALTER SYSTEM did exactly the wrong thing,
because it replaced or removed only the first matching entry, leaving
the later one(s) still there and hence still determining the active value.

This patch fixes that by making ALTER SYSTEM sweep through the file and
remove all matching entries, then (if not ALTER SYSTEM RESET) append the
new setting to the end.  This means entries will be in order of last
setting rather than first setting, but that shouldn't hurt anything.

Also, make the comparisons case-insensitive so that the right things
happen if you do, say, ALTER SYSTEM SET "TimeZone" = 'whatever'.

This has been broken since ALTER SYSTEM was invented, so back-patch
to all supported branches.

Ian Barwick, with minor mods by me

Discussion: https://postgr.es/m/aed6cc9f-98f3-2693-ac81-52bb0052307e@2ndquadrant.com
2019-08-14 15:09:42 -04:00
Peter Geoghegan 9c02cf5661 Remove block number field from nbtree stack.
The initial value of the nbtree stack downlink block number field
recorded during an initial descent of the tree wasn't actually used.
Both _bt_getstackbuf() callers overwrote the value with their own value.

Remove the block number field from the stack struct, and add a child
block number argument to _bt_getstackbuf() in its place.  This makes the
overall design of _bt_getstackbuf() clearer.

Author: Peter Geoghegan
Reviewed-By: Anastasia Lubennikova
Discussion: https://postgr.es/m/CAH2-Wzmx+UbXt2YNOUCZ-a04VdXU=S=OHuAuD7Z8uQq-PXTYUg@mail.gmail.com
2019-08-14 11:32:35 -07:00
Peter Eisentraut fded4773eb initdb: Remove obsolete locale handling
The method of passing LC_COLLATE and LC_CTYPE to the backend during
initdb is obsolete as of 61d9674988.
This can all be removed.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://www.postgresql.org/message-id/flat/eeaf2f99-a1a6-8aca-3f43-9ab0b2fb112a%402ndquadrant.com
2019-08-14 06:51:13 +02:00
Michael Paquier 96e7e1bc08 Fix random regression failure in test case "collate.icu.utf8"
This is a fix similar to 2d7d67cc, where slight plan alteration can
cause a random failure of this regression test because of an incorect
tuple ordering, except that this one involves lookups of pg_type.
Similarly to the other case, add ORDER BY clauses to ensure the output
order.

The failure has been seen at least once on buildfarm member skink.

Reported-by: Thomas Munro
Discussion: https://postgr.es/m/CA+hUKGLjR9ZBvhXcr9b-NSBHPw9aRgbjyzGE+kqLsT4vwX+nkQ@mail.gmail.com
Backpatch-through: 12
2019-08-14 13:37:48 +09:00
Peter Geoghegan 68ef887842 Remove obsolete nbtree README commentary.
Commit d2086b08b0 removed almost all cases where nbtree must release a
read buffer lock and acquire a write buffer lock instead, so remaining
cases in which that's still necessary are not notable enough to appear
in the nbtree README.

More importantly, holding on to a buffer pin in cases where nbtree must
trade a read lock for a write lock is very unlikely to save any I/O.
This seems to have been a long overlooked throwback to a time when
nbtree cared about write-ordering dependencies, and performed
synchronous buffer writes.  It hasn't worked that way in many years.
2019-08-13 17:16:44 -07:00
Tom Lane 31d43710fb Un-break pg_dump for pre-8.3 source servers.
Commit 07b39083c inserted an unconditional reference to pg_opfamily,
which of course fails on servers predating that catalog.  Fortunately,
the case it's trying to solve can't occur on such old servers (AFAIK).
Hence, just skip the additional code when the source predates 8.3.

Per bug #15955 from sly.  Back-patch to all supported branches,
like the previous patch.

Discussion: https://postgr.es/m/15955-1daa2e676e903d87@postgresql.org
2019-08-13 16:58:32 -04:00
Peter Geoghegan af0ba49809 Use PageIndexTupleOverwrite() within nbtree.
Use the PageIndexTupleOverwrite() bufpage.c routine within nbtree
instead of deleting a tuple and re-inserting its replacement.  This
makes the intent of affected code slightly clearer.  It also makes
CREATE INDEX slightly faster, since there is no longer a need to shift
every leaf page's line pointer array back and forth during index builds.

Author: Peter Geoghegan, Anastasia Lubennikova
Reviewed-By: Anastasia Lubennikova
Discussion: https://postgr.es/m/CAH2-Wz=Zk=B9+Vwm376WuO7YTjFc2SSskifQm4Nme3RRRPtOSQ@mail.gmail.com
2019-08-13 11:54:26 -07:00
Alvaro Herrera 815ef2f568 Don't constraint-exclude partitioned tables as much
We only need to invoke constraint exclusion on partitioned tables when
they are a partition, and they themselves contain a default partition;
it's not necessary otherwise, and it's expensive, so avoid it.  Also, we
were trying once for each clause separately, but we can do it for all
the clauses at once.

While at it, centralize setting of RelOptInfo->partition_qual instead of
computing it in slightly different ways in different places.

Per complaints from Simon Riggs about 4e85642d935e; reviewed by Yuzuko
Hosoya, Kyotaro Horiguchi.

Author: Amit Langote.  I (Álvaro) again mangled the patch somewhat.
Discussion: https://postgr.es/m/CANP8+j+tMCY=nEcQeqQam85=uopLBtX-2vHiLD2bbp7iQQUKpA@mail.gmail.com
2019-08-13 10:26:04 -04:00
Peter Eisentraut 416c75cf38 Update to DocBook 4.5
This moves us to the latest minor version of DocBook 4.  It requires
no markup changes.
2019-08-13 08:40:17 +02:00
Michael Paquier 66bde49d96 Fix inconsistencies and typos in the tree, take 10
This addresses some issues with unnecessary code comments, fixes various
typos in docs and comments, and removes some orphaned structures and
definitions.

Author: Alexander Lakhin
Discussion: https://postgr.es/m/9aabc775-5494-b372-8bcb-4dfc0bd37c68@gmail.com
2019-08-13 13:53:41 +09:00
Michael Paquier 2d7d67cc74 Fix random regression failure in test case "temp"
This test case could fail because of an incorrect result ordering when
looking up at pg_class entries.  This commit adds an ORDER BY to the
culprit query.  The cause of the failure was likely caused by a plan
switch.  By default, the planner would likely choose an index-only scan
or an index scan, but even a small change in the startup cost could have
caused a bitmap heap scan to be chosen, causing the failure.

While on it, switch some filtering quals to a regular expression as per
an idea of Tom Lane.  As previously shaped, the quals would have
selected any relations whose name begins with "temp".  And that could
cause failures if another test running in parallel began to use similar
relation names.

Per report from buildfarm member anole, though the failure was very
rare.  This test has been introduced by 319a810, so backpatch down to
v10.

Discussion: https://postgr.es/m/20190807132422.GC15695@paquier.xyz
Backpatch-through: 10
2019-08-13 10:55:19 +09:00
Peter Geoghegan 6754fe65a4 amcheck: Skip unlogged relations during recovery.
contrib/amcheck failed to consider the possibility that unlogged
relations will not have any main relation fork files when running in hot
standby mode.  This led to low-level "can't happen" errors that complain
about the absence of a relfilenode file.

To fix, simply skip verification of unlogged index relations during
recovery.  In passing, add a direct check for the presence of a main
fork just before verification proper begins, so that we cleanly verify
the presence of the main relation fork file.

Author: Andrey Borodin, Peter Geoghegan
Reported-By: Andrey Borodin
Diagnosed-By: Andrey Borodin
Discussion: https://postgr.es/m/DA9B33AC-53CB-4643-96D4-7A0BBC037FA1@yandex-team.ru
Backpatch: 10-, where amcheck was introduced.
2019-08-12 15:21:32 -07:00
Tom Lane 03c811a483 Fix planner's test for case-foldable characters in ILIKE with ICU.
As coded, the ICU-collation path in pattern_char_isalpha() failed
to consider regular ASCII letters to be case-varying.  This led to
like_fixed_prefix treating too much of an ILIKE pattern as being a
fixed prefix, so that indexscans derived from an ILIKE clause might
miss entries that they should find.

Per bug #15892 from James Inform.  This is an oversight in the original
ICU patch (commit eccfef81e), so back-patch to v10 where that came in.

Discussion: https://postgr.es/m/15892-e5d2bea3e8a04a1b@postgresql.org
2019-08-12 13:15:47 -04:00
Tom Lane 3c926587b5 Remove EState.es_range_table_array.
Now that list_nth is O(1), there's no good reason to maintain a
separate array of RTE pointers rather than indexing into
estate->es_range_table.  Deleting the array doesn't save all that
much either; but just on cleanliness grounds, it's better not to
have duplicate representations of the identical information.

Discussion: https://postgr.es/m/14960.1565384592@sss.pgh.pa.us
2019-08-12 11:58:35 -04:00
Tom Lane 5ee190f8ec Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input.  Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".

To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input.  (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)

In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it.  We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one.  I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)

Patch by me; thanks to David Rowley for review.

Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 11:20:18 -04:00
Alexander Korotkov 251c8e39bc Fix string comparison in jsonpath
Take into account pg_server_to_any() may return input string "as is".

Reported-by: Andrew Dunstan, Thomas Munro
Discussion: https://postgr.es/m/0ed83a33-d900-466a-880a-70ef456c721f%402ndQuadrant.com
Author: Alexander Korotkov, Thomas Munro
Backpatch-through: 12
2019-08-12 06:26:13 +03:00
Tom Lane b43f7c117e Partially revert "Insert temporary debugging output in regression tests."
This reverts much of commit f03a9ca436,
but leaves the relpages/reltuples probe in select_parallel.sql.
The pg_stat_all_tables probes are unstable enough to be annoying,
and it no longer seems likely that they will teach us anything more
about the underlying problem.  I'd still like some more confirmation
though that the observed plan instability is caused by VACUUM leaving
relpages/reltuples as zero for one of these tables.

Discussion: https://postgr.es/m/CA+hUKG+0CxrKRWRMf5ymN3gm+BECHna2B-q1w8onKBep4HasUw@mail.gmail.com
2019-08-11 18:55:32 -04:00
Alexander Korotkov d54ceb9e17 Adjust string comparison in jsonpath
We have implemented jsonpath string comparison using default database locale.
However, standard requires us to compare Unicode codepoints.  This commit
implements that, but for performance reasons we still use per-byte comparison
for "==" operator.  Thus, for consistency other comparison operators do per-byte
comparison if Unicode codepoints appear to be equal.

In some edge cases, when same Unicode codepoints have different binary
representations in database encoding, we diverge standard to achieve better
performance of "==" operator.  In future to implement strict standard
conformance, we can do normalization of input JSON strings.

Original patch was written by Nikita Glukhov, rewritten by me.

Reported-by: Markus Winand
Discussion: https://postgr.es/m/8B7FA3B4-328D-43D7-95A8-37B8891B8C78%40winand.at
Author: Nikita Glukhov, Alexander Korotkov
Backpatch-through: 12
2019-08-11 22:54:53 +03:00
Tom Lane cabe0f298e Fix "ANALYZE t, t" inside a transaction block.
This failed with either "tuple already updated by self" or "duplicate
key value violates unique constraint", depending on whether the table
had previously been analyzed or not.  The reason is that ANALYZE tried
to insert or update the same pg_statistic rows twice, and there was no
CommandCounterIncrement between.  So add one.  The same case works fine
outside a transaction block, because then there's a whole transaction
boundary between, as a consequence of the way VACUUM works.

This issue has been latent all along, but the problem was unreachable
before commit 11d8d72c2 added the ability to specify multiple tables
in ANALYZE.  We could, perhaps, alternatively fix it by adding code to
de-duplicate the list of VacuumRelations --- but that would add a
lot of overhead to work around dumb commands, so it's not attractive.

Per bug #15946 from Yaroslav Schekin.  Back-patch to v11.

(Note: in v11 I also back-patched the test added by commit 23224563d;
otherwise the problem doesn't manifest in the test I added, because
"vactst" is empty when the tests for multiple ANALYZE targets are
reached.  That seems like not a very good thing anyway, so I did this
rather than rethinking the choice of test case.)

Discussion: https://postgr.es/m/15946-5c7570a2884a26cf@postgresql.org
2019-08-10 11:30:11 -04:00
Peter Geoghegan d8cd68c8d4 Rename tuplesort.c's SortTuple.tupindex field.
Rename the "tupindex" field from tuplesort.c's SortTuple struct to
"srctape", since it can only ever be used to store a source/input tape
number when merging external sort runs.  This has been the case since
commit 8b304b8b72, which removed replacement selection sort from
tuplesort.c.
2019-08-09 17:06:45 -07:00
Tom Lane 0662eb6219 Fix SIGSEGV in pruning for ScalarArrayOp with constant-null array.
Not much to be said here: commit 9fdb675fc should have checked
constisnull, didn't.

Per report from Piotr Włodarczyk.  Back-patch to v11 where
bug was introduced.

Discussion: https://postgr.es/m/CAP-dhMr+vRpwizEYjUjsiZ1vwqpohTm+3Pbdt6Pr7FEgPq9R0Q@mail.gmail.com
2019-08-09 13:20:28 -04:00
Tom Lane 1661a40505 Cosmetic improvements in setup of planner's per-RTE arrays.
Merge setup_append_rel_array into setup_simple_rel_arrays.  There's no
particularly good reason to keep them separate, and it's inconsistent
with the lack of separation in expand_planner_arrays.  The only apparent
benefit was that the fast path for trivial queries in query_planner()
doesn't need to set up the append_rel_array; but all we're saving there
is an if-test and NULL assignment, which surely ought to be negligible.

Also improve some obsolete comments.

Discussion: https://postgr.es/m/17220.1565301350@sss.pgh.pa.us
2019-08-09 12:33:43 -04:00
Michael Paquier b8f2da0ac5 Refactor logic to remove trailing CR/LF characters from strings
b654714 has reworked the way trailing CR/LF characters are removed from
strings.  This commit introduces a new routine in common/string.c and
refactors the code so as the logic is in a single place, mostly.

Author: Michael Paquier
Reviewed-by: Bruce Momjian
Discussion: https://postgr.es/m/20190801031820.GF29334@paquier.xyz
2019-08-09 11:05:14 +09:00
Peter Geoghegan 28b901f73a Update obsolete tuplesort READTUP() comment.
READTUP() routines do not and cannot use the resettable "tuplecontext"
memory context, since it is deleted when merging begins.  Update an
obsolete comment that claimed otherwise.  This was an oversight in
commit e94568ecc1.

In passing, fix an unrelated tuplesort typo.
2019-08-08 13:20:44 -07:00
Alvaro Herrera 956451e8bc Clarify the default partition's role
Reviewed by Tom Lane and Amit Langote
Discussion: https://postgr.es/m/20190806222735.GA9535@alvherre.pgsql
2019-08-08 16:05:59 -04:00
Andrew Dunstan 15077ab63f Fix certificate subjects in ldap test
openssl doesn't like lower case subject attribute names. Error observed
in buildfarm results.

Backpatch to release 11.
2019-08-08 15:00:46 -04:00
Peter Eisentraut 43211c2a02 initdb: Use varargs macro for PG_CMD_PRINTF
I left PG_CMD_PUTS around even though it could be handled by
PG_CMD_PRINTF since PG_CMD_PUTS is sometimes called with non-literal
arguments, and so that would create a potential problem if such a
string contained percent signs.

Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
2019-08-08 08:47:55 +02:00
Tom Lane e94cd0a8eb Doc: document permissions required for ANALYZE.
VACUUM's reference page had this text, but ANALYZE's didn't.  That's
a clear oversight given that section 5.7 explicitly delegates the
responsibility to define permissions requirements to the individual
commands' man pages.

Per gripe from Isaac Morland.  Back-patch to all supported branches.

Discussion: https://postgr.es/m/CAMsGm5fp3oBUs-2iRfii0iEO=fZuJALVyM2zJLhNTjG34gpAVQ@mail.gmail.com
2019-08-07 18:09:28 -04:00
Alvaro Herrera e1f4c481b9 Remove unnecessary #include <limits.h>
This include was probably copied from tuplestore.c, but it's not needed.

Extracted from a larger patch submitted by vignesh C <vignesh21@gmail.com>

Discussion: https://postgr.es/m/CALDaNm1B9naPDTm3ox1m_yZvOm3KA5S4kZQSWWAeLHAQ=3gV1Q@mail.gmail.com
2019-08-07 16:55:31 -04:00
Alvaro Herrera 12afc7145c Add comment on no default partition with hash partitioning
Discussion: https://postgr.es/m/20190806222735.GA9535@alvherre.pgsql
2019-08-07 12:27:47 -04:00
Alvaro Herrera 4e85642d93 Apply constraint exclusion more generally in partitioning
We were applying constraint exclusion on the partition constraint when
generating pruning steps for a clause, but only for the rather
restricted situation of them being boolean OR operators; however it is
possible to have differently shaped clauses that also benefit from
constraint exclusion.  This applies particularly to the default
partition since their constraints are in essence a long list of OR'ed
subclauses ... but it applies to other cases too.  So in certain cases
we're scanning partitions that we don't need to.

Remove the specialized code in OR clauses, and add a generally
applicable test of the clause refuting the partition constraint; mark
the whole pruning operation as contradictory if it hits.

This has the unwanted side-effect of testing some (most? all?)
constraints more than once if constraint_exclusion=on.  That seems
unavoidable as far as I can tell without some additional work, but
that's not the recommended setting for that parameter anyway.
However, because this imposes additional processing cost for all
queries using partitioned tables, I decided not to backpatch this
change.

Author: Amit Langote, Yuzuko Hosoya, Álvaro Herrera
Reviewers: Shawn Wang, Thibaut Madeleine, Yoshikazu Imai, Kyotaro
Horiguchi; they were also uncredited reviewers for commit 489247b0e6.
Discussion: https://postgr.es/m/9bb31dfe-b0d0-53f3-3ea6-e64b811424cf@lab.ntt.co.jp
2019-08-07 12:21:54 -04:00
Alexander Korotkov 1f33f211bc Fix some typos in jsonpath documentation
Discussion: https://postgr.es/m/8B7FA3B4-328D-43D7-95A8-37B8891B8C78%40winand.at
Author: Markus Winand
Backpatch-through: 12
2019-08-07 16:06:45 +03:00
Etsuro Fujita 68343b4ad7 Fix typos in comments. 2019-08-07 19:05:17 +09:00
Heikki Linnakangas 1169fcf129 Fix predicate-locking of HOT updated rows.
In serializable mode, heap_hot_search_buffer() incorrectly acquired a
predicate lock on the root tuple, not the returned tuple that satisfied
the visibility checks. As explained in README-SSI, the predicate lock does
not need to be copied or extended to other tuple versions, but for that to
work, the correct, visible, tuple version must be locked in the first
place.

The original SSI commit had this bug in it, but it was fixed back in 2013,
in commit 81fbbfe335. But unfortunately, it was reintroduced a few months
later in commit b89e151054. Wising up from that, add a regression test
to cover this, so that it doesn't get reintroduced again. Also, move the
code that sets 't_self', so that it happens at the same time that the
other HeapTuple fields are set, to make it more clear that all the code in
the loop operate on the "current" tuple in the chain, not the root tuple.

Bug spotted by Andres Freund, analysis and original fix by Thomas Munro,
test case and some additional changes to the fix by Heikki Linnakangas.
Backpatch to all supported versions (9.4).

Discussion: https://www.postgresql.org/message-id/20190731210630.nqhszuktygwftjty%40alap3.anarazel.de
2019-08-07 12:40:49 +03:00
Michael Paquier 64579be64a Fix some incorrect parsing of time with time zone strings
When parsing a timetz string with a dynamic timezone abbreviation or a
timezone not specified, it was possible to generate incorrect timestamps
based on a date which uses some non-initialized variables if the input
string did not specify fully a date to parse.  This is already checked
when a full timezone spec is included in the input string, but the two
other cases mentioned above missed the same checks.

This gets fixed by generating an error as this input is invalid, or in
short when a date is not fully specified.

Valgrind was complaining about this problem.

Bug: #15910
Author: Alexander Lakhin
Discussion: https://postgr.es/m/15910-2eba5106b9aa0c61@postgresql.org
Backpatch-through: 9.4
2019-08-07 18:16:31 +09:00
Michael Paquier 75c1921cd6 Adjust tuple data lookup logic in multi-insert logical decoding
As of now, logical decoding of a multi-insert has been scanning all
xl_multi_insert_tuple entries only if XLH_INSERT_CONTAINS_NEW_TUPLE was
getting set in the record.  This is not an issue on HEAD as multi-insert
records are not used for system catalogs, but the logical decoding logic
includes all the code necessary to handle that properly, except that the
code missed to iterate correctly over all xl_multi_insert_tuple entries
when the flag is not set.  Hence, when trying to use multi-insert for
system catalogs, an assertion would be triggered.

An upcoming patch is going to make use of multi-insert for system
catalogs, and this fixes the logic to make sure that all entries are
scanned correctly without softening the existing assertions.

Reported-by: Daniel Gustafsson
Author: Michael Paquier
Reviewed-by: Daniel Gustafsson
Discussion: https://postgr.es/m/CBFFD532-C033-49EB-9A5A-F67EAEE9EB0B@yesql.se
2019-08-07 10:28:16 +09:00
Tom Lane efc77cf5f1 Fix intarray's GiST opclasses to not fail for empty arrays with <@.
contrib/intarray considers "arraycol <@ constant-array" to be indexable,
but its GiST opclass code fails to reliably find index entries for empty
array values (which of course should trivially match such queries).
This is because the test condition to see whether we should descend
through a non-leaf node is wrong.

Unfortunately, empty array entries could be anywhere in the index,
as these index opclasses are currently designed.  So there's no way
to fix this except by lobotomizing <@ indexscans to scan the whole
index ... which is what this patch does.  That's pretty unfortunate:
the performance is now actually worse than a seqscan, in most cases.
We'd be better off to remove <@ from the GiST opclasses entirely,
and perhaps a future non-back-patchable patch will do so.

In the meantime, applications whose performance is adversely impacted
have a couple of options.  They could switch to a GIN index, which
doesn't have this bug, or they could replace "arraycol <@ constant-array"
with "arraycol <@ constant-array AND arraycol && constant-array".
That will provide about the same performance as before, and it will find
all non-empty subsets of the given constant-array, which is all that
could reliably be expected of the query before.

While at it, add some more regression test cases to improve code
coverage of contrib/intarray.

In passing, adjust resize_intArrayType so that when it's returning an
empty array, it uses construct_empty_array for that rather than
cowboy hacking on the input array.  While the hack produces an array
that looks valid for most purposes, it isn't bitwise equal to empty
arrays produced by other code paths, which could have subtle odd
effects.  I don't think this code path is performance-critical
enough to justify such shortcuts.  (Back-patch this part only as far
as v11; before commit 01783ac36 we were not careful about this in
other intarray code paths either.)

Back-patch the <@ fixes to all supported versions, since this was
broken from day one.

Patch by me; thanks to Alexander Korotkov for review.

Discussion: https://postgr.es/m/458.1565114141@sss.pgh.pa.us
2019-08-06 18:04:51 -04:00
Tom Lane 4ecd05cb77 Save Kerberos and LDAP daemon logs where the buildfarm can find them.
src/test/kerberos and src/test/ldap try to run private authentication
servers, which of course might fail.  The logs from these servers
were being dropped into the tmp_check/ subdirectory, but they should
be put in tmp_check/log/, because the buildfarm will only capture
log files in that subdirectory.  Without the log output there's
little hope of diagnosing buildfarm failures related to these servers.

Backpatch to v11 where these test suites were added.

Discussion: https://postgr.es/m/16017.1565047605@sss.pgh.pa.us
2019-08-06 17:08:07 -04:00
Michael Paquier 940c8b01b0 Fix typo in pathnode.c
Author: Amit Langote
Discussion: https://postgr.es/m/CA+HiwqFhZ6ABoz-i=JZ5wMMyz-orx4asjR0og9qBtgEwOww6Yg@mail.gmail.com
2019-08-06 18:11:02 +09:00
Peter Geoghegan 98eab30b93 Show specific OID suggestion in unused_oids output.
Commit a6417078 established a new project policy around OID assignment:
new patches are encouraged to choose a random OID in the 8000..9999
range when a manually-assigned OID is required (if multiple OIDs are
required, a consecutive block of OIDs starting from the random point
should be used).  Catalog entries added by committed patches that use
OIDs from this "unstable" range are renumbered after feature freeze.
This practice minimizes OID collisions among concurrently-developed
patches.

Show a specific random OID suggestion when the unused_oids script is
run.  This makes it easy for patch authors to use a random OID from the
unstable range, per the new policy.

Author: Julien Rouhaud, Peter Geoghegan
Reviewed-By: Tom Lane
Discussion: https://postgr.es/m/CAH2-WzkkRs2ScmuBQ7xWi7xzp7fC1B3w0Nt8X+n4rBw5k+Z=zA@mail.gmail.com
2019-08-05 11:47:34 -07:00
Tom Lane 4766dce0dd Fix choice of comparison operators for cross-type hashed subplans.
Commit bf6c614a2 rearranged the lookup of the comparison operators
needed in a hashed subplan, and in so doing, broke the cross-type
case: it caused the original LHS-vs-RHS operator to be used to compare
hash table entries too (which of course are all of the RHS type).
This leads to C functions being passed a Datum that is not of the
type they expect, with the usual hazards of crashes and unauthorized
server memory disclosure.

For the set of hashable cross-type operators present in v11 core
Postgres, this bug is nearly harmless on 64-bit machines, which
may explain why it escaped earlier detection.  But it is a live
security hazard on 32-bit machines; and of course there may be
extensions that add more hashable cross-type operators, which
would increase the risk.

Reported by Andreas Seltenreich.  Back-patch to v11 where the
problem came in.

Security: CVE-2019-10209
2019-08-05 11:20:31 -04:00
Noah Misch ffa2d37e5f Require the schema qualification in pg_temp.type_name(arg).
Commit aa27977fe2 introduced this
restriction for pg_temp.function_name(arg); do likewise for types
created in temporary schemas.  Programs that this breaks should add
"pg_temp." schema qualification or switch to arg::type_name syntax.
Back-patch to 9.4 (all supported versions).

Reviewed by Tom Lane.  Reported by Tom Lane.

Security: CVE-2019-10208
2019-08-05 07:48:41 -07:00
Michael Paquier a76cfba663 Add safeguards in LSN, numeric and float calculation for custom errors
Those data types use parsing and/or calculation wrapper routines which
can generate some generic error messages in the event of a failure.  The
caller of these routines can also pass a pointer variable settable by
the routine to track if an error has happened, letting the caller decide
what to do in the event of an error and what error message to generate.

Those routines have been slacking the initialization of the tracking
flag, which can be confusing when reading the code, so add some
safeguards against calls of these parsing routines which could lead to a
dubious result.

The LSN parsing gains an assertion to make sure that the tracking flag
is set, while numeric and float paths initialize the flag to a saner
state.

Author: Jeevan Ladhe
Reviewed-by: Álvaro Herrera, Michael Paquier
Discussion: https://postgr.es/m/CAOgcT0NOM9oR0Hag_3VpyW0uF3iCU=BDUFSPfk9JrWXRcWQHqw@mail.gmail.com
2019-08-05 15:35:16 +09:00