Commit Graph

45021 Commits

Author SHA1 Message Date
Peter Eisentraut 6b30d1386f Fix whitespace 2018-05-17 23:04:41 -04:00
Andrew Gierth 93f6c6328d doc: fix description of backward_scan
The description of the index property backward_scan was incorrect and
misleading; rectify.

Backpatch to 9.6 where the amutils functionality was introduced.
2018-05-17 21:23:48 +01:00
Bruce Momjian 91ff29f036 doc: fix PG 11 attribution 2018-05-17 14:23:22 -04:00
Tom Lane d1fc750b51 Make numeric power() handle NaNs according to the modern POSIX spec.
In commit 6bdf1303b, we ensured that power()/^ for float8 would honor
the NaN behaviors specified by POSIX standards released in this century,
ie NaN ^ 0 = 1 and 1 ^ NaN = 1.  However, numeric_power() was not
touched and continued to follow the once-common behavior that every
case involving NaN input produces NaN.  For consistency, let's switch
the numeric behavior to the modern spec in the same release that ensures
that behavior for float8.

(Note that while 6bdf1303b was initially back-patched, we later undid
that, concluding that any behavioral change should appear only in v11.)

Discussion: https://postgr.es/m/10898.1526421338@sss.pgh.pa.us
2018-05-17 11:10:50 -04:00
Bruce Momjian b2b82228ee doc: PG 11 release notes adjustments 2018-05-16 20:36:22 -04:00
Tom Lane 2efc924180 Detoast plpgsql variables if they might live across a transaction boundary.
Up to now, it's been safe for plpgsql to store TOAST pointers in its
variables because the ActiveSnapshot for whatever query called the plpgsql
function will surely protect such TOAST values from being vacuumed away,
even if the owning table rows are committed dead.  With the introduction of
procedures, that assumption is no longer good in "non atomic" executions
of plpgsql code.  We adopt the slightly brute-force solution of detoasting
all TOAST pointers at the time they are stored into variables, if we're in
a non-atomic context, just in case the owning row goes away.

Some care is needed to avoid long-term memory leaks, since plpgsql tends
to run with CurrentMemoryContext pointing to its call-lifespan context,
but we shouldn't assume that no memory is leaked by heap_tuple_fetch_attr.
In plpgsql proper, we can do the detoasting work in the "eval_mcontext".

Most of the code thrashing here is due to the need to add this capability
to expandedrecord.c as well as plpgsql proper.  In expandedrecord.c,
we can't assume that the caller's context is short-lived, so make use of
the short-term sub-context that was already invented for checking domain
constraints.  In view of this repurposing, it seems good to rename that
variable and associated code from "domain_check_cxt" to "short_term_cxt".

Peter Eisentraut and Tom Lane

Discussion: https://postgr.es/m/5AC06865.9050005@anastigmatix.net
2018-05-16 14:56:52 -04:00
Tom Lane a11b3bd37f Fix misprocessing of equivalence classes involving record_eq().
canonicalize_ec_expression() is supposed to agree with coerce_type() as to
whether a RelabelType should be inserted to make a subexpression be valid
input for the operators of a given opclass.  However, it did the wrong
thing with named-composite-type inputs to record_eq(): it put in a
RelabelType to RECORDOID, which the parser doesn't.  In some cases this was
harmless because all code paths involving a particular equivalence class
did the same thing, but in other cases this would result in failing to
recognize a composite-type expression as being a member of an equivalence
class that it actually is a member of.  The most obvious bad effect was to
fail to recognize that an index on a composite column could provide the
sort order needed for a mergejoin on that column, as reported by Teodor
Sigaev.  I think there might be other, subtler, cases that result in
misoptimization.  It also seems possible that an unwanted RelabelType
would sometimes get into an emitted plan --- but because record_eq and
friends don't examine the declared type of their input expressions, that
would not create any visible problems.

To fix, just treat RECORDOID as if it were a polymorphic type, which in
some sense it is.  We might want to consider formalizing that a bit more
someday, but for the moment this seems to be the only place where an
IsPolymorphicType() test ought to include RECORDOID as well.

This has been broken for a long time, so back-patch to all supported
branches.

Discussion: https://postgr.es/m/a6b22369-e3bf-4d49-f59d-0c41d3551e81@sigaev.ru
2018-05-16 13:46:23 -04:00
Robert Haas 7fc7dac1a7 Pass the correct PlannerInfo to PlanForeignModify/PlanDirectModify.
Previously, we passed the toplevel PlannerInfo, but we actually want
to pass the relevant subroot.  One problem with passing the toplevel
PlannerInfo is that the FDW which wants to push down an UPDATE or
DELETE against a join won't find the relevant joinrel there.
As of commit 1bc0100d27, postgres_fdw
tries to do exactly this and can be made to fail an assertion as a
result.

It's possible that this should be regarded as a bug fix and
back-patched to earlier releases, but for lack of a test case that
fails in earlier releases, no back-patch for now.

Etsuro Fujita, reviewed by Amit Langote.

Discussion: http://postgr.es/m/5AF43E02.30000@lab.ntt.co.jp
2018-05-16 11:32:38 -04:00
Robert Haas 09b12d52db Improve comment in get_partition_dispatch_recurse.
David Rowley, reviewed by Amit Langote, and revised a bit by me.

Discussion: http://postgr.es/m/CAKJS1f9yyimYyFzbHM4EwE+tkj4jvrHqSH0H4S4Kbas=UFpc9Q@mail.gmail.com
2018-05-16 10:47:57 -04:00
Bruce Momjian 6bd1b4c31b docs: add space in PG 11 release notes, huge/large
Reported-by: Tatsuo Ishii
2018-05-15 20:03:42 -04:00
Bruce Momjian 7a4a375423 docs: PG 11 rel notes, 'ps' display/background worker item 2018-05-15 19:57:40 -04:00
Bruce Momjian 3b07f6dadb doc: adjust Extended Query PG 11 release note item
Reported-by: Tatsuo Ishii
2018-05-15 19:12:16 -04:00
Bruce Momjian 3122e32091 docs: update PG 11 release notes based on feedback 2018-05-15 15:55:53 -04:00
Tom Lane 05ca21b878 Fix type checking for support functions of parallel VARIADIC aggregates.
The impact of VARIADIC on the combine/serialize/deserialize support
functions of an aggregate wasn't thought through carefully.  There is
actually no impact, because variadicity isn't passed through to these
functions (and it doesn't seem like it would need to be).  However,
lookup_agg_function was mistakenly told to check things as though it were
passed through.  The net result was that it was impossible to declare an
aggregate that had both VARIADIC input and parallelism support functions.

In passing, fix a runtime check in nodeAgg.c for the combine function's
strictness to make its error message agree with the creation-time check.
The previous message was actually backwards, and it doesn't seem like
there's a good reason to have two versions of this message text anyway.

Back-patch to 9.6 where parallel aggregation was introduced.

Alexey Bashtanov; message fix by me

Discussion: https://postgr.es/m/f86dde87-fef4-71eb-0480-62754aaca01b@imap.cc
2018-05-15 15:06:53 -04:00
Bruce Momjian 185f4f84d5 doc: clarify SCRAM channel binding
Discussion: https://postgr.es/m/20180514231020.GB1600@paquier.xyz

Reviewed-by: Michael Paquier
2018-05-14 20:45:35 -04:00
Bruce Momjian cf9c75ccea doc: update PG 11 release notes with suggested changes 2018-05-14 16:41:49 -04:00
Alvaro Herrera 4eaa537275 Don't allow partitioned index on foreign-table partitions
Creating indexes on foreign tables is already forbidden, but local
partitioned indexes (commit 8b08f7d482) forgot to check for them.  Add
a preliminary check to prevent wasting time.

Another school of thought says to allow the index to be created if it's
not a unique index; but it's possible to do better in the future (enable
indexing of foreign tables, somehow), so we avoid painting ourselves in
a corner by rejecting all cases, to avoid future grief (a.k.a. backward
incompatible changes).

Reported-by: Arseny Sher
Author: Amit Langote, Álvaro Herrera
Discussion: https://postgr.es/m/87sh71cakz.fsf@ars-thinkpad
2018-05-14 13:23:07 -04:00
Magnus Hagander fc2a41e23e Fix file paths in comments
Author: Daniel Gustafsson <daniel@yesql.se>
2018-05-14 18:59:43 +02:00
Alvaro Herrera 93316299d6 psql: have \d show FKs on partitioned tables
Commit 3de241dba8 missed to update psql to display foreign keys on
partitioned tables.  Add that.

Reported-by: Amit Langote
Author: Amit Langote
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/a66879e5-636f-d4dd-b4a4-92bdca5a828f@lab.ntt.co.jp
2018-05-14 12:14:17 -04:00
Tom Lane 60e2d9ab14 Doc: fix minor release-note typo.
Liudmila Mantrova

Discussion: https://postgr.es/m/d13458be-c4b9-0fd8-f333-c8de4d0c9120@postgrespro.ru
2018-05-14 12:08:02 -04:00
Bruce Momjian 8c6227a2f3 doc: update PG 11 rel. notes for ALTER TABLE's non-null default
Reported-by: Peter Geoghegan
2018-05-12 20:46:37 -04:00
Alvaro Herrera bebc46931a docs: Rework sections on partition pruning/exclusion
Explain partition pruning more thoroughly, in a section above the one
that explains constraint exclusion, since the new feature is the one
that will be used more extensively from now on.  Move some of the
material from the constraint exclusion subsection to the one on
partition pruning, so that we can explain the legacy method by
explaining the differences with the new one instead of repeating it.

Author: David Rowley, Álvaro Herrera
Reviewed-by: Amit Langote, David G. Johnston, Justin Pryzby
Discussion: https://postgr.es/m/CAKJS1f8PECxEi1YQ9nhVtshtfOMHUzAMm_Zp4gGCOCnMPjEKJA@mail.gmail.com
2018-05-12 12:08:17 -03:00
Tom Lane d9fcf7f5e3 Doc: fix overenthusiastic markup.
I get "WARNING: nested link may be undefined in output: <xref @linkend =
'pgbench'> nested inside parent element link" from this.

Also remove some trailing whitespace.
2018-05-11 17:19:21 -04:00
Bruce Momjian fb68638ae8 docs: more PG 11 markup and email suggestions 2018-05-11 17:06:22 -04:00
Bruce Momjian 6186d0bd61 doc: markup for PG 11 release notes and included email tips 2018-05-11 14:47:26 -04:00
Bruce Momjian 5631c99d2a docs: initial draft of PG 11 release notes 2018-05-11 10:54:03 -04:00
Bruce Momjian 25468994ca docs: clarify that CREATE TABLE ... _AS_ can be parallelized
CREATE TABLE without AS doesn't have anything to parallelize.
2018-05-10 22:37:26 -04:00
Teodor Sigaev 8e12f4a250 Various improvements of skipping index scan during vacuum technics
- Change vacuum_cleanup_index_scale_factor GUC to PGC_USERSET.
  vacuum_cleanup_index_scale_factor GUC was defined as PGC_SIGHUP.  But this
  GUC affects not only autovacuum.  So it might be useful to change it from user
  session in order to influence manually runned VACUUM.
- Add missing tab-complete support for vacuum_cleanup_index_scale_factor
  reloption.
- Fix condition for B-tree index cleanup.
  Zero value of vacuum_cleanup_index_scale_factor means that user wants B-tree
  index cleanup to be never skipped.
- Documentation and comment improvements

Authors: Justin Pryzby, Alexander Korotkov, Liudmila Mantrova
Reviewed by: all authors and Robert Haas
Discussion: https://www.postgresql.org/message-id/flat/20180502023025.GD7631%40telsasoft.com
2018-05-10 13:31:47 +03:00
Robert Haas ddc1f32ee5 doc: Restrictions on InitPlans in parallel queries relaxed.
This updates the documentation for changes originally made in commit
e89a71fb44.

Patch by me, reviewed (but not entirely endorsed) by Amit Kapila.

Discussion: http://postgr.es/m/CA+Tgmoa+vupW8V_gBonz6hU7WwN2zJ=UTsVWCVB+rN6vaaXfZw@mail.gmail.com
2018-05-09 15:15:03 -04:00
Tom Lane 234bb985c5 Update time zone data files to tzdata release 2018e.
DST law changes in North Korea.  Redefinition of "daylight savings" in
Ireland, as well as for some past years in Namibia and Czechoslovakia.
Additional historical corrections for Czechoslovakia.

With this change, the IANA database models Irish timekeeping as following
"standard time" in summer, and "daylight savings" in winter, so that the
daylight savings offset is one hour behind standard time not one hour
ahead.  This does not change their UTC offset (+1:00 in summer, 0:00 in
winter) nor their timezone abbreviations (IST in summer, GMT in winter),
though now "IST" is more correctly read as "Irish Standard Time" not "Irish
Summer Time".  However, the "is_dst" column in the pg_timezone_names view
will now be true in winter and false in summer for the Europe/Dublin zone.

Similar changes were made for Namibia between 1994 and 2017, and for
Czechoslovakia between 1946 and 1947.

So far as I can find, no Postgres internal logic cares about which way
tm_isdst is reported; in particular, since commit b2cbced9e we do not
rely on it to decide how to interpret ambiguous timestamps during DST
transitions.  So I don't think this change will affect any Postgres
behavior other than the timezone-view outputs.

Discussion: https://postgr.es/m/30996.1525445902@sss.pgh.pa.us
2018-05-09 13:56:22 -04:00
Alvaro Herrera bef5fcc36b pgstatindex, pageinspect: handle partitioned indexes
Commit 8b08f7d482 failed to update these modules to at least give
non-broken error messages for partitioned indexes.  Add appropriate
error support to them.

Peter G. was complaining about a problem of unfriendly error messages;
while we haven't fixed that yet, subsequent discussion let to discovery
of these unhandled cases.

Author: Michaël Paquier
Reported-by: Peter Geoghegan
Discussion: https://postgr.es/m/CAH2-WzkOKptQiE51Bh4_xeEHhaBwHkZkGtKizrFMgEkfUuRRQg@mail.gmail.com
2018-05-09 14:22:59 -03:00
Alvaro Herrera c8478f4fd9 pgstatindex: HASH -> hash
Fix the lone error message in the whole source tree to use capitalized
HASH when referring to hash indexes, making it look like all the other
messages.

Someday it would be good to standardize 'B-Tree', 'B-tree', 'btree', and
random other spellings, too, but that's a larger patch ...

Author: Álvaro Herrera
2018-05-09 14:21:59 -03:00
Alvaro Herrera 550091f218 Add relkind 'I' to catalog.sgml's list relkinds
Commit 8b08f7d482 added a relkind for local partitioned indexes, but
failed to add it to pg_class's list of possible relkinds.  Repair.

Author: Peter Geoghegan, Michaël Paquier
Discussion: https://postgr.es/m/CAH2-WzkOKptQiE51Bh4_xeEHhaBwHkZkGtKizrFMgEkfUuRRQg@mail.gmail.com
2018-05-09 13:46:12 -03:00
Alvaro Herrera d758d9702e Fix assorted partition pruning bugs
match_clause_to_partition_key failed to consider COERCION_PATH_ARRAYCOERCE
cases in scalar-op-array expressions, so it was possible to crash the
server easily.  To handle this case properly (ie. prune partitions) we
would need to run a bit of executor code during planning.  Maybe it can
be improved, but for now let's just not crash.  Add a test case that
used to trigger the crash.
Author: Michaël Paquier

match_clause_to_partition_key failed to indicate that operators that
don't have a commutator in a btree opclass are unsupported.  It is
possible for this to cause a crash later if such an operator is used in
a scalar-op-array expression.  Add a test case that used to the crash.
Author: Amit Langote

One caller of gen_partprune_steps_internal in
match_clause_to_partition_key was too optimistic about the former never
returning an empty step list.  Rid it of its innocence.  (Having fixed
the bug above, I no longer know how to exploit this, so no test case for
it, but it remained a bug.)  Revise code flow a little bit, for
succintness.
Author: Álvaro Herrera

Reported-by: Marina Polyakova
Reviewed-by: Michaël Paquier
Reviewed-by: Amit Langote
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/ff8f9bfa485ff961d6bb43e54120485b@postgrespro.ru
2018-05-09 11:27:04 -03:00
Andrew Dunstan 35361ee788 Restrict vertical tightness to parentheses in Perl code
The vertical tightness settings collapse vertical whitespace between
opening and closing brackets (parentheses, square brakets and braces).
This can make data structures in particular harder to read, and is not
very consistent with our style in non-Perl code. This patch restricts
that setting to parentheses only, and reformats all the perl code
accordingly. Not applying this to parentheses has some unfortunate
effects, so the consensus is to keep the setting for parentheses and not
for the others.

The diff for this patch does highlight some places where structures
should have trailing commas. They can be added manually, as there is no
automatic tool to do so.

Discussion: https://postgr.es/m/a2f2b87c-56be-c070-bfc0-36288b4b41c1@2ndQuadrant.com
2018-05-09 10:14:46 -04:00
Andrew Dunstan 286bb240e1 perltidy some recent code changes before changing perltidy settings 2018-05-09 10:05:35 -04:00
Alvaro Herrera d1e2cac5ff Make gen_partprune_steps static
There's no need to export this function, so don't.  Michaël didn't
actually write the patch, but we list him as first author because with a
trivial one like this, intellectual authorship is as important (if not
more) as bit shovelling.

Author: Michaël Paquier, Amit Langote
Discussion: https://postgr.es/m/c91299c4-199b-0f16-339b-a29d6d2a39ee@lab.ntt.co.jp
2018-05-09 10:40:25 -03:00
Alvaro Herrera c775fb9e18 Remove useless 'default' clause
Author: Michael Paquier
Reviewed-by: Amit Langote
Reviewed-by: Álvaro Herrera
Discussion: https://postgr.es/m/20180424012042.GD1570@paquier.xyz
Discussion: https://postgr.es/m/20180509061039.GC11897@paquier.xyz
2018-05-09 10:33:55 -03:00
Andrew Dunstan 91703ca214 Add a script and a config file to run perlcritic
This is similar to what we do to run perltidy. For now we only run at
severity level 5. Over time we can improve our perl code and reduce the
severity level.

Discussion: https://postgr.es/m/86aa2a3a-0c68-21fb-9560-84ad6914d561@2ndQuadrant.com
2018-05-09 07:55:23 -04:00
Teodor Sigaev cb5d942959 Improve jsonb cast error message
Initial variant of error message didn't follow style of another casting error
messages and wasn't informative. Per gripe from Robert Haas.

Reviewer: Tom Lane
Discussion: https://www.postgresql.org/message-id/flat/CA%2BTgmob08StTV9yu04D0idRFNMh%2BUoyKax5Otvrix7rEZC8rMw%40mail.gmail.com#CA+Tgmob08StTV9yu04D0idRFNMh+UoyKax5Otvrix7rEZC8rMw@mail.gmail.com
2018-05-09 13:23:16 +03:00
Tom Lane c63913ca7d Improve inefficient regexes in vacuumdb TAP test.
The regexes used in 102_vacuumdb_stages.pl to check the postmaster log
for expected output contained several places with ".*.*", which is
underdetermined and can cause exponential runtime growth in Perl's regex
matcher (since it's not bright enough not to waste time seeing whether
different splits of the same substring would allow a match).  We were
fortunate that the amount of text in the postmaster log was generally not
enough to make the runtime go to the moon; although commit 6271fceb8 had
been on the hairy edge of an obvious problem, thanks to its increasing the
default log verbosity to DEBUG1.  Experimentation shows that anyone who
tried to run this test case with an even higher log verbosity would have
been in for serious pain.  But even at default logging level, fixing this
saves several hundred ms on my workstation, more on slower buildfarm
members.

Remove the extra ".*"s, restoring more-or-less-linear matching speed.
Back-patch to 9.4 where the test case was added, mostly in case anyone
tries to do related debugging in a back branch.

Discussion: https://postgr.es/m/32459.1525657786@sss.pgh.pa.us
2018-05-08 20:17:43 -04:00
Tom Lane dec10340d5 Improve initdb's query for generating default descriptions a little.
While poking into initdb's performance, I noticed that this query
wasn't being done very intelligently.  By forcing it to execute
obj_description() for each pg_proc/pg_operator join row, we were
essentially setting up a nestloop join to pg_description, which
is not a bright query plan when there are hundreds of outer rows.
Convert the check for a "deprecated" operator into a NOT EXISTS
so that it can be done as a hashed antijoin.  On my workstation
this reduces the time for this query from ~ 35ms to ~ 10ms.
Which is not a huge win, but it adds up over buildfarm runs.

In passing, insert forced query breaks (\n\n, in single-user mode)
after each SQL-query file that initdb sources, and after some
relatively new queries in setup_privileges().  This doesn't make
a lot of difference normally, but it will result in briefer, saner
error messages if anything goes wrong.
2018-05-08 15:59:05 -04:00
Peter Eisentraut 831f5d11ec Refine error messages
"JSON" when not referring to a data type should be upper case.
2018-05-08 14:36:31 -04:00
Tom Lane 3a675f729e Count heap tuples in non-SnapshotAny path in IndexBuildHeapRangeScan().
Brown-paper-bag bug in commit 7c91a0364: when we rearranged the placement
of "reltuples += 1" statements, we missed including one in this code path.

The net effect of that was that CREATE INDEX CONCURRENTLY would set the
table's pg_class.reltuples to zero, as would index builds done during
bootstrap mode.  (It seems like parallel index builds ought to fail
similarly, but they don't, perhaps because reltuples is computed in some
other way.  You certainly couldn't figure that out from the abysmally
underdocumented parallelism code in this area.)

I was led to this by wondering why initdb seemed to have slowed down as
a result of 7c91a0364, as is evident in the buildfarm's timing history.
The reason is that every system catalog with indexes had pg_class.reltuples
= 0 after bootstrap, causing the planner to make some terrible choices for
queries in the post-bootstrap steps.  On my workstation, this fix causes
the runtime of "initdb -N" to drop from ~2.0 sec to ~1.4 sec, which is
almost though not quite back to where it was in v10.  That's not much of
a deal for production use perhaps, but it makes a noticeable difference
for buildfarm and "make check-world" runs, which do a lot of initdbs.
2018-05-08 00:20:19 -04:00
Andrew Dunstan d2c1512ac4 Clean up some perlcritic warnings
In Catalog.pm, mark eval of a string instead of a block as allowed.
Disallow perlcritic completely in Gen_dummy_probes.pl, as it's
generated code.
Protect a couple of lines in plperl code from  perltidy, so that the
annotation for perlcritic stays on the same line as the construct it
would otherwise object to.
2018-05-07 15:35:32 -04:00
Tom Lane 17551f1a21 Undo extra chattiness of postmaster logs in TAP tests.
Commit 6271fceb8 changed PostgresNode.pm to force log_min_messages = debug1
in all TAP tests, without any discussion and without a concrete need for
it.  This makes some of the TAP tests noticeably slower (although much of
that may be due to poorly-written regexes), and for certain it's bloating
the buildfarm logs.  Revert the change.

Discussion: https://postgr.es/m/32459.1525657786@sss.pgh.pa.us
2018-05-07 15:12:01 -04:00
Tom Lane fbb99e5883 Update oidjoins regression test for v11.
Commit 86f575948 already manually updated the oidjoins test for the
new pg_constraint.conparentid => pg_constraint.oid relationship, but
failed to update findoidjoins/README, thus the apparent inconsistency
here.

Michael Paquier

Discussion: https://postgr.es/m/20180507001811.GA27389@paquier.xyz
2018-05-07 14:32:04 -04:00
Tom Lane 513ff52e81 Suppress compiler warnings when building with --enable-dtrace.
Most versions of "dtrace -h" drop const qualifiers from the declarations
of probe functions (though macOS gets it right).  This causes compiler
warnings when we pass in pointers to const.  Repair by extending our
existing post-processing of the probes.h file.  To do so, assume that all
"char *" arguments should be "const char *"; that seems reasonably safe.

Thomas Munro

Discussion: https://postgr.es/m/CAEepm=2j1pWSruQJqJ91ZDzD8w9ZZDsM4j2C6x75C-VryWg-_w@mail.gmail.com
2018-05-07 13:44:09 -04:00
Tom Lane f34f0e4c58 Last-minute updates for release notes.
The set of functions that need parallel-safety adjustments isn't the
same in 9.6 as 10, so I shouldn't have blindly back-patched that list.
Adjust as needed.  Also, provide examples of the commands to issue.
2018-05-07 13:13:27 -04:00
Tom Lane b56d5f230f Last-minute updates for release notes.
Security: CVE-2018-1115
2018-05-07 11:50:15 -04:00