Commit Graph

41264 Commits

Author SHA1 Message Date
Robert Haas 1d15d0db50 psql: Tab-complete LOCK [TABLE] ... IN {ACCESS|ROW|SHARE}.
Suggest the lock modes that begin with the word in question.

Thomas Munro, reviewed by Marllius Ribeiro.  Comments tweaked by me.
2016-11-03 11:42:13 -04:00
Robert Haas c6ff380e38 Use NIL instead of NULL for an empty List.
Tatsuro Yamada, reviewed by Ashutosh Bapat
2016-11-03 11:24:01 -04:00
Robert Haas 274bb2b385 libpq: Allow connection strings and URIs to specify multiple hosts.
It's also possible to specify a separate port for each host.

Previously, we'd loop over every address returned by looking up the
host name; now, we'll try every address for every host name.

Patch by me.  Victor Wagner wrote an earlier patch for this feature,
which I read, but I didn't use any of his code.  Review by Mithun Cy.
2016-11-03 09:25:20 -04:00
Tom Lane 770671062f Don't make FK-based selectivity estimates in inheritance situations.
The foreign-key-aware logic for estimation of join sizes (added in commit
100340e2d) blindly tried to apply the concept to rels that are actually
parents of inheritance trees.  This is just plain wrong so far as the
referenced relation is concerned, since the inheritance scan may well
produce lots of rows that are not participating in the constraint.  It's
wrong for the referencing relation too, for the same reason; although on
that end we could conceivably detect whether all members of the inheritance
tree have equivalent FK constraints pointing to the same referenced rel,
and then proceed more or less as we do now.  But pending somebody writing
code to do that, we must disable this, because it's producing completely
silly estimates when there's an FK linking the heads of inheritance trees.

Per bug #14404 from Clinton Adams.  Back-patch to 9.6 where the new
estimation logic came in.

Report: <20161028200412.15987.96482@wrigleys.postgresql.org>
2016-11-02 15:50:15 -04:00
Tom Lane da8f3ebf30 Don't convert Consts into Vars during setrefs.c processing.
While converting expressions in an upper-level plan node so that they
reference Vars and expressions provided by the input plan node(s),
don't convert plain Const items, even if there happens to be a matching
Const in the input.  It's silly to do so because a Var is more expensive to
execute than a Const.  Moreover, converting can fool ExecCheckPlanOutput's
check that an insert or update query inserts nulls into dropped columns,
leading to "query provides a value for a dropped column" errors during
INSERT or UPDATE on a table with a dropped column.  We could solve this
by making that check more complicated, but I don't see the point; this fix
should save a marginal number of cycles, and it also makes for less messy
EXPLAIN output, as shown by the ensuing regression test result changes.

Per report from Pavel Hanák.  I have not incorporated a test case based
on that example, as there doesn't seem to be a simple way of checking
this in isolation without making a bunch of assumptions about other
planner and SQL-function behavior.

Back-patch to 9.6.  This setrefs.c behavior exists much further back,
but there is not currently reason to think that it causes problems
before 9.6.

Discussion: <83shraampf.fsf@is-it.eu>
2016-11-02 14:32:13 -04:00
Peter Eisentraut 00a86856c1 pageinspect: Make page test more portable
Choose test data that makes the output independent of endianness.
2016-11-02 08:45:17 -04:00
Tom Lane 14ee35799f Fix portability bug in gin_page_opaque_info().
Somebody apparently thought that "if Int32GetDatum is good,
Int64GetDatum must be better".  Per buildfarm failures now
that Peter has added some regression tests here.
2016-11-02 00:09:27 -04:00
Peter Eisentraut f7c9a6e083 pageinspect: Make btree test more portable
Choose test data that makes the output independent of endianness and
alignment.
2016-11-01 22:02:39 -04:00
Robert Haas 9779f71c40 postgres_fdw: Fix typo in comment.
Etsuro Fujita
2016-11-01 16:21:29 -04:00
Peter Eisentraut b5e3942f50 Add subsection headers in pageinspect documentation
extracted from a patch from Jesper Pedersen <jesper.pedersen@redhat.com>
2016-11-01 15:16:31 -04:00
Peter Eisentraut adfb81d9e1 pageinspect: Add tests 2016-11-01 14:02:16 -04:00
Peter Eisentraut 3a47c704fb Add make rules to download raw Unicode mapping files
This serves as implicit documentation and is handy if someone wants to
tweak things.  The rules are not part of a normal build, like this
entire directory.
2016-11-01 11:54:58 -04:00
Robert Haas 6bb9a6177d Remove declarations for pq_putmessage_hook and pq_flush_hook.
Commit 2bd9e412f9 added these in error.
They were part of an earlier design for that patch and survived in the
committed version only by inadvertency.

Julien Rouhaud
2016-10-31 09:14:46 -04:00
Tatsuo Ishii 36d154ecb2 Fix typo in sources.sgml.
Per Shinichi Matsuda.
2016-10-31 07:33:00 +09:00
Tom Lane 5ec81aceec Fix nasty performance problem in tsquery_rewrite().
tsquery_rewrite() tries to find matches to subsets of AND/OR conditions;
for example, in the query 'a | b | c' the substitution subquery 'a | c'
should match and lead to replacement of the first and third items.
That's fine, but the matching algorithm apparently takes about O(2^N)
for an N-clause query (I say "apparently" because the code is also both
unintelligible and uncommented).  We could probably do better than that
even without any extra assumptions --- but actually, we know that the
subclauses are sorted, indeed are depending on that elsewhere in this very
same function.  So we can just scan the two lists a single time to detect
matches, as though we were doing a merge join.

Also do a re-flattening call (QTNTernary()) in tsquery_rewrite_query, just
to make sure that the tree fits the expectations of the next search cycle.
I didn't try to devise a test case for this, but I'm pretty sure that the
oversight could have led to failure to match in some cases where a match
would be expected.

Improve comments, and also stick a CHECK_FOR_INTERRUPTS into
dofindsubquery, just in case it's still too slow for somebody.

Per report from Andreas Seltenreich.  Back-patch to all supported branches.

Discussion: <8760oasf2y.fsf@credativ.de>
2016-10-30 17:35:42 -04:00
Tom Lane 24ebc444c6 Fix bogus tree-flattening logic in QTNTernary().
QTNTernary() contains logic to flatten, eg, '(a & b) & c' into 'a & b & c',
which is all well and good, but it tries to do that to NOT nodes as well,
so that '!!a' gets changed to '!a'.  Explicitly restrict the conversion to
be done only on AND and OR nodes, and add a test case illustrating the bug.

In passing, provide some comments for the sadly naked functions in
tsquery_util.c, and simplify some baroque logic in QTNFree(), which
I think may have been leaking some items it intended to free.

Noted while investigating a complaint from Andreas Seltenreich.
Back-patch to all supported versions.
2016-10-30 15:24:40 -04:00
Tom Lane 9a00f03e47 Improve speed of aggregates that use array_append as transition function.
In the previous coding, if an aggregate's transition function returned an
expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus
force it into the flat representation.  This led to ping-ponging between
flat and expanded formats, which costs a lot.  For an aggregate using
array_append as transition function, I measured about a 15X slowdown
compared to the pre-9.5 code, when working on simple int[] arrays.
Of course, the old code was already O(N^2) in this usage due to copying
flat arrays all the time, but it wasn't quite this inefficient.

To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition
values without copying, so long as the transition function takes care to
return the transition value already properly parented under the aggcontext.
That puts a bit of extra responsibility on the transition function, but
doing it this way allows us to not need any extra logic in the fast path
of advance_transition_function (ie, with a pass-by-value transition value,
or with a modified-in-place pass-by-reference value).  We already know
that that's a hot spot so I'm loath to add any cycles at all there.  Also,
while only array_append currently knows how to follow this convention,
this solution allows other transition functions to opt-in without needing
to have a whitelist in the core aggregation code.

(The reason we would need a whitelist is that currently, if you pass a
R/W expanded-object pointer to an arbitrary function, it's allowed to do
anything with it including deleting it; that breaks the core agg code's
assumption that it should free discarded values.  Returning a value under
aggcontext is the transition function's signal that it knows it is an
aggregate transition function and will play nice.  Possibly the API rules
for expanded objects should be refined, but that would not be a
back-patchable change.)

With this fix, an aggregate using array_append is no longer O(N^2), so it's
much faster than pre-9.5 code rather than much slower.  It's still a bit
slower than the bespoke infrastructure for array_agg, but the differential
seems to be only about 10%-20% rather than orders of magnitude.

Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 12:27:41 -04:00
Magnus Hagander a775406ec4 Fix memory leak in tar file padding
Spotted by Coverity, patch by Michael Paquier
2016-10-30 14:10:39 +01:00
Robert Haas d4b5d4cadd pgstattuple: Don't take heavyweight locks when examining a hash index.
It's currently necessary to take a heavyweight lock when scanning a
hash bucket, but pgstattuple only examines individual pages, so it
doesn't need to do this.  If, for some hypothetical reason, it did
need to do any heavyweight locking here, this logic would probably
still be incorrect, because most of the locks that it is taking are
meaningless.  Only a heavyweight lock on a primary bucket page has any
meaning, but this takes heavyweight locks on all pages regardless of
function - and in particular overflow pages, where you might imagine
that we'd want to lock the primary bucket page if we needed to lock
anything at all.

This is arguably a bug that has existed since this code was added in
commit dab42382f4, but I'm not going to
bother back-patching it because in most cases the only consequence is
that running pgstattuple() on a hash index is a little slower than it
otherwise might be, which is no big deal.

Extracted from a vastly larger patch by Amit Kapila which heavyweight
locking for hash indexes entirely; analysis of why this can be done
independently of the rest by me.
2016-10-28 12:21:15 -04:00
Robert Haas 33839b5ffb Fix leftover reference to background writer performing checkpoints.
This was changed in PostgreSQL 9.2, but somehow this comment never
got updated.
2016-10-28 09:09:00 -04:00
Peter Eisentraut a94b70356b doc: Small style improvements 2016-10-27 20:33:47 -04:00
Peter Eisentraut ce4dc97056 Remove invitation to report a bug about unknown encoding
The error message when we couldn't determine the encoding from a locale
said to report a bug about that.  That might have been appropriate when
this code was first added, but by now this works pretty solidly and any
encodings we don't recognize we probably just don't support.  We still
print the warning, but no longer invite the bug report.
2016-10-27 18:43:46 -04:00
Peter Eisentraut eaed88ce12 Add function name to PyArg_ParseTuple()
This causes the supplied function name to appear in any error message,
making the error message friendlier and relieving us from having to
provide our own in some cases.
2016-10-27 15:41:29 -04:00
Peter Eisentraut 84d457edaf Format PL/Python module contents test vertically
It makes it readable again and makes merges more manageable.
2016-10-27 15:41:29 -04:00
Robert Haas 4f714b2fd2 If the stats collector dies during Hot Standby, restart it.
This bug exists as far back as 9.0, when Hot Standby was introduced,
so back-patch to all supported branches.

Report and patch by Takayuki Tsunakawa, reviewed by Michael Paquier
and Kuntal Ghosh.
2016-10-27 14:27:40 -04:00
Robert Haas f267c1c244 Fix possible pg_basebackup failure on standby with "include WAL".
If a restartpoint flushed no dirty buffers, it could fail to update
the minimum recovery point, leading to a minimum recovery point prior
to the starting REDO location.  perform_base_backup() would interpret
that as meaning that no WAL files at all needed to be included in the
backup, failing an internal sanity check.  To fix, have restartpoints
always update the minimum recovery point to just after the checkpoint
record itself, so that the file (or files) containing the checkpoint
record will always be included in the backup.

Code by Amit Kapila, per a design suggestion by me, with some
additional work on the code comment by me.  Test case by Michael
Paquier.  Report by Kyotaro Horiguchi.
2016-10-27 11:19:51 -04:00
Peter Eisentraut c32fe432af Avoid using a C++ keyword in header file
per cpluspluscheck
2016-10-26 22:41:56 -04:00
Bruce Momjian 586a46c22c Properly indent postgresql.conf comments to align
A few comments were misaligned.
2016-10-26 21:16:50 -04:00
Tom Lane a522fc3d80 Fix incorrect trigger-property updating in ALTER CONSTRAINT.
The code to change the deferrability properties of a foreign-key constraint
updated all the associated triggers to match; but a moment's examination of
the code that creates those triggers in the first place shows that only
some of them should track the constraint's deferrability properties.  This
leads to odd failures in subsequent exercise of the foreign key, as the
triggers are fired at the wrong times.  Fix that, and add a regression test
comparing the trigger properties produced by ALTER CONSTRAINT with those
you get by creating the constraint as-intended to begin with.

Per report from James Parks.  Back-patch to 9.4 where this ALTER
functionality was introduced.

Report: <CAJ3Xv+jzJ8iNNUcp4RKW8b6Qp1xVAxHwSXVpjBNygjKxcVuE9w@mail.gmail.com>
2016-10-26 17:05:06 -04:00
Tom Lane 19b2094d96 Fix not-HAVE_SYMLINK code in zic.c.
I broke this in commit f3094920a.  Apparently it's dead code anyway,
at least as far as our buildfarm is concerned (and the upstream IANA
code doesn't worry at all about symlink() not being present).
But as long as the rest of our code is willing to guard against not
having symlink(), this should too.  Noted while investigating a
tangentially-related complaint from Sandeep Thakkar.

Back-patch to keep branches in sync.
2016-10-26 13:40:41 -04:00
Tom Lane 162477a63d Doc: improve documentation about inheritance.
Clarify documentation about inheritance of check constraints, in
particular mentioning the NO INHERIT option, which didn't exist when
this text was written.

Document that in an inherited query, the applicable row security policies
are those of the explicitly-named table, not its children.  This is the
intended behavior (per off-list discussion with Stephen Frost), and there
are regression tests for it, but it wasn't documented anywhere user-facing
as far as I could find.

Do a bit of wordsmithing on the description of inherited access-privilege
checks.

Back-patch to 9.5 where RLS was added.
2016-10-26 11:46:25 -04:00
Tom Lane 8529686ccb Suppress unused-variable warning in non-assert builds.
Introduced in commit 7012b132d.

Kyotaro Horiguchi
2016-10-26 10:19:41 -04:00
Heikki Linnakangas 8eb6337f9f Remove platform-dependent PL/python test case.
Turns out that the output format of Python Decimal isn't totally platform-
independent either. There are other tests for multi-dimensional arrays,
so rather than try to fix this test case, just remove it.

Per buildfarm member prairiedog.
2016-10-26 17:09:18 +03:00
Heikki Linnakangas cfd9c87a54 Only treat Python Lists as array dimensions.
Instead of treating all python sequence types as array dimensions, except
for tuples and various kinds of strings, only treat Python lists as
dimensions. The PyBytes_Check() function used previously is only available
on Python 2.6 and newer, and it was a bit fiddly anyway. The list of
exceptions would require adjustment if Python got a new kind of a sequence
similar to bytes/unicodes/strings, so only checking for Lists seems more
future-proof. The documentation only mentioned using Lists, so this is
closer to what was documented, anyway.

This should fix the buildfarm failures on systems building with Python 2.5,
although I don't have Python 2.5 installed myself to test with.
2016-10-26 14:44:55 +03:00
Heikki Linnakangas 73c8e8506c Avoid using platform-dependent floats in test case.
The number of decimals printed for floats varied in this test case, as
noted by several buildfarm members. There's nothing special about floats
and arrays in the code being tested, so replace the floats with numerics to
make the output platform-independent.
2016-10-26 14:17:07 +03:00
Heikki Linnakangas e131ba4fe5 Fix regression test to also work with Python 2.
Per buildfarm.
2016-10-26 11:18:04 +03:00
Heikki Linnakangas 56f39009c5 Fix typos in comments.
Vinayak Pokale
2016-10-26 11:12:31 +03:00
Heikki Linnakangas 8a2f08fbea Fix typo in comment.
Daniel Gustafsson
2016-10-26 11:10:13 +03:00
Heikki Linnakangas 510e1b8ecf Give a hint, when [] is incorrectly used for a composite type in array.
That used to be accepted, so let's try to give a hint to users on why
their PL/python functions no longer work.

Reviewed by Pavel Stehule.

Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com>
2016-10-26 10:56:56 +03:00
Heikki Linnakangas 94aceed317 Support multi-dimensional arrays in PL/python.
Multi-dimensional arrays can now be used as arguments to a PL/python function
(used to throw an error), and they can be returned as nested Python lists.

This makes a backwards-incompatible change to the handling of composite
types in arrays. Previously, you could return an array of composite types
as "[[col1, col2], [col1, col2]]", but now that is interpreted as a two-
dimensional array. Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2),
(col1, col2)]".

To avoid breaking backwards-compatibility, when not necessary, () is still
accepted for arrays at the top-level, but it is always treated as a
single-dimensional array. Likewise, [] is still accepted for composite types,
when they are not in an array. Update the documentation to recommend using []
for arrays, and () for composite types, with a mention that those other things
are also accepted in some contexts.

This needs to be mentioned in the release notes.

Alexey Grishchenko, Dave Cramer and me. Reviewed by Pavel Stehule.

Discussion: <CAH38_tmbqwaUyKs9yagyRra=SMaT45FPBxk1pmTYcM0TyXGG7Q@mail.gmail.com>
2016-10-26 10:56:30 +03:00
Peter Eisentraut 8c035e55c4 pg_dump: Simplify internal archive version handling
The ArchiveHandle structure contained the archive format version number
twice, once as a single field and once split into components.  Simplify
that by just keeping the single field and adding some macros to extract
the components.  Introduce some macros for composing version numbers, to
eliminate the repeated use of magic formulas.  Drop the unused trailing
zero byte from the run-time composite version representation.

reviewed by Tom Lane
2016-10-25 17:02:22 -04:00
Magnus Hagander 78d109150b Free walmethods before exiting
Not strictly necessary since we quite after, but could become important
in the future if we do restarts etc.

Michael Paquier with nitpicking from me
2016-10-25 19:00:12 +02:00
Magnus Hagander 8c46f0c9ce Don't fsync() files when --no-sync is specified
Michael Paquier
2016-10-25 19:00:01 +02:00
Bruce Momjian 10c064ce4d Consistently mention 'SELECT pg_reload_conf()' in config files
Previously we only mentioned SIGHUP and 'pg_ctl reload' in
postgresql.conf and pg_hba.conf.
2016-10-25 11:26:15 -04:00
Robert Haas f5d6bce63c postgres_fdw: Try again to stabilize aggregate pushdown regression tests.
A query that only aggregates one row isn't a great argument for pushdown,
and buildfarm member brolga decides against it.  Adjust the query a bit
in the hopes of getting remote aggregation to win consistently.

Jeevan Chalke, per suggestion from Tom Lane
2016-10-24 22:36:24 -04:00
Magnus Hagander 2dde01ccbf Use ssize_t where signed results can happen
Noted by Alexander Korotkov
2016-10-24 20:10:18 +02:00
Tom Lane 30a6f98ed8 Update release notes for last-minute commit timestamp fix. 2016-10-24 09:37:23 -04:00
Alvaro Herrera 00f15338b2 Preserve commit timestamps across clean restart
An oversight in setting the boundaries of known commit timestamps during
startup caused old commit timestamps to become inaccessible after a
server restart.

Author and reporter: Julien Rouhaud
Review, test code: Craig Ringer
2016-10-24 09:45:48 -03:00
Tom Lane 7d80417d3d Release notes for 9.6.1, 9.5.5, 9.4.10, 9.3.15, 9.2.19, 9.1.24. 2016-10-23 22:13:28 -04:00
Tom Lane 8f1fb7d621 Avoid testing tuple visibility without buffer lock.
INSERT ... ON CONFLICT (specifically ExecCheckHeapTupleVisible) contains
another example of this unsafe coding practice.  It is much harder to get
a failure out of it than the case fixed in commit 6292c2339, because in
most scenarios any hint bits that could be set would have already been set
earlier in the command.  However, Konstantin Knizhnik reported a failure
with a custom transaction manager, and it's clearly possible to get a
failure via a race condition in async-commit mode.

For lack of a reproducible example, no regression test case in this
commit.

I did some testing with Asserts added to tqual.c's functions, and can say
that running "make check-world" exposed these two bugs and no others.
The Asserts are messy enough that I've not added them to the code for now.

Report: <57EE93C8.8080504@postgrespro.ru>
Related-Discussion: <CAO3NbwOycQjt2Oqy2VW-eLTq2M5uGMyHnGm=RNga4mjqcYD7gQ@mail.gmail.com>
2016-10-23 19:14:32 -04:00