Commit Graph

35745 Commits

Author SHA1 Message Date
Fujii Masao
b1543cc8a8 Add tab completion for \pset in psql.
Pavel Stehule, reviewed by Ian Lawrence Barwick
2013-11-19 23:44:14 +09:00
Peter Eisentraut
bd5a9a50e4 pg_upgrade: Report full disk better
Previously, pg_upgrade would abort copy_file() on a short write without
setting errno, which the caller would report as an error with the
message "Success".  We assume ENOSPC in that case, as we do elsewhere in
the code.  Also set errno in some other error cases in copy_file() to
avoid bogus "Success" error messages.

This was broken in 6b711cf37c, so 9.2 and
before are OK.
2013-11-18 22:01:59 -05:00
Bruce Momjian
0dbf9a6a91 unaccent: Revert patch 9299f61798
The reverted patch to change functions from strict to immutable was
incorrect and needs additional research.
2013-11-18 15:54:34 -05:00
Heikki Linnakangas
fea437681d Spell SQL keywords in uppercase in pg_dump's query.
The server won't care, but let's be consistent.

David Rowley.
2013-11-18 18:34:51 +02:00
Heikki Linnakangas
32ceba3ea7 Replace appendPQExpBuffer(..., <constant>) with appendPQExpBufferStr
Arguably makes the code a bit more readable, and might give a small
performance gain.

David Rowley
2013-11-18 18:34:51 +02:00
Robert Haas
f1df4731ee Use cstring_to_text_with_len when length is known.
This avoids a potentially-expensive extra call to strlen().

David Rowley
2013-11-18 10:19:00 -05:00
Heikki Linnakangas
4c697d8f48 Count locked pages that don't need vacuuming as scanned.
Previously, if VACUUM skipped vacuuming a page because it's pinned, it
didn't count that page as scanned. However, that meant that relfrozenxid
was not bumped up either, which prevented anti-wraparound vacuum from
doing its job.

Report by Миша Тюрин, analysis and patch by Sergey Burladyn and Jeff Janes.
Backpatch to 9.2, where the skip-locked-pages behavior was introduced.
2013-11-18 09:51:09 +02:00
Tom Lane
f901bb50e3 Add make_date() and make_time() functions.
Pavel Stehule, reviewed by Jeevan Chalke and Atri Sharma
2013-11-17 15:06:50 -05:00
Tom Lane
69c8fbac20 Improve performance of numeric sum(), avg(), stddev(), variance(), etc.
This patch improves performance of most built-in aggregates that formerly
used a NUMERIC or NUMERIC array as their transition type; this includes
not only aggregates on numeric inputs, but some aggregates on integer
inputs where overflow of an int8 value is a possibility.  The code now
uses a special-purpose data structure to avoid array construction and
deconstruction overhead, as well as packing and unpacking overhead for
numeric values.

These aggregates' transition type is now declared as INTERNAL, since
it doesn't correspond to any SQL data type.  To keep the planner from
thinking that that means a lot of storage will be used, we make use
of the just-added pg_aggregate.aggtransspace feature.  The space estimate
is set to 128 bytes, which is at least in the right ballpark.

Hadi Moshayedi, reviewed by Pavel Stehule and Tomas Vondra
2013-11-16 18:46:34 -05:00
Tom Lane
6cb86143e8 Allow aggregates to provide estimates of their transition state data size.
Formerly the planner had a hard-wired rule of thumb for guessing the amount
of space consumed by an aggregate function's transition state data.  This
estimate is critical to deciding whether it's OK to use hash aggregation,
and in many situations the built-in estimate isn't very good.  This patch
adds a column to pg_aggregate wherein a per-aggregate estimate can be
provided, overriding the planner's default, and infrastructure for setting
the column via CREATE AGGREGATE.

It may be that additional smarts will be required in future, perhaps even
a per-aggregate estimation function.  But this is already a step forward.

This is extracted from a larger patch to improve the performance of numeric
and int8 aggregates.  I (tgl) thought it was worth reviewing and committing
this infrastructure separately.  In this commit, all built-in aggregates
are given aggtransspace = 0, so no behavior should change.

Hadi Moshayedi, reviewed by Pavel Stehule and Tomas Vondra
2013-11-16 16:03:40 -05:00
Peter Eisentraut
55c3d86a2a pg_upgrade: Fix some whitespace oddities 2013-11-16 11:35:44 -05:00
Tom Lane
61a07bae47 Remove pgbench's hardwired limit on line length in custom script files.
pgbench formerly failed on lines longer than BUFSIZ, unexpectedly
splitting them into multiple commands.  Allow it to work with any
length of input line.

Sawada Masahiko
2013-11-15 19:41:09 -05:00
Tom Lane
f1f21b2d6f Fix incorrect loop counts in tidbitmap.c.
A couple of places that should have been iterating over WORDS_PER_CHUNK
words were iterating over WORDS_PER_PAGE words instead.  This thinko
accidentally failed to fail, because (at least on common architectures
with default BLCKSZ) WORDS_PER_CHUNK is a bit less than WORDS_PER_PAGE,
and the extra words being looked at were always zero so nothing happened.
Still, it's a bug waiting to happen if anybody ever fools with the
parameters affecting TIDBitmap sizes, and it's a small waste of cycles
too.  So back-patch to all active branches.

Etsuro Fujita
2013-11-15 18:34:14 -05:00
Tom Lane
97e1ec4670 Speed up printing of INSERT statements in pg_dump.
In --inserts and especially --column-inserts mode, we can get a useful
speedup by generating the common prefix of all a table's INSERT commands
just once, and then printing the prebuilt string for each row.  This avoids
multiple invocations of fmtId() and other minor fooling around.

David Rowley
2013-11-15 18:02:06 -05:00
Tom Lane
3172eea062 Clean up password prompting logic in streamutil.c.
The previous coding was fairly unreadable and drew double-free warnings
from clang.  I believe the double free was actually not reachable, because
PQconnectionNeedsPassword is coded to not return true if a password was
provided, so that the loop can't iterate more than twice.  Nonetheless
it seems worth rewriting.  No back-patch since this is just cosmetic.
2013-11-15 17:27:41 -05:00
Tom Lane
f3b3b8d5be Compute correct em_nullable_relids in get_eclass_for_sort_expr().
Bug #8591 from Claudio Freire demonstrates that get_eclass_for_sort_expr
must be able to compute valid em_nullable_relids for any new equivalence
class members it creates.  I'd worried about this in the commit message
for db9f0e1d9a, but claimed that it wasn't a
problem because multi-member ECs should already exist when it runs.  That
is transparently wrong, though, because this function is also called by
initialize_mergeclause_eclasses, which runs during deconstruct_jointree.
The example given in the bug report (which the new regression test item
is based upon) fails because the COALESCE() expression is first seen by
initialize_mergeclause_eclasses rather than process_equivalence.

Fixing this requires passing the appropriate nullable_relids set to
get_eclass_for_sort_expr, and it requires new code to compute that set
for top-level expressions such as ORDER BY, GROUP BY, etc.  We store
the top-level nullable_relids in a new field in PlannerInfo to avoid
computing it many times.  In the back branches, I've added the new
field at the end of the struct to minimize ABI breakage for planner
plugins.  There doesn't seem to be a good alternative to changing
get_eclass_for_sort_expr's API signature, though.  There probably aren't
any third-party extensions calling that function directly; moreover,
if there are, they probably need to think about what to pass for
nullable_relids anyway.

Back-patch to 9.2, like the previous patch in this area.
2013-11-15 16:46:18 -05:00
Tom Lane
c7b849a896 Prevent leakage of cached plans and execution trees in plpgsql DO blocks.
plpgsql likes to cache query plans and simple-expression execution state
trees across calls.  This is a considerable win for multiple executions
of the same function.  However, it's useless for DO blocks, since by
definition those are executed only once and discarded.  Nonetheless,
we were allowing a DO block's expression execution trees to survive
until end of transaction, resulting in a significant intra-transaction
memory leak, as reported by Yeb Havinga.  Worse, if the DO block exited
with an error, the compiled form of the block's code was leaked till
end of session --- along with subsidiary plancache entries.

To fix, make DO blocks keep their expression execution trees in a private
EState that's deleted at exit from the block, and add a PG_TRY block
to plpgsql_inline_handler to make sure that memory cleanup happens
even on error exits.  Also add a regression test covering error handling
in a DO block, because my first try at this broke that.  (The test is
not meant to prove that we don't leak memory anymore, though it could
be used for that with a much larger loop count.)

Ideally we'd back-patch this into all versions supporting DO blocks;
but the patch needs to add a field to struct PLpgSQL_execstate, and that
would break ABI compatibility for third-party plugins such as the plpgsql
debugger.  Given the small number of complaints so far, fixing this in
HEAD only seems like an acceptable choice.
2013-11-15 13:52:03 -05:00
Tom Lane
80e3a470ba Minor comment corrections for sequence hashtable patch.
There were enough typos in the comments to annoy me ...
2013-11-15 12:17:12 -05:00
Kevin Grittner
7cb964acb7 Fix buffer overrun in isolation test program.
Commit 061b88c732 saved argv0 to a
global buffer without ensuring that it was zero terminated,
allowing references to it to overrun the buffer and access other
memory.  This probably would not have presented any security risk,
but could have resulted in very confusing failures if the path to
the executable was very long.

Reported by David Rowley
2013-11-15 08:27:42 -06:00
Robert Haas
71dd54ada9 doc: Restore proper alphabetical order.
Colin 't Hart
2013-11-15 08:46:12 -05:00
Heikki Linnakangas
5cb719beee Fix bogus hash table creation.
Andres Freund
2013-11-15 14:23:40 +02:00
Heikki Linnakangas
21025d4a53 Use a hash table to store current sequence values.
This speeds up nextval() and currval(), when you touch a lot of different
sequences in the same backend.

David Rowley
2013-11-15 12:29:38 +02:00
Tom Lane
982b82d6b1 Add a regression test case for \d on an index.
Previous commit shows the need for this.  The coverage isn't really
thorough, but it's better than nothing.
2013-11-14 10:35:15 -05:00
Tom Lane
e694cf25d7 Fix incorrect column name in psql \d code.
pg_index.indisreplident had at one time in its development been called
indisidentity.  describe.c got missed when it was renamed.
Bug introduced in commit 07cacba983.

Andres Freund
2013-11-14 10:27:24 -05:00
Peter Eisentraut
87d8378f60 Fix whitespace 2013-11-13 21:25:52 -05:00
Tom Lane
5d924f067c Clarify CREATE FUNCTION documentation about handling of typmods.
The previous text was a bit misleading, as well as unnecessarily vague
about what information would be discarded.  Per gripe from Craig Skinner.
2013-11-13 13:30:15 -05:00
Andrew Dunstan
869b1e4a67 Fix isolation check for MSVC to handle recent changes. 2013-11-13 12:59:48 -05:00
Robert Haas
c46c803f8a Fix relfilenodemap.c's handling of cache invalidations.
The old code entered a new hash table entry first, then scanned
pg_class to determine what value to fill in, and then populated the
entry.  This fails to work properly if a cache invalidation happens
as a result of opening pg_class.  Repair.

Along the way, get rid of the idea of blowing away the entire hash
table as a method of processing invalidations.  Instead, just delete
all the entries one by one.  This is probably not quite as cheap but
it's simpler, and shouldn't happen often.

Andres Freund
2013-11-13 10:52:59 -05:00
Bruce Momjian
cd8115e009 docs: clarify MVCC introduction to allow for per-statement snapshots 2013-11-13 10:14:08 -05:00
Kevin Grittner
fe67d25233 Free ignorelist after each regression test schedule.
It's a trivial amount of RAM held until the end of the regression
test run; but it's probably worth fixing to silence future warnings
from code analyzers.

This was the only memory leak pointed out by clang's static code
analysis tool.
2013-11-13 09:01:06 -06:00
Heikki Linnakangas
07fca603b5 Fix bug in GIN posting tree root creation.
The root page is filled with as many items as fit, and the rest are inserted
using normal insertions. However, I fumbled the variable names, and the code
actually memcpy'd all the items on the page, overflowing the buffer. While
at it, rename the variable to make the distinction more clear.

Reported by Teodor Sigaev. This bug was introduced by my recent
refactorings, so no backpatching required.
2013-11-13 13:47:59 +02:00
Peter Eisentraut
aa04b323c3 Move variable closer to where it is used
This avoids an unused variable warning on Windows when building without
asserts

From: David Rowley <dgrowleyml@gmail.com>
2013-11-13 06:26:27 -05:00
Peter Eisentraut
c0764a5425 gitattributes: Make syntax compatible with older Git versions
Avoid the use of **, which was only introduced in Git version 1.8.2.
2013-11-12 21:58:46 -05:00
Robert Haas
061b88c732 Try again to make pg_isolation_regress work its build directory.
We can't search for the isolationtester binary until after we've set
up the environment, because otherwise when find_other_exec() tries
to invoke it with the -V option, it might fail for inability to
locate a working libpq.  So postpone that step.

Andres Freund
2013-11-12 11:23:47 -05:00
Robert Haas
9cab81b572 doc: Fix typo.
Reported by Thom Brown.
2013-11-12 10:24:43 -05:00
Magnus Hagander
f1d6875916 Fix doc links in README file to work with new website layout
Per report from Colin 't Hart
2013-11-12 12:53:32 +01:00
Peter Eisentraut
3626adf266 Remove leftovers of IRIX port
This removes the remaining pieces of the IRIX port that was removed by
ea91a6be89.
2013-11-12 06:39:36 -05:00
Tom Lane
ebefbb5fde Fix failure with whole-row reference to a subquery.
Simple oversight in commit 1cb108efb0 ---
recursively examining a subquery output column is only sane if the
original Var refers to a single output column.  Found by Kevin Grittner.
2013-11-11 16:36:27 -05:00
Tom Lane
0b7e660d6c Fix ruleutils pretty-printing to not generate trailing whitespace.
The pretty-printing logic in ruleutils.c operates by inserting a newline
and some indentation whitespace into strings that are already valid SQL.
This naturally results in leaving some trailing whitespace before the
newline in many cases; which can be annoying when processing the output
with other tools, as complained of by Joe Abbate.  We can fix that in
a pretty localized fashion by deleting any trailing whitespace before
we append a pretty-printing newline.  In addition, we have to modify the
code inserted by commit 2f582f76b1 so that
we also delete trailing whitespace when transposing items from temporary
buffers into the main result string, when a temporary item starts with a
newline.

This results in rather voluminous changes to the regression test results,
but it's easily verified that they are only removal of trailing whitespace.

Back-patch to 9.3, because the aforementioned commit resulted in many
more cases of trailing whitespace than had occurred in earlier branches.
2013-11-11 13:36:38 -05:00
Tom Lane
648bd05b13 Re-allow duplicate aliases within aliased JOINs.
Although the SQL spec forbids duplicate table aliases, historically
we've allowed queries like
    SELECT ... FROM tab1 x CROSS JOIN (tab2 x CROSS JOIN tab3 y) z
on the grounds that the aliased join (z) hides the aliases within it,
therefore there is no conflict between the two RTEs named "x".  The
LATERAL patch broke this, on the misguided basis that "x" could be
ambiguous if tab3 were a LATERAL subquery.  To avoid breaking existing
queries, it's better to allow this situation and complain only if
tab3 actually does contain an ambiguous reference.  We need only remove
the check that was throwing an error, because the column lookup code
is already prepared to handle ambiguous references.  Per bug #8444.
2013-11-11 10:42:57 -05:00
Magnus Hagander
705556a631 Don't abort pg_basebackup when receiving empty WAL block
This is a similar fix as c6ec8793aa
9.2. This should never happen in 9.3 and newer since the special case
cannot happen there, but this patch synchronizes up the code so there
is no confusion on why they're different. An empty block is as harmless
in 9.3 as it was in 9.2, and can safely be ignored.
2013-11-11 14:59:55 +01:00
Peter Eisentraut
001e114b8d Fix whitespace issues found by git diff --check, add gitattributes
Set per file type attributes in .gitattributes to fine-tune whitespace
checks.  With the associated cleanups, the tree is now clean for git
2013-11-10 14:48:29 -05:00
Robert Haas
dca09ac533 Fix ECPG compiler warning.
Commit 9b4d52f209 failed to notice
that pg_regress_ecpg needed updating.

This patch was independently submitted by both David Rowley
and Andres Freund.
2013-11-09 18:53:57 -05:00
Heikki Linnakangas
ac4ab97ec0 Fix race condition in GIN posting tree page deletion.
If a page is deleted, and reused for something else, just as a search is
following a rightlink to it from its left sibling, the search would continue
scanning whatever the new contents of the page are. That could lead to
incorrect query results, or even something more curious if the page is
reused for a different kind of a page.

To fix, modify the search algorithm to lock the next page before releasing
the previous one, and refrain from deleting pages from the leftmost branch
of the tree.

Add a new Concurrency section to the README, explaining why this works.
There is a lot more one could say about concurrency in GIN, but that's for
another patch.

Backpatch to all supported versions.
2013-11-08 22:21:42 +02:00
Robert Haas
636b868f17 doc: Clarify under what circumstances pg_dump needs superuser access.
Inspired by, but different from, a patch from Ivan Lezhnjov IV
2013-11-08 15:08:11 -05:00
Robert Haas
9b4d52f209 Fix pg_isolation_regress to work outside its build directory.
This makes it possible to, for example, use the isolation tester to
test a contrib module.

Andres Freund
2013-11-08 14:40:41 -05:00
Robert Haas
07cacba983 Add the notion of REPLICA IDENTITY for a table.
Pending patches for logical replication will use this to determine
which columns of a tuple ought to be considered as its candidate key.

Andres Freund, with minor, mostly cosmetic adjustments by me
2013-11-08 12:30:43 -05:00
Tom Lane
b97ee66cc1 Make contain_volatile_functions/contain_mutable_functions look into SubLinks.
This change prevents us from doing inappropriate subquery flattening in
cases such as dangerous functions hidden inside a sub-SELECT in the
targetlist of another sub-SELECT.  That could result in unexpected behavior
due to multiple evaluations of a volatile function, as in a recent
complaint from Etienne Dube.  It's been questionable from the very
beginning whether these functions should look into subqueries (as noted in
their comments), and this case seems to provide proof that they should.

Because the new code only descends into SubLinks, not SubPlans or
InitPlans, the change only affects the planner's behavior during
prepjointree processing and not later on --- for example, you can still get
it to use a volatile function in an indexqual if you wrap the function in
(SELECT ...).  That's a historical behavior, for sure, but it's reasonable
given that the executor's evaluation rules for subplans don't depend on
whether there are volatile functions inside them.  In any case, we need to
constrain the behavioral change as narrowly as we can to make this
reasonable to back-patch.
2013-11-08 11:36:57 -05:00
Tom Lane
060b22a99a Fix subtly-wrong volatility checking in BeginCopyFrom().
contain_volatile_functions() is best applied to the output of
expression_planner(), not its input, so that insertion of function
default arguments and constant-folding have been done.  (See comments
at CheckMutability, for instance.)  It's perhaps unlikely that anyone
will notice a difference in practice, but still we should do it properly.

In passing, change variable type from Node* to Expr* to reduce the net
number of casts needed.

Noted while perusing uses of contain_volatile_functions().
2013-11-08 08:59:39 -05:00
Tom Lane
20803d7881 Make LOCK_PRINT & PROCLOCK_PRINT expand to ((void) 0) when not in use.
This avoids warnings from more-anal-than-average compilers, and might
prevent hidden syntax problems in the future.

Andres Freund
2013-11-07 19:07:48 -05:00