Commit Graph

33692 Commits

Author SHA1 Message Date
Heikki Linnakangas
d1996ed5e8 Change the way parent pages are tracked during buffered GiST build.
We used to mimic the way a stack is constructed when descending the tree
during normal GiST inserts, but that was quite complicated during a buffered
build. It was also wrong: in GiST, the left-to-right relationships on
different levels might not match each other, so that when you know the
parent of a child page, you won't necessarily find the parent of the page to
the right of the child page by following the rightlinks at the parent level.
This sometimes led to "could not re-find parent" errors while building a
GiST index.

We now use a simple hash table to track the parent of every internal page.
Whenever a page is split, and downlinks are moved from one page to another,
we update the hash table accordingly. This is also better for performance
than the old method, as we never need to move right to re-find the parent
page, which could take a significant amount of time for buffers that were
created much earlier in the index build.
2012-05-30 12:05:57 +03:00
Heikki Linnakangas
be02b16826 Delete the temporary file used in buffered GiST build, after the build.
There were two bugs here: We forgot to call gistFreeBuildBuffers() function
at the end of build, and we passed interXact == true to BufFileCreateTemp,
so the file wasn't automatically cleaned up at end-of-transaction either.
2012-05-30 12:05:57 +03:00
Tom Lane
4317e0246c Rewrite --section option to decouple it from --schema-only/--data-only.
The initial implementation of pg_dump's --section option supposed that the
existing --schema-only and --data-only options could be made equivalent to
--section settings.  This is wrong, though, due to dubious but long since
set-in-stone decisions about where to dump SEQUENCE SET items, as seen in
bug report from Martin Pitt.  (And I'm not totally convinced there weren't
other bugs, either.)  Undo that coupling and instead drive --section
filtering off current-section state tracked as we scan through the TOC
list to call _tocEntryRequired().

To make sure those decisions don't shift around and hopefully save a few
cycles, run _tocEntryRequired() only once per TOC entry and save the result
in a new TOC field.  This required minor rejiggering of ACL handling but
also allows a far cleaner implementation of inhibit_data_for_failed_table.

Also, to ensure that pg_dump and pg_restore have the same behavior with
respect to the --section switches, add _tocEntryRequired() filtering to
WriteToc() and WriteDataChunks(), rather than trying to implement section
filtering in an entirely orthogonal way in dumpDumpableObject().  This
required adjusting the handling of the special ENCODING and STDSTRINGS
items, but they were pretty weird before anyway.

Minor other code review for the patch, too.
2012-05-29 23:22:14 -04:00
Heikki Linnakangas
4bc6fb57f7 Fix integer overflow bug in GiST buffering build calculations.
The result of (maintenance_work_mem * 1024) / BLCKSZ doesn't fit in a signed
32-bit integer, if maintenance_work_mem >= 2GB. Use double instead. And
while we're at it, write the calculations in an easier to understand form,
with the intermediary steps written out and commented.
2012-05-29 22:27:42 +03:00
Tom Lane
2755abf386 Teach AbortOutOfAnyTransaction to clean up partially-started transactions.
AbortOutOfAnyTransaction failed to do anything if the state it saw on
entry corresponded to failing partway through StartTransaction.  I fixed
AbortCurrentTransaction to cope with that case way back in commit
60b2444cc3, but evidently overlooked that
AbortOutOfAnyTransaction should do likewise.

Back-patch to all supported branches.  It's not clear that this omission
has any more-than-cosmetic consequences, but it's also not clear that it
doesn't, so back-patching seems the least risky choice.
2012-05-28 23:57:06 -04:00
Tom Lane
c89bdf7690 Eliminate some more O(N^2) behaviors in pg_dump/pg_restore.
This patch fixes three places (which AFAICT is all of them) where runtime
was O(N^2) in the number of TOC entries, by using an index array to replace
linear searches of the TOC list.  This performance issue is a bit less bad
than those recently fixed, because it depends on the number of items dumped
not the number in the source database, so the problem can be dodged by
doing partial dumps.

The previous coding already had an instance of one of the two index arrays
needed, but it was only calculated in parallel-restore cases; now we need
it all the time.  I also chose to move the arrays into the ArchiveHandle
data structure, to make this code a bit more ready for the day that we
try to sling multiple ArchiveHandles around in pg_dump or pg_restore.

Since we still need some server-side work before pg_dump can really cope
nicely with tens of thousands of tables, there's probably little point in
back-patching.
2012-05-28 20:38:28 -04:00
Peter Eisentraut
2d612abd4d libpq: URI parsing fixes
Drop special handling of host component with slashes to mean
Unix-domain socket.  Specify it as separate parameter or using
percent-encoding now.

Allow omitting username, password, and port even if the corresponding
designators are present in URI.

Handle percent-encoding in query parameter keywords.

Alex Shulgin

some documentation improvements by myself
2012-05-28 22:44:34 +03:00
Peter Eisentraut
388d251679 Update SQL features list
Set E081 Basic Privileges to supported, since by the letter of it, we
support it, even though not all possible forms of USAGE privileges are
implemented.
2012-05-27 23:34:16 +03:00
Peter Eisentraut
8e497c731b psql: Remove notice about readline from --version output
This was from a time when readline support wasn't standard.  And it
doesn't help analyzing current line editing library problems.
2012-05-27 22:48:20 +03:00
Peter Eisentraut
27314d32a8 Suppress -Wunused-result warning about write()
This is related to aa90e148ca, but this
code is only used under -DLINUX_OOM_ADJ, so it was apparently
overlooked then.
2012-05-27 22:35:01 +03:00
Peter Eisentraut
a8b92b6090 PL/Perl: Avoid compiler warning from clang
Use SvREFCNT_inc_simple_void() instead of SvREFCNT_inc() to avoid
warning about unused return value.
2012-05-27 22:30:34 +03:00
Bruce Momjian
4498a3407a Improve pg_upgrade C comment. 2012-05-27 06:45:29 -04:00
Bruce Momjian
20a3830b81 Add C comment explaining why we can't exclude checking functions in the
pg_catalog schema, even though they are not explicitly dumped (they are
implicitly dumped, e.g. create language plperl).
2012-05-27 06:35:10 -04:00
Bruce Momjian
d4696fa68f Add pg_update C comment about problems with plpython_call_handler(). 2012-05-27 06:29:45 -04:00
Magnus Hagander
16282ae688 Make pg_recievexlog by default loop on connection failures
Avoids the need for an external script in the most common
scenario. Behavior can be overridden using the -n/--noloop
commandline parameter.
2012-05-27 11:05:24 +02:00
Magnus Hagander
2b97db61dd Fix handling of pg_stat_statements.stat temporary file
Write the file to a temporary name and then rename() it into the
permanent name, to ensure it can't end up half-written and corrupt
in case of a crash during shutdown.

Unlink the file after it has been read so it's removed from the data
directory and not included in base backups going to replication slaves.
2012-05-27 10:54:31 +02:00
Tom Lane
532fe28dad Prevent synchronized scanning when systable_beginscan chooses a heapscan.
The only interesting-for-performance case wherein we force heapscan here
is when we're rebuilding the relcache init file, and the only such case
that is likely to be examining a catalog big enough to be syncscanned is
RelationBuildTupleDesc.  But the early-exit optimization in that code gets
broken if we start the scan at a random place within the catalog, so that
allowing syncscan is actually a big deoptimization if pg_attribute is large
(at least for the normal case where the rows for core system catalogs have
never been changed since initdb).  Hence, prevent syncscan here.  Per my
testing pursuant to complaints from Jeff Frost and Greg Sabino Mullane,
though neither of them seem to have actually hit this specific problem.

Back-patch to 8.3, where syncscan was introduced.
2012-05-26 19:09:52 -04:00
Tom Lane
d3b97d1488 Fix string truncation to be multibyte-aware in text_name and bpchar_name.
Previously, casts to name could generate invalidly-encoded results.

Also, make these functions match namein() more exactly, by consistently
using palloc0() instead of ad-hoc zeroing code.

Back-patch to all supported branches.

Karl Schnaitter and Tom Lane
2012-05-25 17:34:51 -04:00
Tom Lane
73cc7d3b24 Use binary search instead of brute-force scan in findNamespace().
The previous coding presented a significant bottleneck when dumping
databases containing many thousands of schemas, since the total time
spent searching would increase roughly as O(N^2) in the number of objects.
Noted by Jeff Janes, though I rewrote his proposed patch to use the
existing findObjectByOid infrastructure.

Since this is a longstanding performance bug, backpatch to all supported
versions.
2012-05-25 14:35:37 -04:00
Bruce Momjian
45ca31d6a7 Have pg_upgrade only use one extra log file for Win32, not two. 2012-05-25 09:10:07 -04:00
Magnus Hagander
31d965819b Fix base backup streaming xlog from standby
When backing up from a standby server, the backup process
will not automatically switch xlog segment. So we must
accept a partially transferred xlog file in this case, but
rename it into position anyway.

In passing, merge the two callbacks for segment end and
stop stream into a single callback, since their implementations
were close to identical, and rename this callback to
reflect that it stops streaming rather than continues it.

Patch by Magnus Hagander, review by Fujii Masao
2012-05-25 11:36:22 +02:00
Bruce Momjian
d226e236c6 On Windows, have pg_upgrade use different two files to log pg_ctl
start/stop output, to fix file share error reported by Edmund Horner
2012-05-24 20:30:39 -04:00
Bruce Momjian
32cebaaed5 Clarify 9.2 release notes items about pg_stat_statements, to better
document fix of double counting and read/write count addition, per Peter
Geoghegan
2012-05-24 19:10:06 -04:00
Bruce Momjian
7672366a89 Change pg_stat_statements order of release note items, per Peter
Geoghegan
2012-05-24 17:51:56 -04:00
Bruce Momjian
eeef7bd2da Remove PL/Perl null array 9.2 release note item, per Andrew Dunstan 2012-05-24 17:36:15 -04:00
Tom Lane
2a4c46e0ba Fix array overrun in regex code.
zaptreesubs() was coded to unconditionally reset a capture subre's
corresponding pmatch[] entry.  However, in regexes without backrefs, that
array is caller-supplied and might not have as many entries as the regex
has capturing parens.  So check the array length and do nothing if there
is no corresponding entry, much as subset() does.  Failure to check this
resulted in a stack clobber in the case reported by Marko Kreen.

This bug appears to have been latent in the regex library from the
beginning.  It was not exposed because find() called dissect() not
cdissect(), and the dissect() code path didn't ever call zaptreesubs()
(formerly zapmem()).  When I unified dissect() and cdissect() in commit
4dd78bf37a, the problem was exposed.

Now that I've seen this, I'm rather suspicious that we might need to
back-patch it; but will refrain for now, for lack of evidence that
the case can be hit in the previous coding.
2012-05-24 13:56:16 -04:00
Peter Eisentraut
ace397e9d2 Update SQL key word list to SQL:2011
For space reasons, drop SQL:1999 and SQL:2003.  Only keep the latest
two and SQL-92 for historical comparison.
2012-05-24 20:06:25 +03:00
Bruce Momjian
476291bef0 Adjust pg_upgrade to output a separate log file for pg_ctl output on
Windows, to avoid opening a file by multiple processes.
2012-05-23 20:19:34 -04:00
Magnus Hagander
77f93cb32d Add missing PQfinish() calls
Fujii Masao
2012-05-23 21:52:23 +02:00
Peter Eisentraut
72155b29a8 pg_standby: Remove tabs from string literals
And align a bit better with the rest of the debug output.
2012-05-23 19:58:17 +03:00
Bruce Momjian
db2cd07482 Mention Peter Geoghegan as primary author of pg_stat_statements changes. 2012-05-23 10:12:44 -04:00
Bruce Momjian
13477c0197 Improve wording of 9.2 clog release note item. 2012-05-23 09:29:39 -04:00
Bruce Momjian
b9c895ee6f Add 9.2 release note authorship mentions for Heikki and Alexander
Korotkov, per Alexander Korotkov.
2012-05-22 22:59:31 -04:00
Tom Lane
ed962fd712 Ensure that seqscans check for interrupts at least once per page.
If a seqscan encounters many consecutive pages containing only dead tuples,
it can remain in the loop in heapgettup for a long time, and there was no
CHECK_FOR_INTERRUPTS anywhere in that loop.  This meant there were
real-world situations where a query would be effectively uncancelable for
long stretches.  Add a check placed to occur once per page, which should be
enough to provide reasonable response time without adding any measurable
overhead.

Report and patch by Merlin Moncure (though I tweaked it a bit).
Back-patch to all supported branches.
2012-05-22 19:42:05 -04:00
Peter Eisentraut
92a953fbf8 Add small example about pg_archivecleanup -x option
Every time I read this I had doubts about whether the argument to the
-x option should include the dot (yes).  A small example should
clarify this.
2012-05-23 01:06:33 +03:00
Bruce Momjian
0013a665b7 Fix markup for HISTORY generation. 2012-05-22 17:36:33 -04:00
Bruce Momjian
e6b7170c33 Fix reindexdb manual page to say --maintenance-db controls what is
reindexed, not vacuumed (typo).  Per report from Thomas REISS
2012-05-22 17:31:46 -04:00
Bruce Momjian
007522e6b4 9.2 release notes: GiST index improvements are for all geometric index
types, per Alexander Korotkov
2012-05-22 17:26:42 -04:00
Bruce Momjian
8a46e25189 Remove reviewers from 9.2 release notes; improve attributions. 2012-05-22 17:17:15 -04:00
Robert Haas
8fbe5a317d Fix error message for COMMENT/SECURITY LABEL ON COLUMN xxx IS 'yyy'
When the column name is an unqualified name, rather than table.column,
the error message complains about too many dotted names, which is
wrong.  Report by Peter Eisentraut based on examination of the
sepgsql regression test output, but the problem also affects COMMENT.
New wording as suggested by Tom Lane.
2012-05-22 11:23:36 -04:00
Robert Haas
b536458e73 Release note improvements.
Document some more things as incompatibilities, and improve wording of
another item.

Noah Misch
2012-05-22 10:20:25 -04:00
Robert Haas
304aa339b2 Prevent pg_basebackup when integer_datetimes flag doesn't match.
Magnus Hagander, reviewed by Fujii Masao, with slight wording changes
by me.
2012-05-22 10:02:47 -04:00
Robert Haas
219c024c64 Repair out-of-date information in src/backend/storage/buffer/README.
In commit d526575f89, we changed things so
that buffer usage counts are incremented when the buffer is pinned, rather
than when it is unpinned, but the README file didn't get the memo.

Report by Amit Kapila.
2012-05-22 09:32:09 -04:00
Tom Lane
b94ce6e807 Move postmaster's RemovePgTempFiles call to a less randomly chosen place.
There is no reason to do this as early as possible in postmaster startup,
and good reason not to do it until we have completely created the
postmaster's lock file, namely that it might contribute to pg_ctl thinking
that postmaster startup has timed out.  (This would require a rather
unusual amount of time to be spent scanning temp file directories, but we
have at least one field report of it happening reproducibly.)

Back-patch to 9.1.  Before that, pg_ctl didn't wait for additional info to
be added to the lock file, so it wasn't a problem.

Note that this is not a complete fix to the slow-start issue in 9.1,
because we still had identify_system_timezone being run during postmaster
start in 9.1.  But that's at least a reasonably well-defined delay, with
an easy workaround if needed, whereas the temp-files scan is not so
predictable and cannot be avoided.
2012-05-21 22:50:30 -04:00
Tom Lane
efae4653c9 Update woefully-obsolete comment.
The accurate info about what's in a lock file has been in miscadmin.h
for some time, so let's just make this comment point there instead of
maintaining a duplicative copy.
2012-05-21 22:11:00 -04:00
Peter Eisentraut
cdf8bcb8d9 pg_ctl: Sort signal list in --help output
The list was neither logical nor numerical nor alphabetical.  Let's go
with alphabetical.
2012-05-21 20:12:30 +03:00
Peter Eisentraut
4c39a09089 libpq: Add missing file to GETTEXT_FILES list
For the record, fe-print.c is also missing, but it's sort of
deprecated, and the string internationalization there has some issues,
and it doesn't seem worth fixing that.  So let's leave that out.
2012-05-21 20:08:50 +03:00
Tom Lane
e79da56b85 Adjust documentation of ALTER TABLE CLUSTER ON for more consistency.
Josh Kupershmidt
2012-05-21 12:06:56 -04:00
Tom Lane
8be950f5c0 Document that we deviate from ISO 8601 by not using 'T' on output.
Per discussion, we should explain that we follow RFC 3339 and not really
the letter of the ISO 8601 spec for timestamp output format.  Mostly
Brendan Jurd's wording, though I tweaked it to clarify that we do take 'T'
on input.  Minor additional copy-editing and markup-tweaking, too.
2012-05-21 11:56:00 -04:00
Peter Eisentraut
f1f6737e15 Fix incorrect logic in JSON number lexer
Detectable by gcc -Wlogical-op.

Add two regression test cases that would previously allow incorrect
values to pass.
2012-05-20 02:24:46 +03:00