Commit Graph

23036 Commits

Author SHA1 Message Date
Peter Eisentraut 4e1c72079a Run maintainer-check on all PO files, not only configured ones
The intent is to allow configure --enable-nls=xx for installation
speed and size, but have maintainer-check check all source files
regardless.
2012-03-28 21:06:45 +03:00
Robert Haas 7f63527c82 Attempt to unbreak pg_test_timing on Windows.
Per buildfarm, and Álvaro Herrera.
2012-03-28 12:22:57 -04:00
Robert Haas ada763cfcd pg_basebackup: Error handling fixes.
Thomas Ogrisegg and Fujii Masao
2012-03-28 12:19:22 -04:00
Robert Haas 81f6bbe8ad pg_basebackup: Error message improvements.
Fujii Masao
2012-03-28 08:43:35 -04:00
Tom Lane 5d3fcc4c2e Bend parse location rules for the convenience of pg_stat_statements.
Generally, the parse location assigned to a multiple-token construct is
the location of its leftmost token.  This commit breaks that rule for
the syntaxes TYPENAME 'LITERAL' and CAST(CONSTANT AS TYPENAME) --- the
resulting Const will have the location of the literal string, not the
typename or CAST keyword.  The cases where this matters are pretty thin on
the ground (no error messages in the regression tests change, for example),
and it's unlikely that any user would be confused anyway by an error cursor
pointing at the literal.  But still it's less than consistent.  The reason
for changing it is that contrib/pg_stat_statements wants to know the parse
location of the original literal, and it was agreed that this is the least
unpleasant way to preserve that information through parse analysis.

Peter Geoghegan
2012-03-27 15:17:41 -04:00
Tom Lane a40fa613b5 Add some infrastructure for contrib/pg_stat_statements.
Add a queryId field to Query and PlannedStmt.  This is not used by the
core backend, except for being copied around at appropriate times.
It's meant to allow plug-ins to track a particular query forward from
parse analysis to execution.

The queryId is intentionally not dumped into stored rules (and hence this
commit doesn't bump catversion).  You could argue that choice either way,
but it seems better that stored rule strings not have any dependency
on plug-ins that might or might not be present.

Also, add a post_parse_analyze_hook that gets invoked at the end of
parse analysis (but only for top-level analysis of complete queries,
not cases such as analyzing a domain's default-value expression).
This is mainly meant to be used to compute and assign a queryId,
but it could have other applications.

Peter Geoghegan
2012-03-27 15:17:40 -04:00
Robert Haas 40b9b95769 New GUC, track_iotiming, to track I/O timings.
Currently, the only way to see the numbers this gathers is via
EXPLAIN (ANALYZE, BUFFERS), but the plan is to add visibility through
the stats collector and pg_stat_statements in subsequent patches.

Ants Aasma, reviewed by Greg Smith, with some further changes by me.
2012-03-27 14:55:02 -04:00
Peter Eisentraut dd024c22f1 pg_dump: Small message adjustment for consistency 2012-03-27 19:52:39 +03:00
Peter Eisentraut dcb33b1c64 Remove dead assignment
found by Coverity
2012-03-26 21:03:10 +03:00
Robert Haas 7386089d23 Code cleanup for heap_freeze_tuple.
It used to be case that lazy vacuum could call this function with only
a shared lock on the buffer, but neither lazy vacuum nor any other
code path does that any more.  Simplify the code accordingly and clean
up some related, obsolete comments.
2012-03-26 11:03:06 -04:00
Tom Lane e8476f46fc Fix COPY FROM for null marker strings that correspond to invalid encoding.
The COPY documentation says "COPY FROM matches the input against the null
string before removing backslashes".  It is therefore reasonable to presume
that null markers like E'\\0' will work ... and they did, until someone put
the tests in the wrong order during microoptimization-driven rewrites.
Since then, we've been failing if the null marker is something that would
de-escape to an invalidly-encoded string.  Since null markers generally
need to be something that can't appear in the data, this represents a
nontrivial loss of functionality; surprising nobody noticed it earlier.

Per report from Jeff Davis.  Backpatch to 8.4 where this got broken.
2012-03-25 23:17:22 -04:00
Tom Lane c7cea267de Replace empty locale name with implied value in CREATE DATABASE and initdb.
setlocale() accepts locale name "" as meaning "the locale specified by the
process's environment variables".  Historically we've accepted that for
Postgres' locale settings, too.  However, it's fairly unsafe to store an
empty string in a new database's pg_database.datcollate or datctype fields,
because then the interpretation could vary across postmaster restarts,
possibly resulting in index corruption and other unpleasantness.

Instead, we should expand "" to whatever it means at the moment of calling
CREATE DATABASE, which we can do by saving the value returned by
setlocale().

For consistency, make initdb set up the initial lc_xxx parameter values the
same way.  initdb was already doing the right thing for empty locale names,
but it did not replace non-empty names with setlocale results.  On a
platform where setlocale chooses to canonicalize the spellings of locale
names, this would result in annoying inconsistency.  (It seems that popular
implementations of setlocale don't do such canonicalization, which is a
pity, but the POSIX spec certainly allows it to be done.)  The same risk
of inconsistency leads me to not venture back-patching this, although it
could certainly be seen as a longstanding bug.

Per report from Jeff Davis, though this is not his proposed patch.
2012-03-25 21:47:22 -04:00
Tom Lane 8279eb4191 Fix planner's handling of outer PlaceHolderVars within subqueries.
For some reason, in the original coding of the PlaceHolderVar mechanism
I had supposed that PlaceHolderVars couldn't propagate into subqueries.
That is of course entirely possible.  When it happens, we need to treat
an outer-level PlaceHolderVar much like an outer Var or Aggref, that is
SS_replace_correlation_vars() needs to replace the PlaceHolderVar with
a Param, and then when building the finished SubPlan we have to provide
the PlaceHolderVar expression as an actual parameter for the SubPlan.
The handling of the contained expression is a bit delicate but it can be
treated exactly like an Aggref's expression.

In addition to the missing logic in subselect.c, prepjointree.c was failing
to search subqueries for PlaceHolderVars that need their relids adjusted
during subquery pullup.  It looks like everyplace else that touches
PlaceHolderVars got it right, though.

Per report from Mark Murawski.  In 9.1 and HEAD, queries affected by this
oversight would fail with "ERROR: Upper-level PlaceHolderVar found where
not expected".  But in 9.0 and 8.4, you'd silently get possibly-wrong
answers, since the value transmitted into the subquery wouldn't go to null
when it should.
2012-03-24 16:21:39 -04:00
Tom Lane ed61127be4 Cast some printf arguments to avoid possibly-nonportable behavior.
Per compiler warnings on buildfarm member black_firefly.
2012-03-23 20:18:04 -04:00
Tom Lane 81a646febe Refactor simplify_function et al to centralize argument simplification.
We were doing the recursive simplification of function/operator arguments
in half a dozen different places, with rather baroque logic to ensure it
didn't get done multiple times on some arguments.  This patch improves that
by postponing argument simplification until after we've dealt with named
parameters and added any needed default expressions.

Marti Raudsepp, somewhat hacked on by me
2012-03-23 19:15:58 -04:00
Tom Lane 0339047bc9 Code review for protransform patches.
Fix loss of previous expression-simplification work when a transform
function fires: we must not simply revert to untransformed input tree.
Instead build a dummy FuncExpr node to pass to the transform function.
This has the additional advantage of providing a simpler, more uniform
API for transform functions.

Move documentation to a somewhat less buried spot, relocate some
poorly-placed code, be more wary of null constants and invalid typmod
values, add an opr_sanity check on protransform function signatures,
and some other minor cosmetic adjustments.

Note: although this patch touches pg_proc.h, no need for catversion
bump, because the changes are cosmetic and don't actually change the
intended catalog contents.
2012-03-23 17:29:57 -04:00
Tom Lane 88a4cb30a4 Fix GET DIAGNOSTICS for case of assignment to function's first variable.
An incorrect and entirely unnecessary "safety check" in exec_stmt_getdiag()
caused the code to treat an assignment to a variable with dno zero as a
no-op.  Unfortunately, that's a perfectly valid dno.  This has been broken
since GET DIAGNOSTICS was invented.  It's not terribly surprising that the
bug went unnoticed for so long, since in most cases you probably wouldn't
use the function's first-created variable (normally its first parameter)
as a GET DIAGNOSTICS target.  Nonetheless, it's broken.  Per bug #6551
from Adam Buraczewski.
2012-03-22 14:13:17 -04:00
Tom Lane e9ce658b62 Refactor to eliminate duplicate copies of conninfo default-finding code.
Alex Shulgin, lightly edited by me
2012-03-22 12:08:34 -04:00
Tom Lane 04dfc87749 If a role has a password expiration date, show that in psql's \du output.
Per a suggestion from Euler Taveira, it seems like a good idea to include
this information in \du (and \dg) output.  This costs nothing for people
who are not using the VALID UNTIL feature, while for those who are, it's
rather critical information.

Fabrízio de Royes Mello
2012-03-22 02:08:25 -04:00
Peter Eisentraut 0e85abd658 Clean up compiler warnings from unused variables with asserts disabled
For those variables only used when asserts are enabled, use a new
macro PG_USED_FOR_ASSERTS_ONLY, which expands to
__attribute__((unused)) when asserts are not enabled.
2012-03-21 23:33:10 +02:00
Peter Eisentraut 621eb156f1 Add installing entab to pgindent instructions
And minor other pgindent documentation tweaks.
2012-03-21 23:33:10 +02:00
Tom Lane f70f095c90 Allow new relmapper entries when allow_system_table_mods is true.
This restores the pre-9.0 situation that it's possible to add new indexes
on pg_class and other mapped-but-not-shared catalogs, so long as you broke
the glass and flipped the big red Dont-Touch-Me switch.  As before, there
are a lot of gotchas, and you'd have to be pretty desperate to try this
on a production database; but there doesn't seem to be a reason for
relmapper.c to be preventing such things all by itself.  Per
experimentation with a case suggested by Cody Cutrer.
2012-03-21 14:09:39 -04:00
Robert Haas 99081c63e4 Improve connectMaintenanceDatabase() error reporting.
The prior coding instructs the user to pick an alternative maintenance
database, but this is overly clever, since it obscures whatever the real
cause of the failure is.

Josh Kupershmidt
2012-03-21 10:56:26 -04:00
Robert Haas aefa6d163e Add some CHECK_FOR_INTERRUPTS() calls to the heap-sort call path.
I broke this in commit 337b6f5ecf, which
among other things arranged for quicksorts to CHECK_FOR_INTERRUPTS()
slightly less frequently.  Sadly, it also arranged for heapsorts to
CHECK_FOR_INTERRUPTS() much less frequently.  Repair.
2012-03-20 21:26:39 -04:00
Alvaro Herrera 9d23a70d51 pg_dump: get rid of die_horribly
The old code was using exit_horribly or die_horribly other depending on
whether it had an ArchiveHandle on which to close the connection or not;
but there were places that were passing a NULL ArchiveHandle to
die_horribly, and other places that used exit_horribly while having an
AH available.  So there wasn't all that much consistency.

Improve the situation by keeping only one of the routines, and instead
of having to pass the AH down from the caller, arrange for it to be
present for an on_exit_nicely callback to operate on.

Author: Joachim Wieland
Some tweaks by me

Per a suggestion from Robert Haas, in the ongoing "parallel pg_dump"
saga.
2012-03-20 18:58:00 -03:00
Peter Eisentraut 19f45565f5 pg_dump: Remove undocumented "files" output format
This was for demonstration only, and now it was creating compiler
warnings from zlib without an obvious fix (see also
d923125b77), let's just remove it.  The
"directory" format is presumably similar enough anyway.
2012-03-20 20:39:59 +02:00
Tom Lane 9dbf2b7d75 Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements.  The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.

In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs.  Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.

Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.

Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn".  There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.

Andres Freund and Tom Lane
2012-03-19 21:38:12 -04:00
Alvaro Herrera 77503a7638 pg_dump: fix double free of query results
This bug was introduced while refactoring in commit 1631598e --- no need
to back-patch.

Bug report and fix from Joachim Wieland.
2012-03-19 17:55:51 -03:00
Alvaro Herrera 814e08e895 plperl: Package-qualify _TD
Failing to do so causes trigger invocation to fail when they are nested
within a function invocation that changes the current package.

Backpatch to 9.1; previous releases used a different method to obtain
_TD.  Per bug report from Mark Murawski (bug #6511)

Author: Alex Hunsaker
2012-03-19 17:29:05 -03:00
Andrew Dunstan e3fc4a97bc Honor inputdir and outputdir when converting regression files.
When converting source files, pg_regress' inputdir and outputdir options were
ignored when computing the locations of the destination files. In consequence,
these options were effectively unusable when the regression inputs need to
be adjusted by pg_regress. This patch makes pg_regress put the converted files
in the same place that these options specify non-converted input or results
files are to be found. Backpatched to all live branches.
2012-03-17 17:24:15 -04:00
Peter Eisentraut d4318483e1 libpq: Fix minor memory leaks
When using connection info arrays with a conninfo string in the dbname
slot, some memory would be leaked if an error occurred while
processing the following array slots.

found by Coverity
2012-03-16 20:35:00 +02:00
Peter Eisentraut 598bb8cdbd psql: Remove inappropriate const qualifiers
Since mbvalidate() can alter the string it validates, having the
callers claim that the strings they accept are const is inappropriate.
2012-03-16 20:35:00 +02:00
Peter Eisentraut 359dea2859 pg_dump: Fix crash with invalid pg_cast row
An invalid combination of pg_cast.castfunc and pg_cast.castmethod
would result in a segmentation fault.  Now it prints a warning.

found by Coverity
2012-03-16 20:34:59 +02:00
Peter Eisentraut d8e5b3427a pg_restore: Fix memory and file descriptor leak with directory format
found by Coverity
2012-03-16 20:34:59 +02:00
Peter Eisentraut 693ff85d47 backend: Fix minor memory leak in configuration file processing
Just for consistency with the other code paths.

found by Coverity
2012-03-16 20:34:59 +02:00
Tom Lane b67ad046e6 Improve commentary in match_pathkeys_to_index().
For a little while there I thought match_pathkeys_to_index() was broken
because it wasn't trying to match index columns to pathkeys in order.
Actually that's correct, because GiST can support ordering operators
on any random collection of index columns, but it sure needs a comment.
2012-03-16 14:07:21 -04:00
Tom Lane dd4134ea56 Revisit handling of UNION ALL subqueries with non-Var output columns.
In commit 57664ed25e I tried to fix a bug
reported by Teodor Sigaev by making non-simple-Var output columns distinct
(by wrapping their expressions with dummy PlaceHolderVar nodes).  This did
not work too well.  Commit b28ffd0fcc fixed
some ensuing problems with matching to child indexes, but per a recent
report from Claus Stadler, constraint exclusion of UNION ALL subqueries was
still broken, because constant-simplification didn't handle the injected
PlaceHolderVars well either.  On reflection, the original patch was quite
misguided: there is no reason to expect that EquivalenceClass child members
will be distinct.  So instead of trying to make them so, we should ensure
that we can cope with the situation when they're not.

Accordingly, this patch reverts the code changes in the above-mentioned
commits (though the regression test cases they added stay).  Instead, I've
added assorted defenses to make sure that duplicate EC child members don't
cause any problems.  Teodor's original problem ("MergeAppend child's
targetlist doesn't match MergeAppend") is addressed more directly by
revising prepare_sort_from_pathkeys to let the parent MergeAppend's sort
list guide creation of each child's sort list.

In passing, get rid of add_sort_column; as far as I can tell, testing for
duplicate sort keys at this stage is dead code.  Certainly it doesn't
trigger often enough to be worth expending cycles on in ordinary queries.
And keeping the test would've greatly complicated the new logic in
prepare_sort_from_pathkeys, because comparing pathkey list entries against
a previous output array requires that we not skip any entries in the list.

Back-patch to 9.1, like the previous patches.  The only known issue in
this area that wasn't caused by the ill-advised previous patches was the
MergeAppend planning failure, which of course is not relevant before 9.1.
It's possible that we need some of the new defenses against duplicate child
EC entries in older branches, but until there's some clear evidence of that
I'm going to refrain from back-patching further.
2012-03-16 13:11:55 -04:00
Heikki Linnakangas aef5fe7efe Add comments explaining why our Itanium spinlock implementation is safe. 2012-03-16 10:14:45 +02:00
Peter Eisentraut eb990a2b9e Add const qualifier to tzn returned by timestamp2tm()
The tzn value might come from tm->tm_zone, which libc declares as
const, so it's prudent that the upper layers know about this as well.
2012-03-15 21:17:19 +02:00
Peter Eisentraut 531e60aec0 Remove unused tzn arguments for timestamp2tm() 2012-03-15 21:13:35 +02:00
Peter Eisentraut ad4fb0d0d2 Improve EncodeDateTime and EncodeTimeOnly APIs
Use an explicit argument to tell whether to include the time zone in
the output, rather than using some undocumented pointer magic.
2012-03-14 23:03:34 +02:00
Peter Eisentraut 942b63193c Add missing va_end() calls
found by Coverity
2012-03-14 22:47:21 +02:00
Peter Eisentraut 6f018c6dda COPY: Add an assertion
This is for tools such as Coverity that don't know that the grammar
enforces that the case of not having a relation (but instead a query)
cannot happen in the FROM case.
2012-03-14 22:44:40 +02:00
Peter Eisentraut e684ab5e1e Add additional safety check against invalid backup label file
It was already checking for invalid data after "BACKUP FROM", but
would possibly crash if "BACKUP FROM" was missing altogether.

found by Coverity
2012-03-14 22:41:50 +02:00
Peter Eisentraut acfaa596cc pg_dump: Fix some minor memory leaks
Although we often don't care about freeing all memory in pg_dump,
these functions already freed the same memory in other code paths, so
we might as well do it consistently.

found by Coverity
2012-03-13 21:36:43 +02:00
Tom Lane 5cd72c7a7c Patch some corner-case bugs in pl/python.
Dave Malcolm of Red Hat is working on a static code analysis tool for
Python-related C code.  It reported a number of problems in plpython,
most of which were failures to check for NULL results from object-creation
functions, so would only be an issue in very-low-memory situations.

Patch in HEAD and 9.1.  We could go further back but it's not clear that
these issues are important enough to justify the work.

Jan Urbański
2012-03-13 15:26:32 -04:00
Tom Lane a14fa84693 Fix minor memory leak in PLy_typeinfo_dealloc().
We forgot to free the per-attribute array element descriptors.

Jan Urbański
2012-03-13 13:28:11 -04:00
Tom Lane ed75380bda Create a stack of pl/python "execution contexts".
This replaces the former global variable PLy_curr_procedure, and provides
a place to stash per-call-level information.  In particular we create a
per-call-level scratch memory context.

For the moment, the scratch context is just used to avoid leaking memory
from datatype output function calls in PLyDict_FromTuple.  There probably
will be more use-cases in future.

Although this is a fix for a pre-existing memory leakage bug, it seems
sufficiently invasive to not want to back-patch; it feels better as part
of the major rearrangement of plpython code that we've already done as
part of 9.2.

Jan Urbański
2012-03-13 13:19:06 -04:00
Tom Lane b4af1c25bb Fix SPGiST vacuum algorithm to handle concurrent tuple motion properly.
A leaf tuple that we need to delete could get moved as a consequence of an
insertion happening concurrently with the VACUUM scan.  If it moves from a
page past the current scan point to a page before, we'll miss it, which is
not acceptable.  Hence, when we see a leaf-page REDIRECT that could have
been made since our scan started, chase down the redirection pointer much
as if we were doing a normal index search, and be sure to vacuum every page
it leads to.  This fixes the issue because, if the tuple was on page N at
the instant we start our scan, we will surely find it as a consequence of
chasing the redirect from page N, no matter how much it moves around in
between.  Problem noted by Takashi Yamamoto.
2012-03-12 16:10:28 -04:00
Peter Eisentraut bad250f4f3 Use correct sizeof operand in qsort call
Probably no practical impact, since all pointers ought to have the
same size, but it was wrong nonetheless.  Found by Coverity.
2012-03-12 20:56:13 +02:00
Peter Eisentraut c9f310d377 Add comment for missing break in switch
For clarity, following other sites, and to silence Coverity.
2012-03-12 20:55:09 +02:00
Tom Lane c6be1f43ab Make INSERT/UPDATE queries depend on their specific target columns.
We have always created a whole-table dependency for the target relation,
but that's not really good enough, as it doesn't prevent scenarios such
as dropping an individual target column or altering its type.  So we
have to create an individual dependency for each target column, as well.

Per report from Bill MacArthur of a rule containing UPDATE breaking
after such an alteration.  Note that this patch doesn't try to make
such cases work, only to ensure that the attempted ALTER TABLE throws
an error telling you it can't cope with adjusting the rule.

This is a long-standing bug, but given the lack of prior reports
I'm not going to risk back-patching it.  A back-patch wouldn't do
anything to fix existing rules' dependency lists, anyway.
2012-03-11 18:14:23 -04:00
Tom Lane c6a11b89e4 Teach SPGiST to store nulls and do whole-index scans.
This patch fixes the other major compatibility-breaking limitation of
SPGiST, that it didn't store anything for null values of the indexed
column, and so could not support whole-index scans or "x IS NULL"
tests.  The approach is to create a wholly separate search tree for
the null entries, and use fixed "allTheSame" insertion and search
rules when processing this tree, instead of calling the index opclass
methods.  This way the opclass methods do not need to worry about
dealing with nulls.

Catversion bump is for pg_am updates as well as the change in on-disk
format of SPGiST indexes; there are some tweaks in SPGiST WAL records
as well.

Heavily rewritten version of a patch by Oleg Bartunov and Teodor Sigaev.
(The original also stored nulls separately, but it reused GIN code to do
so; which required undesirable compromises in the on-disk format, and
would likely lead to bugs due to the GIN code being required to work in
two very different contexts.)
2012-03-11 16:29:59 -04:00
Peter Eisentraut 86947e666d Add more detail to error message for invalid arguments for server process
It now prints the argument that was at fault.

Also fix a small misbehavior where the error message issued by
getopt() would complain about a program named "--single", because
that's what argv[0] is in the server process.
2012-03-11 02:03:52 +02:00
Tom Lane 03e56f798e Restructure SPGiST opclass interface API to support whole-index scans.
The original API definition was incapable of supporting whole-index scans
because there was no way to invoke leaf-value reconstruction without
checking any qual conditions.  Also, it was inefficient for
multiple-qual-condition scans because value reconstruction got done over
again for each qual condition, and because other internal work in the
consistent functions likewise had to be done for each qual.  To fix these
issues, pass the whole scankey array to the opclass consistent functions,
instead of only letting them see one item at a time.  (Essentially, the
loop over scankey entries is now inside the consistent functions not
outside them.  This makes the consistent functions a bit more complicated,
but not unreasonably so.)

In itself this commit does nothing except save a few cycles in
multiple-qual-condition index scans, since we can't support whole-index
scans on SPGiST indexes until nulls are included in the index.  However,
I consider this a must-fix for 9.2 because once we release it will get
very much harder to change the opclass API definition.
2012-03-10 18:36:49 -05:00
Peter Eisentraut 39d74e346c Add support for renaming constraints
reviewed by Josh Berkus and Dimitri Fontaine
2012-03-10 20:19:13 +02:00
Robert Haas 07d1edb954 Extend object access hook framework to support arguments, and DROP.
This allows loadable modules to get control at drop time, perhaps for the
purpose of performing additional security checks or to log the event.
The initial purpose of this code is to support sepgsql, but other
applications should be possible as well.

KaiGai Kohei, reviewed by me.
2012-03-09 14:34:56 -05:00
Tom Lane b14953932d Revise FDW planning API, again.
Further reflection shows that a single callback isn't very workable if we
desire to let FDWs generate multiple Paths, because that forces the FDW to
do all work necessary to generate a valid Plan node for each Path.  Instead
split the former PlanForeignScan API into three steps: GetForeignRelSize,
GetForeignPaths, GetForeignPlan.  We had already bit the bullet of breaking
the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain,
and it's substantially more flexible for complex FDWs.

Add an fdw_private field to RelOptInfo so that the new functions can save
state there rather than possibly having to recalculate information two or
three times.

In addition, we'd not thought through what would be needed to allow an FDW
to set up subexpressions of its choice for runtime execution.  We could
treat ForeignScan.fdw_private as an executable expression but that seems
likely to break existing FDWs unnecessarily (in particular, it would
restrict the set of node types allowable in fdw_private to those supported
by expression_tree_walker).  Instead, invent a separate field fdw_exprs
which will receive the postprocessing appropriate for expression trees.
(One field is enough since it can be a list of expressions; also, we assume
the corresponding expression state tree(s) will be held within fdw_state,
so we don't need to add anything to ForeignScanState.)

Per review of Hanada Shigeru's pgsql_fdw patch.  We may need to tweak this
further as we continue to work on that patch, but to me it feels a lot
closer to being right now.
2012-03-09 12:49:25 -05:00
Heikki Linnakangas 342baf4ce6 Update outdated comment. HeapTupleHeader.t_natts field doesn't exist anymore.
Kevin Grittner
2012-03-09 08:07:56 +02:00
Peter Eisentraut 410ee35ed0 psql: Remove useless code
Apparently a copy-and-paste mistake introduced in
8ddd22f245.

found by Coverity
2012-03-08 23:15:03 +02:00
Tom Lane 08dd23cec7 Fix some issues with temp/transient tables in extension scripts.
Phil Sorber reported that a rewriting ALTER TABLE within an extension
update script failed, because it creates and then drops a placeholder
table; the drop was being disallowed because the table was marked as an
extension member.  We could hack that specific case but it seems likely
that there might be related cases now or in the future, so the most
practical solution seems to be to create an exception to the general rule
that extension member objects can only be dropped by dropping the owning
extension.  To wit: if the DROP is issued within the extension's own
creation or update scripts, we'll allow it, implicitly performing an
"ALTER EXTENSION DROP object" first.  This will simplify cases such as
extension downgrade scripts anyway.

No docs change since we don't seem to have documented the idea that you
would need ALTER EXTENSION DROP for such an action to begin with.

Also, arrange for explicitly temporary tables to not get linked as
extension members in the first place, and the same for the magic
pg_temp_nnn schemas that are created to hold them.  This prevents assorted
unpleasant results if an extension script creates a temp table: the forced
drop at session end would either fail or remove the entire extension, and
neither of those outcomes is desirable.  Note that this doesn't fix the
ALTER TABLE scenario, since the placeholder table is not temp (unless the
table being rewritten is).

Back-patch to 9.1.
2012-03-08 15:53:09 -05:00
Peter Eisentraut c5e073ca2d ecpg: Fix off-by-one error in memory copying
In a rare case, one byte past the end of memory belonging to the
sqlca_t structure would be written to.

found by Coverity
2012-03-08 22:29:01 +02:00
Peter Eisentraut 8dd4d10d48 ecpg: Fix rare memory leaks
found by Coverity
2012-03-08 22:21:12 +02:00
Heikki Linnakangas d93f209f48 Silence warning about unused variable, when building without assertions. 2012-03-08 11:10:02 +02:00
Tom Lane 66a7e6bae9 Improve estimation of IN/NOT IN by assuming array elements are distinct.
In constructs such as "x IN (1,2,3,4)" and "x <> ALL(ARRAY[1,2,3,4])",
we formerly always used a general-purpose assumption that the probability
of success is independent for each comparison of "x" to an array element.
But in real-world usage of these constructs, that's a pretty poor
assumption; it's much saner to assume that the array elements are distinct
and so the match probabilities are disjoint.  Apply that assumption if the
operator appears to behave as equality (for ANY) or inequality (for ALL).
But fall back to the normal independent-probabilities calculation if this
yields an impossible result, ie probability > 1 or < 0.  We could protect
ourselves against bad estimates even more by explicitly checking for equal
array elements, but that is expensive and doesn't seem worthwhile: doing
it would amount to optimizing for poorly-written queries at the expense
of well-written ones.

Daniele Varrazzo and Tom Lane, after a suggestion by Ants Aasma
2012-03-07 22:59:49 -05:00
Tom Lane 1ed7f0e6b9 Fix indentation of \d footers for non-ASCII cases.
Multi-line "Inherits:" and "Child tables:" footers were misindented when
those strings' translations involved multibyte characters, because we were
using strlen() instead of an appropriate display width measurement.

In passing, avoid doing gettext() more than once per loop in these places.

While at it, fix pg_wcswidth(), which has been entirely broken since about
8.2, but fortunately has been unused for the same length of time.

Report and patch by Sergey Burladyan (bug #6480)
2012-03-07 19:25:59 -05:00
Tom Lane 9088d1b965 Add GetForeignColumnOptions() to foreign.c, and add some documentation.
GetForeignColumnOptions provides some abstraction for accessing
column-specific FDW options, on a par with the access functions that were
already provided here for other FDW-related information.

Adjust file_fdw.c to use GetForeignColumnOptions instead of equivalent
hand-rolled code.

In addition, add some SGML documentation for the functions exported by
foreign.c that are meant for use by FDW authors.

(This is the fdw_helper portion of the proposed pgsql_fdw patch.)

Hanada Shigeru, reviewed by KaiGai Kohei
2012-03-07 18:20:58 -05:00
Robert Haas cf7026b64b psql: Avoid some spurious output if the server croaks.
Fixes a regression in commit 08146775ac.

Noah Misch
2012-03-07 16:56:42 -05:00
Peter Eisentraut 1673122127 psql: Fix memory leak
In expanded auto mode, a lot of allocated memory was not cleaned up.

found by Coverity
2012-03-07 23:52:15 +02:00
Peter Eisentraut 561ec76133 psql: Fix invalid memory access
Due to an apparent thinko, when printing a table in expanded mode
(\x), space would be allocated for 1 slot plus 1 byte per line,
instead of 1 slot per line plus 1 slot for the NULL terminator.  When
the line count is small, reading or writing the terminator would
therefore access memory beyond what was allocated.
2012-03-07 23:46:41 +02:00
Peter Eisentraut f9325df0fc libpq: Fix memory leak
If a client encoding is specified as a connection parameter (or
environment variable), internal storage allocated for it would never
be freed.
2012-03-07 23:35:03 +02:00
Tom Lane d4bf3c9c94 Expose an API for calculating catcache hash values.
Now that cache invalidation callbacks get only a hash value, and not a
tuple TID (per commits 632ae6829f and
b5282aa893), the only way they can restrict
what they invalidate is to know what the hash values mean.  setrefs.c was
doing this via a hard-wired assumption but that seems pretty grotty, and
it'll only get worse as more cases come up.  So let's expose a calculation
function that takes the same parameters as SearchSysCache.  Per complaint
from Marko Kreen.
2012-03-07 14:51:13 -05:00
Peter Eisentraut e685a8e665 libpq: Small code clarification, and avoid casting away const 2012-03-06 23:21:43 +02:00
Tom Lane 19dbc34631 Add a hook for processing messages due to be sent to the server log.
Use-cases for this include custom log filtering rules and custom log
message transmission mechanisms (for instance, lossy log message
collection, which has been discussed several times recently).

As is our common practice for hooks, there's no regression test nor
user-facing documentation for this, though the author did exhibit a
sample module using the hook.

Martin Pihlak, reviewed by Marti Raudsepp
2012-03-06 15:35:41 -05:00
Robert Haas bc97c38115 Typo fix.
Fujii Masao
2012-03-06 08:23:51 -05:00
Heikki Linnakangas e587e2e3e3 Make the comments more clear on the fact that UpdateFullPageWrites() is not
safe to call concurrently from multiple processes.
2012-03-06 10:45:58 +02:00
Heikki Linnakangas 7714c63829 Remove extra copies of LogwrtResult.
This simplifies the code a little bit. The new rule is that to update
XLogCtl->LogwrtResult, you must hold both WALWriteLock and info_lck, whereas
before we had two copies, one that was protected by WALWriteLock and another
protected by info_lck. The code that updates them was already holding both
locks, so merging the two is trivial.

The third copy, XLogCtl->Insert.LogwrtResult, was not totally redundant, it
was used in AdvanceXLInsertBuffer to update the backend-local copy, before
acquiring the info_lck to read the up-to-date value. But the value of that
seems dubious; at best it's saving one spinlock acquisition per completed
WAL page, which is not significant compared to all the other work involved.
And in practice, it's probably not saving even that much.
2012-03-06 10:18:33 +02:00
Heikki Linnakangas 3b682df326 Simplify the way changes to full_page_writes are logged.
It's harmless to do full page writes even when not strictly necessary, so
when turning full_page_writes on, we can set the global flag first, and then
call XLogInsert. Likewise, when turning it off, we can write the WAL record
first, and then clear the flag. This way XLogInsert doesn't need any special
handling of the XLOG_FPW_CHANGE record type. XLogInsert is complicated
enough already, so anything we can keep away from there is a good thing.

Actually I don't think the atomicity of the shared memory flag matters,
anyway, because we only write the XLOG_FPW_CHANGE at the end of recovery,
when there are no concurrent WAL insertions going on. But might as well make
it safe, in case we allow changing full_page_writes on the fly in the
future.
2012-03-06 09:48:30 +02:00
Tom Lane 6b289942bf Redesign PlanForeignScan API to allow multiple paths for a foreign table.
The original API specification only allowed an FDW to create a single
access path, which doesn't seem like a terribly good idea in hindsight.
Instead, move the responsibility for building the Path node and calling
add_path() into the FDW's PlanForeignScan function.  Now, it can do that
more than once if appropriate.  There is no longer any need for the
transient FdwPlan struct, so get rid of that.

Etsuro Fujita, Shigeru Hanada, Tom Lane
2012-03-05 16:15:59 -05:00
Peter Eisentraut cecdf6d459 Add isolation test to check-world and installcheck-world 2012-03-05 20:19:20 +02:00
Tom Lane 80da9e68fd Rewrite GiST support code for rangetypes.
This patch installs significantly smarter penalty and picksplit functions
for ranges, making GiST indexes for them smaller and faster to search.

There is no on-disk format change, so no catversion bump, but you'd need
to REINDEX to get the benefits for any existing index.

Alexander Korotkov, reviewed by Jeff Davis
2012-03-04 22:50:06 -05:00
Tom Lane e2eed78910 Remove useless "rough estimate" path from mcelem_array_contained_selec.
The code in this function that tried to cope with a missing count histogram
was quite ineffective for anything except a perfectly flat distribution.
Furthermore, since we were already punting for missing MCELEM slot, it's
rather useless to sweat over missing DECHIST: there are no cases where
ANALYZE will create the first but not the second.  So just simplify the
code by punting rather than pretending we can do something useful.
2012-03-04 16:03:38 -05:00
Tom Lane 4fb694aebc Improve histogram-filling loop in new compute_array_stats() code.
Do "frac" arithmetic in int64 to prevent overflow with large statistics
targets, and improve the comments so people have some chance of
understanding how it works.

Alexander Korotkov and Tom Lane
2012-03-04 15:40:16 -05:00
Magnus Hagander 141b89826d More carefully validate xlog location string inputs
Now that we have validate_xlog_location, call it from the previously
existing functions taking xlog locatoins as a string input.

Suggested by Fujii Masao
2012-03-04 12:25:47 +01:00
Magnus Hagander bc5ac36865 Add function pg_xlog_location_diff to help comparisons
Comparing two xlog locations are useful for example when calculating
replication lag.

Euler Taveira de Oliveira, reviewed by Fujii Masao, and some cleanups
from me
2012-03-04 12:22:38 +01:00
Tom Lane 0e5e167aae Collect and use element-frequency statistics for arrays.
This patch improves selectivity estimation for the array <@, &&, and @>
(containment and overlaps) operators.  It enables collection of statistics
about individual array element values by ANALYZE, and introduces
operator-specific estimators that use these stats.  In addition,
ScalarArrayOpExpr constructs of the forms "const = ANY/ALL (array_column)"
and "const <> ANY/ALL (array_column)" are estimated by treating them as
variants of the containment operators.

Since we still collect scalar-style stats about the array values as a
whole, the pg_stats view is expanded to show both these stats and the
array-style stats in separate columns.  This creates an incompatible change
in how stats for tsvector columns are displayed in pg_stats: the stats
about lexemes are now displayed in the array-related columns instead of the
original scalar-related columns.

There are a few loose ends here, notably that it'd be nice to be able to
suppress either the scalar-style stats or the array-element stats for
columns for which they're not useful.  But the patch is in good enough
shape to commit for wider testing.

Alexander Korotkov, reviewed by Noah Misch and Nathan Boley
2012-03-03 20:20:57 -05:00
Andrew Dunstan 34c978442c Provide environment overrides for psql file locations.
PSQL_HISTORY provides an alternative for the command history file,
and PSQLRC provides an alternative location for the .psqlrc file.
2012-03-03 16:39:26 -05:00
Peter Eisentraut b59ca98209 Allow CREATE TABLE (LIKE ...) from composite type
The only reason this didn't work before was that parserOpenTable()
rejects composite types.  So use relation_openrv() directly and
manually do the errposition() setup that parserOpenTable() does.
2012-03-03 16:03:05 +02:00
Peter Eisentraut d923125b77 Fix incorrect uses of gzFile
gzFile is already a pointer, so code like

gzFile *handle = gzopen(...)

is wrong.

This used to pass silently because gzFile used to be defined as void*,
and you can assign a void* to a void**.  But somewhere between zlib
versions 1.2.3.4 and 1.2.6, the definition of gzFile was changed to
struct gzFile_s *, and with that new definition this usage causes
compiler warnings.

So remove all those extra pointer decorations.

There is a related issue in pg_backup_archiver.h, where

FILE       *FH;             /* General purpose file handle */

is used throughout pg_dump as sometimes a real FILE* and sometimes a
gzFile handle, which also causes warnings now.  This is not yet fixed
here, because it might need more code restructuring.
2012-03-02 22:30:01 +02:00
Peter Eisentraut 8e5f4300fd Re-add "make check" target in src/test/isolation/Makefile
This effectively reverts 7886cc73ad,
which was done under the impression that isolationtester needs libpq,
which it no longer does (and never really did).
2012-03-02 22:11:57 +02:00
Tom Lane 44634e474f Allow child-relation entries to be made in ec_has_const EquivalenceClasses.
This fixes an oversight in commit 11cad29c91,
which introduced MergeAppend plans.  Before that happened, we never
particularly cared about the sort ordering of scans of inheritance child
relations, since appending their outputs together would destroy any
ordering anyway.  But now it's important to be able to match child relation
sort orderings to those of the surrounding query.  The original coding of
add_child_rel_equivalences skipped ec_has_const EquivalenceClasses, on the
originally-correct grounds that adding child expressions to them was
useless.  The effect of this is that when a parent variable is equated to
a constant, we can't recognize that index columns on the equivalent child
variables are not sort-significant; that is, we can't recognize that a
child index on, say, (x, y) is able to generate output in "ORDER BY y"
order when there is a clause "WHERE x = constant".  Adding child
expressions to the (x, constant) EquivalenceClass fixes this, without any
downside that I can see other than a few more planner cycles expended on
such queries.

Per recent gripe from Robert McGehee.  Back-patch to 9.1 where MergeAppend
was introduced.
2012-03-02 14:29:07 -05:00
Peter Eisentraut 6688d2878e Add COLLATION FOR expression
reviewed by Jaime Casanova
2012-03-02 21:12:16 +02:00
Peter Eisentraut d41f510c80 ecpg: Clean up some const usage 2012-03-02 20:51:29 +02:00
Heikki Linnakangas 2502f45979 When a GiST page is split during index build, it might not have a buffer.
Previously it was thought that it's impossible as the code stands, because
insertions create buffers as tuples are cascaded downwards, and index
split also creaters buffers eagerly for all halves. But the example from
Jay Levitt demonstrates that it can happen, when the root page is split.
It's in fact OK if the buffer doesn't exist, so we just need to remove the
sanity check. In fact, we've been discussing the possibility of destroying
empty buffers to conserve memory, which would render the sanity check
completely useless anyway.

Fix by Alexander Korotkov
2012-03-02 13:16:09 +02:00
Peter Eisentraut 36a1a8c33d Don't link pg_isolation_regress with libpq
It's not necessary and can only create confusion about which libpq
installation should be used.

Also remove some dead code from the makefile that was apparently
copied from elsewhere.
2012-03-01 20:51:59 +02:00
Peter Eisentraut 89c2f573a3 psql: Improve error display for psql -f -
Running "psql -f -" used to print

psql:<stdin>:1: ERROR:  blah

but that got broken between 8.4 and 9.0 (commit
b291c0fba8), and now it printed

psql:-:1: ERROR:  blah

This reverts to the old behavior and cleans up some code that was left
dead or useless by the mentioned commit.
2012-03-01 19:58:10 +02:00
Alvaro Herrera 3433c6ba00 Remove TOAST table from pg_database
The only toastable column now is datacl, but we don't really support
long ACLs anyway.  The TOAST table should have been removed when the
pg_db_role_setting catalog was introduced in commit
2eda8dfb52, but I forgot to do that.

Per -hackers discussion on March 2011.
2012-03-01 12:50:52 -03:00
Heikki Linnakangas d6a7271958 Correctly detect SSI conflicts of prepared transactions after crash.
A prepared transaction can get new conflicts in and out after preparing, so
we cannot rely on the in- and out-flags stored in the statefile at prepare-
time. As a quick fix, make the conservative assumption that after a restart,
all prepared transactions are considered to have both in- and out-conflicts.
That can lead to unnecessary rollbacks after a crash, but that shouldn't be
a big problem in practice; you don't want prepared transactions to hang
around for a long time anyway.

Dan Ports
2012-02-29 15:42:36 +02:00
Tom Lane 8cae5810eb Fix MSVC builds for previous patch's addition of a src/port file.
(And why in the world is this OBJS list not being scraped from the
corresponding Makefile?)
2012-02-29 00:24:01 -05:00
Alvaro Herrera 58e9f974dc Fix typo in comment
Haifeng Liu
2012-02-28 23:52:52 -03:00