Commit Graph

10168 Commits

Author SHA1 Message Date
Magnus Hagander 7356381ef5 * make pg_hba authoption be a set of 0 or more name=value pairs
* make LDAP use this instead of the hacky previous method to specify
  the DN to bind as
* make all auth options behave the same when they are not compiled
  into the server
* rename "ident maps" to "user name maps", and support them for all
  auth methods that provide an external username

This makes a backwards incompatible change in the format of pg_hba.conf
for the ident, PAM and LDAP authentication methods.
2008-10-23 13:31:10 +00:00
Peter Eisentraut 2675d043b9 Feature T173 "Extended LIKE clause in table definition" is supported
(INCLUDING/EXCLUDING DEFAULTS)
2008-10-23 08:52:51 +00:00
Peter Eisentraut 9c9cb59ba0 Feature T401 is not listed in the SQL standard. Must have been a mistake. 2008-10-23 06:58:02 +00:00
Tom Lane 7f3eba30c9 When estimating without benefit of MCV lists (suggesting that one or both
inputs is unique or nearly so), make eqjoinsel() clamp the ndistinct estimates
to be not more than the estimated number of rows coming from the input
relations.  This allows the estimate to change in response to the selectivity
of restriction conditions on the inputs.

This is a pretty narrow patch and maybe we should be more aggressive about
similarly clamping ndistinct in other cases; but I'm worried about
double-counting the effects of the restriction conditions.  However, it seems
to help for the case exhibited by Grzegorz Jaskiewicz (antijoin against a
small subset of a relation), so let's try this for awhile.
2008-10-23 00:24:50 +00:00
Tom Lane 31468d05d8 Dept of better ideas: refrain from creating the planner's placeholder_list
until vars are distributed to rels during query_planner() startup.  We don't
really need it before that, and not building it early has some advantages.
First, we don't need to put it through the various preprocessing steps, which
saves some cycles and eliminates the need for a number of routines to support
PlaceHolderInfo nodes at all.  Second, this means one less unused plan for any
sub-SELECT appearing in a placeholder's expression, since we don't build
placeholder_list until after sublink expansion is complete.
2008-10-22 20:17:52 +00:00
Teodor Sigaev b9856b67a7 Fix GiST's killing tuple: GISTScanOpaque->curpos wasn't
correctly set. As result, killtuple() marks as dead
wrong tuple on page. Bug was introduced by me while fixing
possible duplicates during GiST index scan.
2008-10-22 12:53:56 +00:00
Peter Eisentraut 361bfc3572 SQL:2008 alternative syntax for LIMIT/OFFSET:
OFFSET num {ROW|ROWS} FETCH {FIRST|NEXT} [num] {ROW|ROWS} ONLY
2008-10-22 11:00:34 +00:00
Tom Lane e6ae3b5dbf Add a concept of "placeholder" variables to the planner. These are variables
that represent some expression that we desire to compute below the top level
of the plan, and then let that value "bubble up" as though it were a plain
Var (ie, a column value).

The immediate application is to allow sub-selects to be flattened even when
they are below an outer join and have non-nullable output expressions.
Formerly we couldn't flatten because such an expression wouldn't properly
go to NULL when evaluated above the outer join.  Now, we wrap it in a
PlaceHolderVar and arrange for the actual evaluation to occur below the outer
join.  When the resulting Var bubbles up through the join, it will be set to
NULL if necessary, yielding the correct results.  This fixes a planner
limitation that's existed since 7.1.

In future we might want to use this mechanism to re-introduce some form of
Hellerstein's "expensive functions" optimization, ie place the evaluation of
an expensive function at the most suitable point in the plan tree.
2008-10-21 20:42:53 +00:00
Peter Eisentraut d1b02e7648 Use format_type_be() instead of TypeNameToString() for some more user-facing
error messages where the type existence is established.
2008-10-21 10:38:51 +00:00
Peter Eisentraut 1471e3843d Allow SQL:2008 syntax ALTER TABLE ... ALTER COLUMN ... SET DATA TYPE
alongside our traditional syntax.
2008-10-21 08:38:16 +00:00
Alvaro Herrera 089ae3bc9a Properly access a buffer's LSN using existing access macros instead of abusing
knowledge of page layout.

Stolen from Jonah Harris' CRC patch
2008-10-20 21:11:15 +00:00
Alvaro Herrera 97227e9ec0 These functions no longer return a value, per complaint from gothic_moth via
Zdenek Kotala.
2008-10-20 20:38:24 +00:00
Alvaro Herrera 06da3c570f Rework subtransaction commit protocol for hot standby.
This patch eliminates the marking of subtransactions as SUBCOMMITTED in pg_clog
during their commit; instead they remain in-progress until main transaction
commit.  At main transaction commit, the commit protocol is atomic-by-page
instead of one transaction at a time.  To avoid a race condition with some
subtransactions appearing committed before others in the case where they span
more than one pg_clog page, we conserve the logic that marks them subcommitted
before marking the parent committed.

Simon Riggs with minor help from me
2008-10-20 19:18:18 +00:00
Teodor Sigaev 3afffbc902 Remove support of backward scan in GiST. Per discussion
http://archives.postgresql.org/pgsql-hackers/2008-10/msg00857.php
2008-10-20 16:35:14 +00:00
Peter Eisentraut a6ebb1f2f4 SQL 200N -> SQL:2003 2008-10-20 14:26:28 +00:00
Peter Eisentraut 0fd2756c19 Feature T411 is not found in SQL:2003 or 2008 anymore, so it must have been
dropped or it was a mistake.
2008-10-20 14:22:57 +00:00
Peter Eisentraut a3bf6d2cf5 Feature T152 "DISTINCT predicate with negation" is supported. 2008-10-20 13:58:18 +00:00
Teodor Sigaev 77db9d9ff2 Remove mark/restore support in GIN and GiST indexes.
Per Tom's comment.
Also revome useless GISTScanOpaque->flags field.
2008-10-20 13:39:44 +00:00
Peter Eisentraut 7f6bc33fe3 Feature F402 "Named column joins for LOBs, arrays, and multisets" is
supported, to the extent that LOBs, arrays, and multisets are supported.
2008-10-20 12:47:48 +00:00
Peter Eisentraut fa46050245 AS is no longer required in SELECT list 2008-10-20 12:09:46 +00:00
Tom Lane c6d05f81e0 Fix broken SQL features data, per buildfarm results. 2008-10-18 02:53:26 +00:00
Peter Eisentraut 123c8efd89 Update feature list for SQL:2008. 2008-10-18 00:35:32 +00:00
Tom Lane af59a0650b Remove useless mark/restore support in hash index AM, per discussion.
(I'm leaving GiST/GIN cleanup to Teodor.)
2008-10-17 23:50:57 +00:00
Alvaro Herrera 3e00496d88 Refactor some duplicate code to set up formatted_log_time and
formatted_start_time.
2008-10-17 22:56:16 +00:00
Tom Lane e4fb8ff06a Add a new column to pg_am to specify whether an index AM supports backward
scanning; GiST and GIN do not, and it seems like too much trouble to make
them do so.  By teaching ExecSupportsBackwardScan() about this restriction,
we ensure that the planner will protect a scroll cursor from the problem
by adding a Materialize node.

In passing, fix another longstanding bug in the same area: backwards scan of
a plan with set-returning functions in the targetlist did not work either,
since the TupFromTlist expansion code pays no attention to direction (and
has no way to run a SRF backwards anyway).  Again the fix is to make
ExecSupportsBackwardScan check this restriction.

Also adjust the index AM API specification to note that mark/restore support
is unnecessary if the AM can't produce ordered output.
2008-10-17 22:10:30 +00:00
Tom Lane 2a64931c4b Salvage a little bit of work from a failed patch: simplify and speed up
set_rel_width().  The code had been catering for the possibility of different
varnos in the relation targetlist, but this is impossible for a base relation
(and if it were possible, putting all the widths in the same RelOptInfo would
be wrong anyway).
2008-10-17 20:27:24 +00:00
Teodor Sigaev 2a0083ede8 Improve headeline generation. Now headline can contain
several fragments a-la Google.

Sushant Sinha <sushant354@gmail.com>
2008-10-17 18:05:19 +00:00
Teodor Sigaev 906b7e5f6c Fix small bug in headline generation.
Patch from Sushant Sinha <sushant354@gmail.com>
http://archives.postgresql.org/pgsql-hackers/2008-07/msg00785.php
2008-10-17 17:27:46 +00:00
Teodor Sigaev beeb3562dd During repeated rescan of GiST index it's possible that scan key
is NULL but SK_SEARCHNULL is not set. Add checking IS NULL of keys
to set during key initialization. If key is NULL and SK_SEARCHNULL is not
set then nothnig can be satisfied.
With assert-enabled compilation that causes coredump.

Bug was introduced in 8.3 by support of IS NULL index scan.
2008-10-17 17:02:21 +00:00
Neil Conway e034e517a7 Fix a small memory leak in ExecReScanAgg() in the hashed aggregation case.
In the previous coding, the list of columns that needed to be hashed on
was allocated in the per-query context, but we reallocated every time
the Agg node was rescanned. Since this information doesn't change over
a rescan, just construct the list of columns once during ExecInitAgg().
2008-10-16 19:25:55 +00:00
Tom Lane bcf188a218 Fix SPI_getvalue and SPI_getbinval to range-check the given attribute number
according to the TupleDesc's natts, not the number of physical columns in the
tuple.  The previous coding would do the wrong thing in cases where natts is
different from the tuple's column count: either incorrectly report error when
it should just treat the column as null, or actually crash due to indexing off
the end of the TupleDesc's attribute array.  (The second case is probably not
possible in modern PG versions, due to more careful handling of inheritance
cases than we once had.  But it's still a clear lack of robustness here.)

The incorrect error indication is ignored by all callers within the core PG
distribution, so this bug has no symptoms visible within the core code, but
it might well be an issue for add-on packages.  So patch all the way back.
2008-10-16 13:23:21 +00:00
Tom Lane ce0fb501d9 Make the system-attributes loop in AddNewAttributeTuples depend on
lengthof(SysAtt) not FirstLowInvalidHeapAttributeNumber, for consistency with
the other uses of the SysAtt array, and to make it clearer that it doesn't
walk off the end of that array.
2008-10-14 23:27:40 +00:00
Tom Lane 5b5ee14a4b Add a defense to prevent storing pseudo-type data into index columns.
Formerly, the lack of any opclasses that could accept such data was enough
of a defense, but now with a "record" opclass we need to check more carefully.
(You can still use that opclass for an index, but you have to store a named
composite type not an anonymous one.)
2008-10-14 21:47:39 +00:00
Alvaro Herrera c5eabafb6a Ensure that CLUSTER leaves the toast table and index with consistent names,
by renaming the new copies after the catalog games.
2008-10-14 17:19:50 +00:00
Tom Lane a303e4dc43 Extend the date type to support infinity and -infinity, analogously to
the timestamp types.  Turns out this doesn't even reduce the available
range of dates, since the restriction to dates that work for Julian-date
arithmetic is much tighter than the int32 range anyway.  Per a longstanding
TODO item.
2008-10-14 17:12:33 +00:00
Tom Lane 791359fe0e Fix EncodeSpecialTimestamp to throw error on unrecognized input, rather than
returning a failure code that none of its callers bothered to check for.
2008-10-14 15:44:29 +00:00
Heikki Linnakangas 84c3769482 Fix oversight in the relation forks patch: forgot to copy fork number to
fsync requests. This should fix the installcheck failure of the buildfarm
member "kudu".
2008-10-14 08:06:39 +00:00
Tom Lane e3b0117459 Implement comparison of generic records (composite types), and invent a
pseudo-type record[] to represent arrays of possibly-anonymous composite
types.  Since composite datums carry their own type identification, no
extra knowledge is needed at the array level.

The main reason for doing this right now is that it is necessary to support
the general case of detection of cycles in recursive queries: if you need to
compare more than one column to detect a cycle, you need to compare a ROW()
to an array built from ROW()s, at least if you want to do it as the spec
suggests.  Add some documentation and regression tests concerning the cycle
detection issue.
2008-10-13 16:25:20 +00:00
Tom Lane 0a7abcd4c9 Fix corner case wherein a WorkTableScan node could get initialized before the
RecursiveUnion to which it refers.  It turns out that we can just postpone the
relevant initialization steps until the first exec call for the node, by which
time the ancestor node must surely be initialized.  Per report from Greg Stark.
2008-10-13 00:41:41 +00:00
Tom Lane 30584cda35 Fix small query-lifespan memory leak introduced by 8.4 change in index AM API
for bitmap index scans.  Per report and test case from Kevin Grittner.
2008-10-10 14:17:08 +00:00
Tom Lane 8fc4197f7d Fix omission of DiscardStmt in GetCommandLogLevel, per report from Hubert
Depesz Lubaczewski.  In HEAD, also move a couple of other cases to make the
code ordering match up with ProcessUtility.
2008-10-10 13:48:05 +00:00
Tom Lane 76e6602417 Improve the recently-added code for inlining set-returning functions so that
it can handle functions returning setof record.  The case was left undone
originally, but it turns out to be simple to fix.
2008-10-09 19:27:40 +00:00
Alvaro Herrera 2532c54d82 Improve translatability of error messages for external modules by tweaking
the ereport macro.  Included in this commit are enough files for starting
plpgsql, plpython, plperl and pltcl translations.
2008-10-09 17:24:05 +00:00
Tom Lane 1b0f58a9ce Fix crash in bytea-to-XML mapping when the source value is toasted.
Report and fix by Michael McMaster.  Some minor code beautification by me,
also avoid memory leaks in the special-case paths.
2008-10-09 15:49:04 +00:00
Heikki Linnakangas db31addaae Force a checkpoint in CREATE DATABASE before starting to copy the files,
to process any pending unlinks for the source database.

Before, if you dropped a relation in the template database just before
CREATE DATABASE, and a checkpoint happened during copydir(), the checkpoint
might delete a file that we're just about to copy, causing lstat() in
copydir() to fail with ENOENT.

Backpatch to 8.3, where the pending unlinks were introduced.

Per report by Matthew Wakeling and analysis by Tom Lane.
2008-10-09 10:34:06 +00:00
Tom Lane 3437286356 Modify the parser's error reporting to include a specific hint for the case
of referencing a WITH item that's not yet in scope according to the SQL
spec's semantics.  This seems to be an easy error to make, and the bare
"relation doesn't exist" message doesn't lead one's mind in the correct
direction to fix it.
2008-10-08 01:14:44 +00:00
Tom Lane dd4c165bc3 Improve some of the comments in fsmpage.c. 2008-10-07 21:10:11 +00:00
Tom Lane 0d115dde82 Extend CTE patch to support recursive UNION (ie, without ALL). The
implementation uses an in-memory hash table, so it will poop out for very
large recursive results ... but the performance characteristics of a
sort-based implementation would be pretty unpleasant too.
2008-10-07 19:27:04 +00:00
Heikki Linnakangas fa3938fcb1 When a relation is moved to another tablespace, we can't assume that we can
use the old relfilenode in the new tablespace. There might be another relation
in the new tablespace with the same relfilenode, so we must generate a fresh
relfilenode in the new tablespace.

The 8.3 patch to let deleted relation files linger as zero-length files until
the next checkpoint made this more obvious: moving a relation from one table
space another, and then back again, caused a collision with the lingering
file.

Back-patch to 8.1. The issue is present in 8.0 as well, but it doesn't seem
worth fixing there, because we didn't have protection from OID collisions
after OID wraparound before 8.1.

Report by Guillaume Lelarge.
2008-10-07 11:15:41 +00:00
Tom Lane 078aaf796e Improve parser error location for cases where an INSERT or UPDATE command
supplies an expression that can't be coerced to the target column type.
The code previously attempted to point at the target column name, which
doesn't work at all in an INSERT with omitted column name list, and is
also not remarkably helpful when the problem is buried somewhere in a
long INSERT-multi-VALUES command.  Make it point at the failed expression
instead.
2008-10-07 01:47:55 +00:00
Tom Lane 34f89cb4af Fix oversight in recent patch to support multiple read positions in
tuplestore: in READFILE state tuplestore_select_read_pointer must
save the current file seek position in the read pointer being
deactivated.
2008-10-07 00:05:55 +00:00
Tom Lane 742fd06d98 Fix up ruleutils.c for CTE features. The main problem was that
get_name_for_var_field didn't have enough context to interpret a reference to
a CTE query's output.  Fixing this requires separate hacks for the regular
deparse case (pg_get_ruledef) and for the EXPLAIN case, since the available
context information is quite different.  It's pretty nearly parallel to the
existing code for SUBQUERY RTEs, though.  Also, add code to make sure we
qualify a relation name that matches a CTE name; else the CTE will mistakenly
capture the reference when reloading the rule.

In passing, fix a pre-existing problem with get_name_for_var_field not working
on variables in targetlists of SubqueryScan plan nodes.  Although latent all
along, this wasn't a problem until we made EXPLAIN VERBOSE try to print
targetlists.  To do this, refactor the deparse_context_for_plan API so that
the special case for SubqueryScan is all on ruleutils.c's side.
2008-10-06 20:29:38 +00:00
Tom Lane bf461538e1 When expanding a whole-row Var into a RowExpr during ResolveNew(), attach
the column alias names of the RTE referenced by the Var to the RowExpr.
This is needed to allow ruleutils.c to correctly deparse FieldSelect nodes
referencing such a construct.  Per my recent bug report.

Adding a field to RowExpr forces initdb (because of stored rules changes)
so this solution is not back-patchable; which is unfortunate because 8.2
and 8.3 have this issue.  But it only affects EXPLAIN for some pretty odd
corner cases, so we can probably live without a solution for the back
branches.
2008-10-06 17:39:26 +00:00
Tom Lane e64bb65aff Fix GetCTEForRTE() to deal with the possibility that the RTE it's given came
from a query level above the current ParseState.
2008-10-06 15:15:22 +00:00
Heikki Linnakangas 5f853c6556 Use fork names instead of numbers in the file names for additional
relation forks. While the file names are not visible to users, for those
that do peek into the data directory, it's nice to have more descriptive
names. Per Greg Stark's suggestion.
2008-10-06 14:13:17 +00:00
Magnus Hagander 3bea93b3b0 Add columns boot_val and reset_val to the pg_settings view, to expose
the value a parameter has at server start and will have after RESET,
respectively.

Greg Smith, with some modifications by me.
2008-10-06 13:05:40 +00:00
Heikki Linnakangas 89f373bf5b Index FSMs needs to be vacuumed as well. Report by Jeff Davis. 2008-10-06 08:04:11 +00:00
Tom Lane 557faa4fb3 Random speculation about the reason for PPC64 buildfarm failures:
maybe isalnum is returning a value with the low-order byte all zero?
2008-10-06 05:03:27 +00:00
Tom Lane 0ff384f0bc Fix the implicit-RTE code to be able to handle implicit RTEs for CTEs, as
well as regular tables.  Per discussion, this seems necessary to meet the
principle of least astonishment.

In passing, simplify the error messages in warnAutoRange().  Now that we
have parser error position info for these errors, it doesn't seem very
useful to word the error message differently depending on whether we are
inside a sub-select or not.
2008-10-06 02:12:56 +00:00
Tom Lane 8acfc7594d Tweak the overflow checks in integer division functions to complain if the
machine produces zero (rather than the more usual minimum-possible-integer)
for the only possible overflow case.  This has been seen to occur for at least
some word widths on some hardware, and it's cheap enough to check for
everywhere.  Per Peter's analysis of buildfarm reports.

This could be back-patched, but in the absence of any gripes from the field
I doubt it's worth the trouble.
2008-10-05 23:18:37 +00:00
Tom Lane 1e4b03847c Improve behavior of WITH RECURSIVE with an untyped literal in the
non-recursive term.  Per an example from Dickson S. Guedes.
2008-10-05 22:50:55 +00:00
Tom Lane 0814250474 Fix markTargetListOrigin() to not fail on a simple-Var reference to a
recursive CTE that we're still in progress of analyzing.  Add a similar guard
to the similar code in expandRecordVariable(), and tweak regression tests to
cover this case.  Per report from Dickson S. Guedes.
2008-10-05 22:20:17 +00:00
Peter Eisentraut 2cf8afe5d1 Remove obsolete internal functions istrue, isfalse, isnottrue, isnotfalse,
nullvalue, nonvalue.  A long time ago, these were used to implement the SQL
constructs IS TRUE, etc.
2008-10-05 17:33:17 +00:00
Tom Lane 44d5be0e53 Implement SQL-standard WITH clauses, including WITH RECURSIVE.
There are some unimplemented aspects: recursive queries must use UNION ALL
(should allow UNION too), and we don't have SEARCH or CYCLE clauses.
These might or might not get done for 8.4, but even without them it's a
pretty useful feature.

There are also a couple of small loose ends and definitional quibbles,
which I'll send a memo about to pgsql-hackers shortly.  But let's land
the patch now so we can get on with other development.

Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 21:56:55 +00:00
Heikki Linnakangas 706a308806 Add relation fork support to pg_relation_size() function. You can now pass
name of a fork ('main' or 'fsm', at the moment) to pg_relation_size() to
get the size of a specific fork. Defaults to 'main', if none given.

While we're at it, modify pg_relation_size to take a regclass as argument,
instead of separate variants taking oid and name. This change is
transparent to typical use where the table name is passed as a string
literal, like pg_relation_size('table'), but will break queries like
pg_relation_size(namecol), where namecol is of type name. text-type input
still works, and using a non-schema-qualified table name is not very
reliable anyway, so this is unlikely to break anyone's queries in practice.
2008-10-03 07:33:10 +00:00
Bruce Momjian 2cc1633a35 Update README.HOT to reflect new snapshot tracking and xmin advancement
code in 8.4.
2008-10-02 20:59:31 +00:00
Tom Lane 607b39855a Fix improper display of fractional seconds in interval values
when using --enable-integer-datetimes and a non-ISO datestyle.

Ron Mayer
2008-10-02 13:47:38 +00:00
Tom Lane dad4cb6258 Improve tuplestore.c to support multiple concurrent read positions.
This facility replaces the former mark/restore support but is otherwise
upward-compatible with previous uses.  It's expected to be needed for
single evaluation of CTEs and also for window functions, so I'm committing
it separately instead of waiting for either one of those patches to be
finished.  Per discussion with Greg Stark and Hitoshi Harada.

Note: I removed nodeFunctionscan's mark/restore support, instead of bothering
to update it for this change, because it was dead code anyway.
2008-10-01 19:51:50 +00:00
Tom Lane 68827a7ada Suppress an uninitialized-variable warning (not all versions of gcc
complain here, but some do)
2008-10-01 14:59:23 +00:00
Heikki Linnakangas f06ef2bede Fix WAL redo of FSM truncation. We can't call smgrtruncate() during WAL
replay, because it tries to XLogInsert().
2008-10-01 08:12:14 +00:00
Tom Lane 6ca1b1cd95 Fix compiler warning (unportable sprintf usage) 2008-09-30 14:15:58 +00:00
Heikki Linnakangas 15c121b3ed Rewrite the FSM. Instead of relying on a fixed-size shared memory segment, the
free space information is stored in a dedicated FSM relation fork, with each
relation (except for hash indexes; they don't use FSM).

This eliminates the max_fsm_relations and max_fsm_pages GUC options; remove any
trace of them from the backend, initdb, and documentation.

Rewrite contrib/pg_freespacemap to match the new FSM implementation. Also
introduce a new variant of the get_raw_page(regclass, int4, int4) function in
contrib/pageinspect that let's you to return pages from any relation fork, and
a new fsm_page_contents() function to inspect the new FSM pages.
2008-09-30 10:52:14 +00:00
Tom Lane 2dbc0ca937 Dept of second thoughts: let's make sure that get_index_stats_hook is only
applied to expression indexes, not to plain relations.  The original coding
in btcostestimate conflated the two cases, but it's not hard to use
get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 20:42:12 +00:00
Tom Lane 7b7df9f0b1 Add hooks to let plugins override the planner's lookups in pg_statistic.
Simon Riggs, with some editorialization by me.
2008-09-28 19:51:40 +00:00
Andrew Dunstan bc965e840a Compare escaped chars case insensitively for ILIKE - per gripe from TGL. 2008-09-27 16:53:54 +00:00
Tom Lane b1e929f295 Fix pointer-advancement bugs in MS and US cases of new to_timestamp() code.
Alex Hunsaker
2008-09-26 15:35:28 +00:00
Tom Lane 3d8fd75732 Make LIKE throw an error if the escape character is at the end of the pattern
(ie, has nothing to quote), rather than silently ignoring the character as has
been our historical behavior.  This is required by SQL spec and should help
reduce the sort of user confusion seen in bug #4436.  Per discussion.

This is not so much a bug fix as a definitional change, and it could break
existing applications; so not back-patched.  It might deserve being mentioned
as an incompatibility in the 8.4 release notes.
2008-09-26 02:16:40 +00:00
Tom Lane e8e746de34 Establish the rule that array types should have the same typdelim as their
element types.  Since the backend doesn't actually pay attention to the array
type's delimiter, this has no functional effect, but it seems better for the
catalog entries to be consistent.  Per gripe from Greg Mullane and subsequent
discussion.
2008-09-25 03:28:56 +00:00
Bruce Momjian fb4bb8b9c5 Fix integral timestamps so the output is consistent in all cases to
round:

	select interval '0:0:0.7', interval '@ 0.70 secs', interval '0.7
		seconds';

Ron Mayer
2008-09-24 19:46:44 +00:00
Tom Lane 96a25d393c Fix more problems with rewriter failing to set Query.hasSubLinks when inserting
a SubLink expression into a rule query.  We missed cases where the original
query contained a sub-SELECT in a function in FROM, a multi-row VALUES list,
or a RETURNING list.  Per bug #4434 from Dean Rasheed and subsequent
investigation.

Back-patch to 8.1; older releases don't have the issue because they didn't
try to be smart about setting hasSubLinks only when needed.
2008-09-24 16:52:46 +00:00
Magnus Hagander cdf5357ec9 Only show source file and line numbers to superusers, for consistent
security level with other parts of the system.

Per gripe from Tom
2008-09-23 21:12:03 +00:00
Bruce Momjian 5f7b25d5d5 Add comment about the use of EXEC_BACKEND. 2008-09-23 20:35:38 +00:00
Heikki Linnakangas c2d4526495 Tighten the check in initdb and CREATE DATABASE that the chosen encoding
matches the encoding of the locale. LC_COLLATE is now checked in addition
to LC_CTYPE.
2008-09-23 10:58:03 +00:00
Heikki Linnakangas 61d9674988 Make LC_COLLATE and LC_CTYPE database-level settings. Collation and
ctype are now more like encoding, stored in new datcollate and datctype
columns in pg_database.

This is a stripped-down version of Radek Strnad's patch, with further
changes by me.
2008-09-23 09:20:39 +00:00
Tom Lane 579c025e5f Simplify the definitions of a couple of system views by using SELECT *
instead of listing all the columns returned by the underlying function.

initdb not forced since this patch doesn't actually change anything about
the stored form of the views.  It just means there's one less place to change
if someone wants to add columns to them.
2008-09-21 19:38:56 +00:00
Tom Lane 4e57668da4 Create a selectivity estimation function for the text search @@ operator.
Jan Urbanski
2008-09-19 19:03:41 +00:00
Alvaro Herrera 5817d861e9 Optimize CleanupTempFiles by having a boolean flag that keeps track of whether
there are FD_XACT_TEMPORARY files to clean up at transaction end.

Per performance profiling results on AWeber's huge systems.

Patch by me after an idea suggested by Simon Riggs.
2008-09-19 04:57:10 +00:00
Tom Lane 35c2a3c3cf Allow ShowBufferUsage() to report the number of reads/writes that have
occurred to temporary files.  This replaces the unused
NDirectFileRead/NDirectFileWrite counters.

Itagaki Takahiro
2008-09-17 13:15:55 +00:00
Tom Lane b73c0c2a51 Clean up a couple of weird corner cases in interval parsing: make -yyyy-mm be
interpreted as expected (the sign should affect months too), and get rid of
hard-wired assumption that unmarked signed values must be hours (if integers)
or seconds (if floats).  The former was just a bug in my previous patch,
while the latter may have made sense at one time but seems illogical now
that we support determination of the units from typmod information.
Ron Mayer and myself.
2008-09-16 22:31:21 +00:00
Tom Lane 8948ee37e5 Fix multiple memory leaks in xml_out(). Per report from Matt Magoffin. 2008-09-16 00:49:41 +00:00
Tom Lane 1cd935609f Fix caching of foreign-key-checking queries so that when a replan is needed,
we regenerate the SQL query text not merely the plan derived from it.  This
is needed to handle contingencies such as renaming of a table or column
used in an FK.  Pre-8.3, such cases worked despite the lack of replanning
(because the cached plan needn't actually change), so this is a regression.
Per bug #4417 from Benjamin Bihler.
2008-09-15 23:37:40 +00:00
Magnus Hagander 448950b37b Fix error messages from recent pg_hba parsing patch to use errcontext()
to indicate where the error occurred.
2008-09-15 20:55:04 +00:00
Tom Lane 4adc2f72a4 Change hash indexes to store only the hash code rather than the whole indexed
value.  This means that hash index lookups are always lossy and have to be
rechecked when the heap is visited; however, the gain in index compactness
outweighs this when the indexed values are wide.  Also, we only need to
perform datatype comparisons when the hash codes match exactly, rather than
for every entry in the hash bucket; so it could also win for datatypes that
have expensive comparison functions.  A small additional win is gained by
keeping hash index pages sorted by hash code and using binary search to reduce
the number of index tuples we have to look at.

Xiao Meng

This commit also incorporates Zdenek Kotala's patch to isolate hash metapages
and hash bitmaps a bit better from the page header datastructures.
2008-09-15 18:43:41 +00:00
Magnus Hagander 9872381090 Parse pg_hba.conf in postmaster, instead of once in each backend for
each connection. This makes it possible to catch errors in the pg_hba
file when it's being reloaded, instead of silently reloading a broken
file and failing only when a user tries to connect.

This patch also makes the "sameuser" argument to ident authentication
optional.
2008-09-15 12:32:57 +00:00
Tom Lane bf0b6ac43c Skip opfamily check in eclass_matches_any_index() when the index isn't a
btree.  We can't easily tell whether clauses generated from the equivalence
class could be used with such an index, so just assume that they might be.
This bit of over-optimization prevented use of non-btree indexes for nestloop
inner indexscans, in any case where the join uses an equality operator that
is also a btree operator --- which in particular is typically true for hash
indexes.  Noted while trying to test the current hash index patch.
2008-09-12 14:56:13 +00:00
Tom Lane 06edce4c3f Tighten up to_date/to_timestamp so that they are more likely to reject
erroneous input, rather than silently producing bizarre results as formerly
happened.

Brendan Jurd
2008-09-11 17:32:34 +00:00
Tom Lane 70530c808b Adjust the parser to accept the typename syntax INTERVAL ... SECOND(n)
and the literal syntax INTERVAL 'string' ... SECOND(n), as required by the
SQL standard.  Our old syntax put (n) directly after INTERVAL, which was
a mistake, but will still be accepted for backward compatibility as well
as symmetry with the TIMESTAMP cases.

Change intervaltypmodout to show it in the spec's way, too.  (This could
potentially affect clients, if there are any that analyze the typmod of an
INTERVAL in any detail.)

Also fix interval input to handle 'min:sec.frac' properly; I had overlooked
this case in my previous patch.

Document the use of the interval fields qualifier, which up to now we had
never mentioned in the docs.  (I think the omission was intentional because
it didn't work per spec; but it does now, or at least close enough to be
credible.)
2008-09-11 15:27:30 +00:00
Alvaro Herrera d53a56687f Initialize the minimum frozen Xid in vac_update_datfrozenxid using
GetOldestXmin() instead of RecentGlobalXmin; this is safer because we do not
depend on the latter being correctly set elsewhere, and while it is more
expensive, this code path is not performance-critical.  This is a real
risk for autovacuum, because it can execute whole cycles without doing
a single vacuum, which would mean that RecentGlobalXmin would stay at its
initialization value, FirstNormalTransactionId, causing a bogus value to be
inserted in pg_database.  This bug could explain some recent reports of
failure to truncate pg_clog.

At the same time, change the initialization of RecentGlobalXmin to
InvalidTransactionId, and ensure that it's set to something else whenever
it's going to be used.  Using it as FirstNormalTransactionId in HOT page
pruning could incur in data loss.  InitPostgres takes care of setting it
to a valid value, but the extra checks are there to prevent "special"
backends from behaving in unusual ways.

Per Tom Lane's detailed problem dissection in 29544.1221061979@sss.pgh.pa.us
2008-09-11 14:01:10 +00:00
Tom Lane b8646012d5 Tweak newly added set_config_sourcefile() so that the target record
isn't left corrupt if guc_strdup should fail.
2008-09-10 19:16:22 +00:00
Tom Lane f867339c01 Make our parsing of INTERVAL literals spec-compliant (or at least a heck of
a lot closer than it was before).  To do this, tweak coerce_type() to pass
through the typmod information when invoking interval_in() on an UNKNOWN
constant; then fix DecodeInterval to pay attention to the typmod when deciding
how to interpret a units-less integer value.  I changed one or two other
details as well.  I believe the code now reacts as expected by spec for all
the literal syntaxes that are specifically enumerated in the spec.  There
are corner cases involving strings that don't exactly match the set of fields
called out by the typmod, for which we might want to tweak the behavior some
more; but I think this is an area of user friendliness rather than spec
compliance.  There remain some non-compliant details about the SQL syntax
(as opposed to what's inside the literal string); but at least we'll throw
error rather than silently doing the wrong thing in those cases.
2008-09-10 18:29:41 +00:00
Alvaro Herrera 3b9ec4682c Add "source file" and "source line" information to each GUC variable.
initdb forced due to changes in the pg_settings view.

Magnus Hagander and Alvaro Herrera.
2008-09-10 18:09:20 +00:00
Tom Lane ee33b95d9c Improve the plan cache invalidation mechanism to make it invalidate plans
when user-defined functions used in a plan are modified.  Also invalidate
plans when schemas, operators, or operator classes are modified; but for these
cases we just invalidate everything rather than tracking exact dependencies,
since these types of objects seldom change in a production database.

Tom Lane; loosely based on a patch by Martin Pihlak.
2008-09-09 18:58:09 +00:00
Tom Lane ead21631e8 Fix a couple of problems pointed out by Fujii Masao in the 2008-Apr-05 patch
for pg_stop_backup.  First, it is possible that the history file name is not
alphabetically later than the last WAL file name, so we should explicitly
check that both have been archived.  Second, the previous coding would wait
forever if a checkpoint had managed to remove the WAL file before we look for
it.

Simon Riggs, plus some code cleanup by me.
2008-09-08 16:42:15 +00:00
Tom Lane a0b76dc662 Create a separate grantable privilege for TRUNCATE, rather than having it be
always owner-only.  The TRUNCATE privilege works identically to the DELETE
privilege so far as interactions with the rest of the system go.

Robert Haas
2008-09-08 00:47:41 +00:00
Tom Lane a26c7e3d71 Support set-returning functions in the target lists of Agg and Group plan
nodes.  This is a pretty ugly feature but since we don't yet have a
plausible substitute, we'd better support it everywhere.
Per gripe from Jeff Davis.
2008-09-08 00:22:56 +00:00
Tom Lane e6a310b281 Reimplement text_position and related functions to use Boyer-Moore-Horspool
searching instead of naive matching.  In the worst case this has the same
O(M*N) complexity as the naive method, but the worst case is hard to hit,
and the average case is very fast, especially with longer patterns.

David Rowley
2008-09-07 04:20:00 +00:00
Tom Lane 409c144d83 Adjust psql's new \ef command to present an empty CREATE FUNCTION template
for editing if no function name is specified.  This seems a much cleaner way
to offer that functionality than the original patch had.  In passing,
de-clutter the error displays that are given for a bogus function-name
argument, and standardize on "$function$" as the default delimiter for the
function body.  (The original coding would use the shortest possible
dollar-quote delimiter, which seems to create unnecessarily high risk of
later conflicts with the user-modified function body.)
2008-09-06 20:18:08 +00:00
Tom Lane 2c863ca818 Implement a psql command "\ef" to edit the definition of a function.
In support of that, create a backend function pg_get_functiondef().
The psql command is functional but maybe a bit rough around the edges...

Abhijit Menon-Sen
2008-09-06 00:01:25 +00:00
Tom Lane e540b97248 Fix an oversight in the 8.2 patch that improved mergejoin performance by
inserting a materialize node above an inner-side sort node, when the sort is
expected to spill to disk.  (The materialize protects the sort from having
to support mark/restore, allowing it to do its final merge pass on-the-fly.)
We neglected to teach cost_mergejoin about that hack, so it was failing to
include the materialize's costs in the estimated cost of the mergejoin.
The materialize's costs are generally going to be pretty negligible in
comparison to the sort's, so this is only a small error and probably not
worth back-patching; but it's still wrong.

In the similar case where a materialize is inserted to protect an inner-side
node that can't do mark/restore at all, it's still true that the materialize
should not spill to disk, and so we should cost it cheaply rather than
expensively.

Noted while thinking about a question from Tom Raney.
2008-09-05 21:07:29 +00:00
Peter Eisentraut 11f53b1063 Code coverage testing with gcov. Documentation is in the regression test
chapter.

Author: Michelle Caisse <Michelle.Caisse@Sun.COM>
2008-09-05 12:11:18 +00:00
Teodor Sigaev 5373817cf2 Fix strategy propagation to scanEntry for partial match by moving propagation
to initializaion of scanEntry.
2008-09-04 11:47:05 +00:00
Tom Lane ba9f37f066 If a loadable module has wrong values in its magic block, spell out
exactly what they are in the complaint message.  Marko Kreen,
some editorialization by me.
2008-09-03 22:34:50 +00:00
Tom Lane fbb2b69c8f Prevent memory leaks in our various bison parsers when an error occurs
during parsing.  Formerly the parser's stack was allocated with malloc
and so wouldn't be reclaimed; this patch makes it use palloc instead,
so that flushing the current context will reclaim the memory.  Per
Marko Kreen.
2008-09-02 20:37:55 +00:00
Tom Lane b153c09209 Add a bunch of new error location reports to parse-analysis error messages.
There are still some weak spots around JOIN USING and relation alias lists,
but most errors reported within backend/parser/ now have locations.
2008-09-01 20:42:46 +00:00
Heikki Linnakangas 9ac4299163 HeapTupleHeaderAdjustCmax made the incorrect assumption that the raw
command id is the cmin, when it can in fact be a combo cid. That made rows
incorrectly invisible to a transaction where a tuple was deleted by multiple
aborted subtransactions.

Report and patch Karl Schnaitter. Back-patch to 8.3, where combo cids was
introduced.
2008-09-01 18:52:45 +00:00
Tom Lane 449a00fbbd Fix the raw-parsetree representation of star (as in SELECT * FROM or
SELECT foo.*) so that it cannot be confused with a quoted identifier "*".
Instead create a separate node type A_Star to represent this notation.
Per pgsql-hackers discussion of 2007-Sep-27.
2008-08-30 01:39:14 +00:00
Tom Lane 6253f9de67 In GCC-based builds, use a better newNode() macro that relies on GCC-specific
syntax to avoid a useless store into a global variable.  Per experimentation,
this works better than my original thought of trying to push the code into
an out-of-line subroutine.
2008-08-29 22:49:07 +00:00
Tom Lane 4571185111 Suppress gcc warning about possibly-uninitialized variable. It's not
clear to me why I'd not seen this message before --- on F-9 it seems to
only happen if Asserts are disabled, which ought to be irrelevant.
Maybe that affects a decision whether to inline get_ten(), which would
be needed to expose the warning condition to the compiler?  Anyway,
the fix is clear.
2008-08-29 16:34:14 +00:00
Peter Eisentraut 7c31742a07 Remove all traces that suggest that a non-Bison yacc might be supported, and
change build system to use only Bison.  Simplify build rules, make file names
uniform.  Don't build the token table header file where it is not needed.
2008-08-29 13:02:33 +00:00
Tom Lane a2794623d2 Extend the parser location infrastructure to include a location field in
most node types used in expression trees (both before and after parse
analysis).  This allows us to place an error cursor in many situations
where we formerly could not, because the information wasn't available
beyond the very first level of parse analysis.  There's a fair amount
of work still to be done to persuade individual ereport() calls to actually
include an error location, but this gets the initdb-forcing part of the
work out of the way; and the situation is already markedly better than
before for complaints about unimplementable implicit casts, such as
CASE and UNION constructs with incompatible alternative data types.
Per my proposal of a few days ago.
2008-08-28 23:09:48 +00:00
Tom Lane 6734182c16 Teach eval_const_expressions() to simplify an ArrayCoerceExpr to a constant
when its input is constant and the element coercion function is immutable
(or nonexistent, ie, binary-coercible case).  This is an oversight in the
8.3 implementation of ArrayCoerceExpr, and its result is that certain cases
involving IN or NOT IN with constants don't get optimized as they should be.
Per experimentation with an example from Ow Mun Heng.
2008-08-26 02:16:31 +00:00
Tom Lane e5536e77a5 Move exprType(), exprTypmod(), expression_tree_walker(), and related routines
into nodes/nodeFuncs, so as to reduce wanton cross-subsystem #includes inside
the backend.  There's probably more that should be done along this line,
but this is a start anyway.
2008-08-25 22:42:34 +00:00
Tom Lane d320101b5b Get rid of the last remaining uses of var_is_rel(), to wit some debugging
checks in ExecIndexBuildScanKeys() that were inadequate anyway: it's better
to verify the correct varno on an expected index key, not just reject OUTER
and INNER.

This makes the entire current contents of nodeFuncs.c dead code.  I'll be
replacing it with some other stuff later, as per recent proposal.
2008-08-25 20:20:30 +00:00
Magnus Hagander f1e237b6b2 Unconditionally write the statsfile when SIGHUP is received, to minimize
the window during which backends have no statistics file to read.
2008-08-25 18:55:43 +00:00
Alvaro Herrera d96d7be2b5 Update URL to Ross William's paper.
Devrim Gunduz.
2008-08-25 17:37:40 +00:00
Magnus Hagander be8d6c5c34 Make stats_temp_directory PGC_SIGHUP, and document how it may cause a temporary
"outage" of the statistics views.

This requires making the stats collector respond to SIGHUP, like the other
utility processes already did.
2008-08-25 15:11:01 +00:00
Magnus Hagander 8c032adec4 Convert remaining builtin set-returning functions to use OUT parameters, making
it possible to call them without specifying a column list.

Jaime Casanova
2008-08-25 11:18:43 +00:00
Bruce Momjian 31ad4e5396 Add missing descriptions for aggregates, functions and conversions.
Bernd Helmle
2008-08-23 20:31:37 +00:00
Teodor Sigaev 1dcf6fdf1b Fix possible duplicate tuples while GiST scan. Now page is processed
at once and ItemPointers are collected in memory.

Remove tuple's killing by killtuple() if tuple was moved to another
page - it could produce unaceptable overhead.

Backpatch up to 8.1 because the bug was introduced by GiST's concurrency support.
2008-08-23 10:37:24 +00:00
Bruce Momjian 8ddb739e9d Make "log_temp_files" super-user set only, like other logging options.
Simon Riggs
2008-08-22 18:47:07 +00:00
Bruce Momjian 6152de97d3 Minor patch on pgbench
1. -i option should run vacuum analyze only on pgbench tables, not *all*
tables in database.

2. pre-run cleanup step was DELETE FROM HISTORY then VACUUM HISTORY.
This is just a slow version of TRUNCATE HISTORY.

Simon Riggs
2008-08-22 17:57:34 +00:00
Bruce Momjian 03302fd9b4 Improve wording of error message when a postgresql.conf setting is
ignored because it can only be set at server start.
2008-08-22 00:20:40 +00:00
Tom Lane bd3daddaf2 Arrange to convert EXISTS subqueries that are equivalent to hashable IN
subqueries into the same thing you'd have gotten from IN (except always with
unknownEqFalse = true, so as to get the proper semantics for an EXISTS).
I believe this fixes the last case within CVS HEAD in which an EXISTS could
give worse performance than an equivalent IN subquery.

The tricky part of this is that if the upper query probes the EXISTS for only
a few rows, the hashing implementation can actually be worse than the default,
and therefore we need to make a cost-based decision about which way to use.
But at the time when the planner generates plans for subqueries, it doesn't
really know how many times the subquery will be executed.  The least invasive
solution seems to be to generate both plans and postpone the choice until
execution.  Therefore, in a query that has been optimized this way, EXPLAIN
will show two subplans for the EXISTS, of which only one will actually get
executed.

There is a lot more that could be done based on this infrastructure: in
particular it's interesting to consider switching to the hash plan if we start
out using the non-hashed plan but find a lot more upper rows going by than we
expected.  I have therefore left some minor inefficiencies in place, such as
initializing both subplans even though we will currently only use one.
2008-08-22 00:16:04 +00:00
Tom Lane cc0dd43850 Marginal improvement in sublink planning: allow unknownEqFalse optimization
to be used for SubLinks that are underneath a top-level OR clause.  Just as at
the very top level of WHERE, it's not necessary to be accurate about whether
the sublink returns FALSE or NULL, because either result has the same impact
on whether the WHERE will succeed.
2008-08-20 19:58:24 +00:00
Tom Lane 390e59cd5f Fix obsolete comment. It's no longer the case that Param nodes don't
carry typmod.
2008-08-20 15:49:30 +00:00
Tom Lane 9650830bc8 Cause the output from debug_print_parse, debug_print_rewritten, and
debug_print_plan to appear at LOG message level, not DEBUG1 as historically.
Make debug_pretty_print default to on.  Also, cause plans generated via
EXPLAIN to be subject to debug_print_plan.  This is all to make
debug_print_plan a reasonably comfortable substitute for the former behavior
of EXPLAIN VERBOSE.
2008-08-19 18:30:04 +00:00
Tom Lane 719012e013 Add some defenses against constant-FALSE outer join conditions. Since
eval_const_expressions will generally throw away anything that's ANDed with
constant FALSE, what we're left with given an example like

select * from tenk1 a where (unique1,0) in (select unique2,1 from tenk1 b);

is a cartesian product computation, which is really not acceptable.
This is a regression in CVS HEAD compared to previous releases, which were
able to notice the impossible join condition in this case --- though not in
some related cases that are also improved by this patch, such as

select * from tenk1 a left join tenk1 b on (a.unique1=b.unique2 and 0=1);

Fix by skipping evaluation of the appropriate side of the outer join in
cases where it's demonstrably unnecessary.
2008-08-17 19:40:11 +00:00
Tom Lane f2689e421d Remove prohibition against SubLinks in the WHERE clause of an EXISTS subquery
that we're considering pulling up.  I hadn't wanted to think through whether
that could work during the first pass at this stuff.  However, on closer
inspection it seems to be safe enough.
2008-08-17 02:19:19 +00:00
Tom Lane 19e34b6239 Improve sublink pullup code to handle ANY/EXISTS sublinks that are at top
level of a JOIN/ON clause, not only at top level of WHERE.  (However, we
can't do this in an outer join's ON clause, unless the ANY/EXISTS refers
only to the nullable side of the outer join, so that it can effectively
be pushed down into the nullable side.)  Per request from Kevin Grittner.

In passing, fix a bug in the initial implementation of EXISTS pullup:
it would Assert if the EXIST's WHERE clause used a join alias variable.
Since we haven't yet flattened join aliases when this transformation
happens, it's necessary to include join relids in the computed set of
RHS relids.
2008-08-17 01:20:00 +00:00
Tom Lane d4af2a6481 Clean up the loose ends in selectivity estimation left by my patch for semi
and anti joins.  To do this, pass the SpecialJoinInfo struct for the current
join as an additional optional argument to operator join selectivity
estimation functions.  This allows the estimator to tell not only what kind
of join is being formed, but which variable is on which side of the join;
a requirement long recognized but not dealt with till now.  This also leaves
the door open for future improvements in the estimators, such as accounting
for the null-insertion effects of lower outer joins.  I didn't do anything
about that in the current patch but the information is in principle deducible
from what's passed.

The patch also clarifies the definition of join selectivity for semi/anti
joins: it's the fraction of the left input that has (at least one) match
in the right input.  This allows getting rid of some very fuzzy thinking
that I had committed in the original 7.4-era IN-optimization patch.
There's probably room to estimate this better than the present patch does,
but at least we know what to estimate.

Since I had to touch CREATE OPERATOR anyway to allow a variant signature
for join estimator functions, I took the opportunity to add a couple of
additional checks that were missing, per my recent message to -hackers:
* Check that estimator functions return float8;
* Require execute permission at the time of CREATE OPERATOR on the
operator's function as well as the estimator functions;
* Require ownership of any pre-existing operator that's modified by
the command.
I also moved the lookup of the functions out of OperatorCreate() and
into operatorcmds.c, since that seemed more consistent with most of
the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 00:01:38 +00:00
Tom Lane 118461114e Performance fix for new anti-join code in nodeMergejoin.c: after finding a
match in antijoin mode, we should advance to next outer tuple not next inner.
We know we don't want to return this outer tuple, and there is no point in
advancing over matching inner tuples now, because we'd just have to do it
again if the next outer tuple has the same merge key.  This makes a noticeable
difference if there are lots of duplicate keys in both inputs.

Similarly, after finding a match in semijoin mode, arrange to advance to
the next outer tuple after returning the current match; or immediately,
if it fails the extra quals.  The rationale is the same.  (This is a
performance bug in existing releases; perhaps worth back-patching?  The
planner tries to avoid using mergejoin with lots of duplicates, so it may
not be a big issue in practice.)

Nestloop and hash got this right to start with, but I made some cosmetic
adjustments there to make the corresponding bits of logic look more similar.
2008-08-15 19:20:42 +00:00
Magnus Hagander 5b8eb2b4b9 Make the temporary directory for pgstat files configurable by the GUC
variable stats_temp_directory, instead of requiring the admin to
mount/symlink the pg_stat_tmp directory manually.

For now the config variable is PGC_POSTMASTER. Room for further improvment
that would allow it to be changed on-the-fly.
2008-08-15 08:37:41 +00:00
Heikki Linnakangas f24f233f6a Fix pull_up_simple_union_all to copy all rtable entries from child subquery to
parent, not only those with RangeTblRefs. We need them in ExecCheckRTPerms.

Report by Brendan O'Shea. Back-patch to 8.2, where pull_up_simple_union_all
was introduced.
2008-08-14 20:31:29 +00:00
Tom Lane e006a24ad1 Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace
the old JOIN_IN code, but antijoins are new functionality.)  Teach the planner
to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti
joins respectively.  Also, LEFT JOINs with suitable upper-level IS NULL
filters are recognized as being anti joins.  Unify the InClauseInfo and
OuterJoinInfo infrastructure into "SpecialJoinInfo".  With that change,
it becomes possible to associate a SpecialJoinInfo with every join attempt,
which permits some cleanup of join selectivity estimation.  That needs to be
taken much further than this patch does, but the next step is to change the
API for oprjoin selectivity functions, which seems like material for a
separate patch.  So for the moment the output size estimates for semi and
especially anti joins are quite bogus.
2008-08-14 18:48:00 +00:00
Alvaro Herrera 3ccde312ec Have autovacuum consider processing TOAST tables separately from their
main tables.

This requires vacuum() to accept processing a toast table standalone, so
there's a user-visible change in that it's now possible (for a superuser) to
execute "VACUUM pg_toast.pg_toast_XXX".
2008-08-13 00:07:50 +00:00
Heikki Linnakangas 3f0e808c4a Introduce the concept of relation forks. An smgr relation can now consist
of multiple forks, and each fork can be created and grown separately.

The bulk of this patch is about changing the smgr API to include an extra
ForkNumber argument in every smgr function. Also, smgrscheduleunlink and
smgrdounlink no longer implicitly call smgrclose, because other forks might
still exist after unlinking one. The callers of those functions have been
modified to call smgrclose instead.

This patch in itself doesn't have any user-visible effect, but provides the
infrastructure needed for upcoming patches. The additional forks envisioned
are a rewritten FSM implementation that doesn't rely on a fixed-size shared
memory block, and a visibility map to allow skipping portions of a table in
VACUUM that have no dead tuples.
2008-08-11 11:05:11 +00:00
Tom Lane eca1388629 Fix corner-case bug introduced with HOT: if REINDEX TABLE pg_class (or a
REINDEX DATABASE including same) is done before a session has done any other
update on pg_class, the pg_class relcache entry was left with an incorrect
setting of rd_indexattr, because the indexed-attributes set would be first
demanded at a time when we'd forced a partial list of indexes into the
pg_class entry, and it would remain cached after that.  This could result
in incorrect decisions about HOT-update safety later in the same session.
In practice, since only pg_class_relname_nsp_index would be missed out,
only ALTER TABLE RENAME and ALTER TABLE SET SCHEMA could trigger a problem.
Per report and test case from Ondrej Jirman.
2008-08-10 19:02:33 +00:00
Tom Lane 30fd8ec799 Install checks in executor startup to ensure that the tuples produced by an
INSERT or UPDATE will match the target table's current rowtype.  In pre-8.3
releases inconsistency can arise with stale cached plans, as reported by
Merlin Moncure.  (We patched the equivalent hazard on the SELECT side in Feb
2007; I'm not sure why we thought there was no risk on the insertion side.)
In 8.3 and HEAD this problem should be impossible due to plan cache
invalidation management, but it seems prudent to make the check anyway.

Back-patch as far as 8.0.  7.x versions lack ALTER COLUMN TYPE, so there
seems no way to abuse a stale plan comparably.
2008-08-08 17:01:11 +00:00
Tom Lane af95d7aa63 Improve INTERSECT/EXCEPT hashing by realizing that we don't need to make any
hashtable entries for tuples that are found only in the second input: they
can never contribute to the output.  Furthermore, this implies that the
planner should endeavor to put first the smaller (in number of groups) input
relation for an INTERSECT.  Implement that, and upgrade prepunion's estimation
of the number of rows returned by setops so that there's some amount of sanity
in the estimate of which one is smaller.
2008-08-07 19:35:02 +00:00
Tom Lane 368df30427 Support hashing for duplicate-elimination in INTERSECT and EXCEPT queries.
This completes my project of improving usage of hashing for duplicate
elimination (aggregate functions with DISTINCT remain undone, but that's
for some other day).

As with the previous patches, this means we can INTERSECT/EXCEPT on datatypes
that can hash but not sort, and it means that INTERSECT/EXCEPT without ORDER
BY are no longer certain to produce sorted output.
2008-08-07 03:04:04 +00:00
Tom Lane 2d1d96b1ce Teach the system how to use hashing for UNION. (INTERSECT/EXCEPT will follow,
but seem like a separate patch since most of the remaining work is on the
executor side.)  I took the opportunity to push selection of the grouping
operators for set operations into the parser where it belongs.  Otherwise this
is just a small exercise in making prepunion.c consider both alternatives.

As with the recent DISTINCT patch, this means we can UNION on datatypes that
can hash but not sort, and it means that UNION without ORDER BY is no longer
certain to produce sorted output.
2008-08-07 01:11:52 +00:00
Tom Lane 3d40d5e70e Do not allow Unique nodes to be scanned backwards. The code claimed that it
would work, but in fact it didn't return the same rows when moving backwards
as when moving forwards.  This would have no visible effect in a DISTINCT
query (at least assuming the column datatypes use a strong definition of
equality), but it gave entirely wrong answers for DISTINCT ON queries.
2008-08-05 21:28:29 +00:00
Tom Lane c78248c91d Department of second thoughts: fix newly-added code in planner.c to make real
sure that DISTINCT ON does what it's supposed to, ie, sort by the full ORDER
BY list before unique-ifying.  The error seems masked in simple cases by the
fact that query_planner won't return query pathkeys that only partially match
the requested sort order, but I wouldn't want to bet that it couldn't be
exposed in some way or other.
2008-08-05 16:03:10 +00:00
Tom Lane d8b04d5fac In ReadOrZeroBuffer (and related entry points), don't bother to call
PageHeaderIsValid when we zero the buffer instead of reading the page in.
The actual performance improvement is probably marginal since this function
isn't very heavily used, but a cycle saved is a cycle earned.

Zdenek Kotala
2008-08-05 15:09:04 +00:00
Magnus Hagander 70d756970b Move pgstat.tmp into a temporary directory under $PGDATA named pg_stat_tmp.
This allows the use of a ramdrive (either through mount or symlink) for
the temporary file that's written every half second, which should
reduce I/O.

On server shutdown/startup, the file is written to the old location in
the global directory, to preserve data across restarts.

Bump catversion since the $PGDATA directory layout changed.
2008-08-05 12:09:30 +00:00
Tom Lane be3b265c94 Improve SELECT DISTINCT to consider hash aggregation, as well as sort/uniq,
as methods for implementing the DISTINCT step.  This eliminates the former
performance gap between DISTINCT and GROUP BY, and also makes it possible
to do SELECT DISTINCT on datatypes that only support hashing not sorting.

SELECT DISTINCT ON is still always implemented by sorting; it would take
executor changes to support hashing that, and it's not clear it's worth
the trouble.

This is a release-note-worthy incompatibility from previous PG versions,
since SELECT DISTINCT can no longer be counted on to deliver sorted output
without explicitly saying ORDER BY.  (Anyone who can't cope with that
can consider turning off enable_hashagg.)

Several regression test queries needed to have ORDER BY added to preserve
stable output order.  I fixed the ones that manifested here, but there
might be some other cases that show up on other platforms.
2008-08-05 02:43:18 +00:00
Tom Lane 4abd7b49f1 Improve CREATE/DROP/RENAME DATABASE so that when failing because the source
or target database is being accessed by other users, it tells you whether
the "other users" are live sessions or uncommitted prepared transactions.
(Indeed, it tells you exactly how many of each, but that's mostly just
because it was easy to do so.)  This should help forestall the gotcha of
not realizing that a prepared transaction is what's blocking the command.
Per discussion.
2008-08-04 18:03:46 +00:00
Tom Lane ec73b56a31 Make GROUP BY work properly for datatypes that only support hashing and not
sorting.  The infrastructure for this was all in place already; it's only
necessary to fix the planner to not assume that sorting is always an available
option.
2008-08-03 19:10:52 +00:00
Tom Lane 82a1f09953 Tighten up the sanity checks in TypeCreate(): pass-by-value types must have
a size that is one of the supported values, not just anything <= sizeof(Datum).
Cross-check the alignment specification against size as well.
2008-08-03 15:23:58 +00:00
Tom Lane 9511304752 Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items
as per my recent proposal:

1. Fold SortClause and GroupClause into a single node type SortGroupClause.
We were already relying on them to be struct-equivalent, so using two node
tags wasn't accomplishing much except to get in the way of comparing items
with equal().

2. Add an "eqop" field to SortGroupClause to carry the associated equality
operator.  This is cheap for the parser to get at the same time it's looking
up the sort operator, and storing it eliminates the need for repeated
not-so-cheap lookups during planning.  In future this will also let us
represent GROUP/DISTINCT operations on datatypes that have hash opclasses
but no btree opclasses (ie, they have equality but no natural sort order).
The previous representation simply didn't work for that, since its only
indicator of comparison semantics was a sort operator.

3. Add a hasDistinctOn boolean to struct Query to explicitly record whether
the distinctClause came from DISTINCT or DISTINCT ON.  This allows removing
some complicated and not 100% bulletproof code that attempted to figure
that out from the distinctClause alone.

This patch doesn't in itself create any new capability, but it's necessary
infrastructure for future attempts to use hash-based grouping for DISTINCT
and UNION/INTERSECT/EXCEPT.
2008-08-02 21:32:01 +00:00
Alvaro Herrera e36e6b1cab Add a few more DTrace probes to the backend.
Robert Lor
2008-08-01 13:16:09 +00:00
Magnus Hagander 26e6991a2d Rearrange the code in auth.c so that all functions for a single authentication
method is grouped together in a reasonably similar way, keeping the "global
shared functions" together in their own section as well. Makes it a lot easier
to find your way around the code.
2008-08-01 11:41:12 +00:00
Magnus Hagander c30c1b8786 Move ident authentication code into auth.c along with the other authenciation
routines, leaving hba.c to deal only with processing the HBA specific files.
2008-08-01 09:09:49 +00:00
Tom Lane 63247bec28 Fix parser so that we don't modify the user-written ORDER BY list in order
to represent DISTINCT or DISTINCT ON.  This gets rid of a longstanding
annoyance that a view or rule using SELECT DISTINCT will be dumped out
with an overspecified ORDER BY list, and is one small step along the way
to decoupling DISTINCT and ORDER BY enough so that hash-based implementation
of DISTINCT will be possible.  In passing, improve transformDistinctClause
so that it doesn't reject duplicate DISTINCT ON items, as was reported by
Steve Midgley a couple weeks ago.
2008-07-31 22:47:56 +00:00
Tom Lane 7bd7b2002b Require superuser privilege to create base types (but not composites, enums,
or domains).  This was already effectively required because you had to own
the I/O functions, and the I/O functions pretty much have to be written in
C since we don't let PL functions take or return cstring.  But given the
possible security consequences of a malicious type definition, it seems
prudent to enforce superuser requirement directly.  Per recent discussion.
2008-07-31 16:27:16 +00:00
Tom Lane c8572986ad Allow I/O conversion casts to be applied to or from any type that is a member
of the STRING type category, thereby opening up the mechanism for user-defined
types.  This is mainly for the benefit of citext, though; there aren't likely
to be a lot of types that are all general-purpose character strings.
Per discussion with David Wheeler.
2008-07-30 21:23:17 +00:00
Tom Lane 7df49cef72 Flip the default typispreferred setting from true to false. This affects
only type categories in which the previous coding made *every* type
preferred; so there is no change in effective behavior, because the function
resolution rules only do something different when faced with a choice
between preferred and non-preferred types in the same category.  It just
seems safer and less surprising to have CREATE TYPE default to non-preferred
status ...
2008-07-30 19:35:13 +00:00
Tom Lane bac3e83622 Replace the hard-wired type knowledge in TypeCategory() and IsPreferredType()
with system catalog lookups, as was foreseen to be necessary almost since
their creation.  Instead put the information into two new pg_type columns,
typcategory and typispreferred.  Add support for setting these when
creating a user-defined base type.

The category column is just a "char" (i.e. a poor man's enum), allowing
a crude form of user extensibility of the category list: just use an
otherwise-unused character.  This seems sufficient for foreseen uses,
but we could upgrade to having an actual category catalog someday, if
there proves to be a huge demand for custom type categories.

In this patch I have attempted to hew exactly to the behavior of the
previous hardwired logic, except for introducing new type categories for
arrays, composites, and enums.  In particular the default preferred state
for user-defined types remains TRUE.  That seems worth revisiting, but it
should be done as a separate patch from introducing the infrastructure.
Likewise, any adjustment of the standard set of categories should be done
separately.
2008-07-30 17:05:05 +00:00
Tom Lane a77eaa6a95 As noted by Andrew Gierth, there's really no need any more to force a junk
filter to be used when INSERT or SELECT INTO has a plan that returns raw
disk tuples.  The virtual-tuple-slot optimizations that were put in place
awhile ago mean that ExecInsert has to do ExecMaterializeSlot, and that
already copies the tuple if it's raw (and does so more efficiently than
a junk filter, too).  So get rid of that logic.  This in turn means that
we can throw away ExecMayReturnRawTuples, which wasn't used for any other
purpose, and was always a kluge anyway.

In passing, move a couple of SELECT-INTO-specific fields out of EState
and into the private state of the SELECT INTO DestReceiver, as was foreseen
in an old comment there.  Also make intorel_receive use ExecMaterializeSlot
not ExecCopySlotTuple, for consistency with ExecInsert and to possibly save
a tuple copy step in some cases.
2008-07-26 19:15:35 +00:00
Tom Lane 94be06af76 Fix parsing of LDAP URLs so it doesn't reject spaces in the "suffix" part.
Per report from César Miguel Oliveira Alves.
2008-07-24 17:51:55 +00:00
Tom Lane e76ef8d581 Remove some redundant tests and improve comments in next_token().
Cosmetic, but it might make this a bit less confusing to the next reader.
2008-07-24 17:43:45 +00:00
Alvaro Herrera 85dfe376d9 Ratchet up patch to improve autovacuum wraparound messages.
Simon Riggs
2008-07-23 20:20:10 +00:00
Tom Lane 11c794f224 Use guc.c's parse_int() instead of pg_atoi() to parse fillfactor in
default_reloptions().  The previous coding was really a bug because pg_atoi()
will always throw elog on bad input data, whereas default_reloptions is not
supposed to complain about bad input unless its validate parameter is true.
Right now you could only expose the problem by hand-modifying
pg_class.reloptions into an invalid state, so it doesn't seem worth
back-patching; but we should get it right in HEAD because there might be other
situations in future.  Noted while studying GIN fast-update patch.
2008-07-23 17:29:53 +00:00
Alvaro Herrera 0d09688f88 Publish more openly the fact that autovacuum is working for wraparound
protection.

Simon Riggs
2008-07-21 15:27:02 +00:00
Tom Lane b351eba20a Add comment about the two different query strings that ExecuteQuery()
has to deal with.
2008-07-21 15:26:55 +00:00
Tom Lane 5618ece82b Code review for array_fill patch: fix inadequate check for array size overflow
and bogus documentation (dimension arrays are int[] not anyarray).  Also the
errhint() messages seem to be really errdetail(), since there is nothing
heuristic about them.  Some other trivial cosmetic improvements.
2008-07-21 04:47:00 +00:00
Tom Lane 4b362c662e Avoid substituting NAMEDATALEN, FLOAT4PASSBYVAL, and FLOAT8PASSBYVAL into
the postgres.bki file during build, because we want that file to be entirely
platform- and configuration-independent; else it can't safely be put into
/usr/share on multiarch machines.  We can do the substitution during initdb,
instead.  FLOAT4PASSBYVAL and FLOAT8PASSBYVAL are new breakage as of 8.4,
while the NAMEDATALEN hazard has been there all along but I guess no one
tripped over it.  Noticed while trying to build "universal" OS X binaries.
2008-07-19 04:01:29 +00:00
Tom Lane a1c692358b Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement.  That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text.  Fix it to copy the source text over
from the cached plan.  Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.

There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
        ProcessUtility((Node *) atstmt,
                       "(generated ALTER TABLE ADD FOREIGN KEY command)",
                       NULL, false, None_Receiver, NULL);

We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.

ITAGAKI Takahiro and Tom Lane
2008-07-18 20:26:06 +00:00
Tom Lane 6cc88f0af5 Provide a function hook to let plug-ins get control around ExecutorRun.
ITAGAKI Takahiro
2008-07-18 18:23:47 +00:00
Tom Lane dc02a4814a Fix a race condition that I introduced into sinvaladt.c during the recent
rewrite.  When called from SIInsertDataEntries, SICleanupQueue releases
the write lock if it has to issue a kill() to signal some laggard backend.
That still seems like a good idea --- but it's possible that by the time
we get the lock back, there are no longer enough free message slots to
satisfy SIInsertDataEntries' requirement.  Must recheck, and repeat the
whole SICleanupQueue process if not.  Noted while reading code.
2008-07-18 14:45:48 +00:00
Tom Lane 69a785b8bf Implement SQL-spec RETURNS TABLE syntax for functions.
(Unlike the original submission, this patch treats TABLE output parameters
as being entirely equivalent to OUT parameters -- tgl)

Pavel Stehule
2008-07-18 03:32:53 +00:00
Alvaro Herrera 46c5a212ec Avoid crashing when a table is deleted while we're on the process of checking
it.

Per report from Tom Lane based on buildfarm evidence.
2008-07-17 21:02:31 +00:00
Tom Lane a41f73a092 Add dump support for SortBy nodes. Needed this while debugging a reported
problem with DISTINCT, so might as well commit it.
2008-07-17 16:02:12 +00:00
Tom Lane 5ef5abe372 Fix previous patch so that it actually works --- consider TRUNCATE foo, public.foo 2008-07-16 19:33:25 +00:00
Tom Lane 6563e9e2e8 Add a "provariadic" column to pg_proc to eliminate the remarkably expensive
need to deconstruct proargmodes for each pg_proc entry inspected by
FuncnameGetCandidates().  Fixes function lookup performance regression
caused by yesterday's variadic-functions patch.

In passing, make pg_proc.probin be NULL, rather than a dummy value '-',
in cases where it is not actually used for the particular type of function.
This should buy back some of the space cost of the extra column.
2008-07-16 16:55:24 +00:00
Bruce Momjian 895a4bccb6 Allow TRUNCATE foo, foo to succeed, per report from Nikhils. 2008-07-16 16:54:08 +00:00
Tom Lane d89737d31c Support "variadic" functions, which can accept a variable number of arguments
so long as all the trailing arguments are of the same (non-array) type.
The function receives them as a single array argument (which is why they
have to all be the same type).

It might be useful to extend this facility to aggregates, but this patch
doesn't do that.

This patch imposes a noticeable slowdown on function lookup --- a follow-on
patch will fix that by adding a redundant column to pg_proc.

Pavel Stehule
2008-07-16 01:30:23 +00:00
Bruce Momjian 2c773296f8 Add array_fill() to create arrays initialized with a value.
Pavel Stehule
2008-07-16 00:48:54 +00:00
Tom Lane 6f6d863258 Create a type-specific typanalyze routine for tsvector, which collects stats
on the most common individual lexemes in place of the mostly-useless default
behavior of counting duplicate tsvectors.  Future work: create selectivity
estimation functions that actually do something with these stats.

(Some other things we ought to look at doing: using the Lossy Counting
algorithm in compute_minimal_stats, and using the element-counting idea for
stats on regular arrays.)

Jan Urbanski
2008-07-14 00:51:46 +00:00
Tom Lane 6816577a78 Change the PageGetContents() macro to guarantee its result is maxalign'd,
thereby forestalling any problems with alignment of the data structure placed
there.  Since SizeOfPageHeaderData is maxalign'd anyway in 8.3 and HEAD, this
does not actually change anything right now, but it is foreseeable that the
header size will change again someday.  I had to fix a couple of places that
were assuming that the content offset is just SizeOfPageHeaderData rather than
MAXALIGN(SizeOfPageHeaderData).  Per discussion of Zdenek's page-macros patch.
2008-07-13 21:50:04 +00:00
Tom Lane 9d035f4254 Clean up the use of some page-header-access macros: principally, use
SizeOfPageHeaderData instead of sizeof(PageHeaderData) in places where that
makes the code clearer, and avoid casting between Page and PageHeader where
possible.  Zdenek Kotala, with some additional cleanup by Heikki Linnakangas.

I did not apply the parts of the proposed patch that would have resulted in
slightly changing the on-disk format of hash indexes; it seems to me that's
not a win as long as there's any chance of having in-place upgrade for 8.4.
2008-07-13 20:45:47 +00:00
Peter Eisentraut 96193aa803 More replacements of binary compatible to binary coercible. 2008-07-12 10:44:56 +00:00
Tom Lane 960af47efd Const-ify the arguments of str_tolower() and friends to suppress compile
warnings.  Clean up various unneeded cruft that was left behind after
creating those routines.  Introduce some convenience functions str_tolower_z
etc to eliminate tedious and error-prone double arguments in formatting.c.
(Currently there seems no need to export the latter, but maybe reconsider
this later.)
2008-07-12 00:44:38 +00:00
Tom Lane 27cb66fdfe Multi-column GIN indexes. Teodor Sigaev 2008-07-11 21:06:29 +00:00
Peter Eisentraut e3afbb3504 Allow binary-coercible types for cast function arguments and return types.
Document return type of cast functions.

Also change documentation to prefer the term "binary coercible" in its
present sense instead of the previous term "binary compatible".
2008-07-11 07:02:43 +00:00
Alvaro Herrera 110147653a Make sure we only try to free snapshots that have been passed through
CopySnapshot, per Neil Conway.  Also add a comment about the assumption in
GetSnapshotData that the argument is statically allocated.

Also, fix some more typos in comments in snapmgr.c.
2008-07-11 02:10:14 +00:00
Neil Conway 0c2914d4cb Fix a few typos in comments in snapmgr.c, and sort header inclusions
alphabetically.
2008-07-11 00:00:29 +00:00
Tom Lane 7a97abe818 Add unchangeable GUC "variables" segment_size, wal_block_size, and
wal_segment_size to make those configuration parameters available to clients,
in the same way that block_size was previously exposed.  Bernd Helmle, with
comments from Abhijit Menon-Sen and some further tweaking by me.
2008-07-10 22:08:17 +00:00
Tom Lane eaf1b5d348 Tighten up SS_finalize_plan's computation of valid_params to exclude Params of
the current query level that aren't in fact output parameters of the current
initPlans.  (This means, for example, output parameters of regular subplans.)
To make this work correctly for output parameters coming from sibling
initplans requires rejiggering the API of SS_finalize_plan just a bit:
we need the siblings to be visible to it, rather than hidden as
SS_make_initplan_from_plan had been doing.  This is really part of my response
to bug #4290, but I concluded this part probably shouldn't be back-patched,
since all that it's doing is to make a debugging cross-check tighter.
2008-07-10 02:14:03 +00:00
Tom Lane 772a6d45ef Fix mis-calculation of extParam/allParam sets for plan nodes, as seen in
bug #4290.  The fundamental bug is that masking extParam by outer_params,
as finalize_plan had been doing, caused us to lose the information that
an initPlan depended on the output of a sibling initPlan.  On reflection
the best thing to do seemed to be not to try to adjust outer_params for
this case but get rid of it entirely.  The only thing it was really doing
for us was to filter out param IDs associated with SubPlan nodes, and that
can be done (with greater accuracy) while processing individual SubPlan
nodes in finalize_primnode.  This approach was vindicated by the discovery
that the masking method was hiding a second bug: SS_finalize_plan failed to
remove extParam bits for initPlan output params that were referenced in the
main plan tree (it only got rid of those referenced by other initPlans).
It's not clear that this caused any real problems, given the limited use
of extParam by the executor, but it's certainly not what was intended.

I originally thought that there was also a problem with needing to include
indirect dependencies on external params in initPlans' param sets, but it
turns out that the executor handles this correctly so long as the depended-on
initPlan is earlier in the initPlans list than the one using its output.
That seems a bit of a fragile assumption, but it is true at the moment,
so I just documented it in some code comments rather than making what would
be rather invasive changes to remove the assumption.

Back-patch to 8.1.  Previous versions don't have the case of initPlans
referring to other initPlans' outputs, so while the existing logic is still
questionable for them, there are not any known bugs to be fixed.  So I'll
refrain from changing them for now.
2008-07-10 01:17:29 +00:00
Tom Lane 6b7eebc05e Increase PG_SYSLOG_LIMIT (the max line length sent to syslog()) from 128 to
1024 to improve performance when sending large elog messages.  Also add a
comment about why we use that number.

Since this represents an externally visible behavior change, and might
possibly result in portability issues, it seems best not to back-patch it.
2008-07-09 15:56:49 +00:00
Tom Lane 3793310286 Fix performance bug in write_syslog(): the code to preferentially break the
log message at newlines cost O(N^2) for very long messages with few or no
newlines.  For messages in the megabyte range this became the dominant cost.
Per gripe from Achilleas Mantzios.

Patch all the way back, since this is a safe change with no portability
risks.  I am also thinking of increasing PG_SYSLOG_LIMIT, but that should
be done separately.
2008-07-08 22:17:41 +00:00
Neil Conway 68af3752de Minor improvements to the Gin internal documentation. 2008-07-08 03:25:42 +00:00
Bruce Momjian 70d15a51b2 Add comment for deadlock_timeout:
/* This is PGC_SIGHUP so all backends have the same value. */
2008-07-08 02:07:29 +00:00
Tom Lane 170063cd1e Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean
results always contribute two groups, regardless of the expression contents.
This is very substantially more accurate than the regular heuristic for
certain boolean tests like "col IS NULL".  Per gripe from Sam Mason.

Back-patch to all supported releases, since the behavior of
estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 20:24:55 +00:00
Tom Lane c50838533b Fix AT TIME ZONE (in all three variants) so that we first try to interpret
the timezone argument as a timezone abbreviation, and only try it as a full
timezone name if that fails.  The zic database has four zones (CET, EET, MET,
WET) that are full daylight-savings zones and yet have names that are the
same as their abbreviations for standard time, resulting in ambiguity.
In the timestamp input functions we resolve the ambiguity by preferring the
abbreviation, and AT TIME ZONE should work the same way.  (No functionality
is lost because the zic database also has other names for these zones, eg
Europe/Zurich.)  Per gripe from Jaromir Talir.

Backpatch to 8.1.  Older releases did not have the issue because AT TIME ZONE
only accepted abbreviations not zone names.  (Thus, this patch also arguably
fixes a compatibility botch introduced at 8.1: in ambiguous cases we now
behave the same as 8.0 did.)
2008-07-07 18:09:46 +00:00
Tom Lane fbcc69c192 Prevent integer overflows during units conversion when displaying a GUC
variable that has units.  Per report from Stefan Kaltenbrunner.

Backport to 8.2.  I also backported my patch of 2007-06-21 that prevented
comparable overflows on the input side, since that now seems to have enough
field track record to be back-patched safely.  That patch included addition
of hints listing the available unit names, which I did not bother to strip
out of it --- this will make a little more work for the translators, but
they can copy the translation from 8.3, and anyway an untranslated hint
is better than no hint.
2008-07-06 19:48:45 +00:00
Teodor Sigaev 2a59b7910e Fix initialization of GinScanEntryData.partialMatch 2008-07-04 13:21:18 +00:00
Magnus Hagander d06a8d054d Fix a couple of bugs in win32 shmem name generation:
* Don't cut off the prefix. With this fix, it's again readable.
* Properly store it in the Global namespace as intended.
2008-07-04 10:50:18 +00:00
Tom Lane c63147d6f0 Add a function pg_get_keywords() to let clients find out the set of keywords
known to the SQL parser.  Dave Page
2008-07-03 20:58:47 +00:00
Tom Lane c5f4b98fae Fix transaction-lifespan memory leak in xpath(). Report by Matt Magoffin,
fix by Kris Jurka.
2008-07-03 00:04:24 +00:00
Tom Lane 009a6c9a1a Remove GUC extra_desc strings that are redundant with the enum value lists. 2008-07-01 21:07:33 +00:00
Heikki Linnakangas 3ccb2c590c Extend VacAttrStats to allow typanalyze functions to store statistic values
of different types than the underlying column. The capability isn't yet
used for anything, but will be required by upcoming patch to analyze
tsvector columns.

Jan Urbanski
2008-07-01 10:33:09 +00:00
Magnus Hagander baaad2330b "debug" level was supposed to be hidden, since it's just an alias for debug2. 2008-07-01 06:36:11 +00:00
Magnus Hagander 7b39f488b4 Split apart message_level_options into one set for server-side settings and
one for client-side, restoring the previous behaviour with different
sort order for the 'log' level. Also, remove redundant list of available
options, since the enum code will output it automatically.
2008-07-01 06:08:31 +00:00
Tom Lane 5b965bf08b Teach autovacuum how to determine whether a temp table belongs to a crashed
backend.  If so, send a LOG message to the postmaster log, and if the table
is beyond the vacuum-for-wraparound horizon, forcibly drop it.  Per recent
discussions.  Perhaps we ought to back-patch this, but it probably needs
to age a bit in HEAD first.
2008-07-01 02:09:34 +00:00
Bruce Momjian 6b797c852b Fix recovery.conf boolean variables to take the same range of string
values as postgresql.conf.
2008-06-30 22:10:43 +00:00
Heikki Linnakangas 995fb74202 Turn PGBE_ACTIVITY_SIZE into a GUC variable, track_activity_query_size.
As the buffer could now be a lot larger than before, and copying it could
thus be a lot more expensive than before, use strcpy instead of memcpy to
copy the query string, as was already suggested in comments. Also, only copy
the PgBackendStatus struct and string if the slot is in use.

Patch by Thomas Lee, with some changes by me.
2008-06-30 10:58:47 +00:00
Tom Lane 7ea9b997ef Remove unnecessary coziness of GIN code with datum copying. Now that
space is tracked via GetMemoryChunkSpace, there's really no advantage
to duplicating datumCopy's innards here.  This is one bit of my toast
indirection patch that should go in anyway.
2008-06-29 21:04:01 +00:00
Tom Lane 4a8d573cda If pnstrdup is going to be promoted to a generally available function,
it ought to conform to the rest of palloc.h in using Size for sizes.
2008-06-28 16:45:22 +00:00
Tom Lane dcc2334736 Consider a clause to be outerjoin_delayed if it references the nullable side
of any lower outer join, even if it also references the non-nullable side and
so could not get pushed below the outer join anyway.  We need this in case
the clause is an OR clause: if it doesn't get marked outerjoin_delayed,
create_or_index_quals() could pull an indexable restriction for the nullable
side out of it, leading to wrong results as demonstrated by today's bug
report from toruvinn.  (See added regression test case for an example.)

In principle this has been wrong for quite a while.  In practice I don't
think any branch before 8.3 can really show the failure, because
create_or_index_quals() will only pull out indexable conditions, and before
8.3 those were always strict.  So though we might have improperly generated
null-extended rows in the outer join, they'd get discarded from the result
anyway.  The gating factor that makes the failure visible is that 8.3
considers "col IS NULL" to be indexable.  Hence I'm not going to risk
back-patching further than 8.3.
2008-06-27 20:54:37 +00:00
Tom Lane 2c2161a47d Improve planner's estimation of the size of an append relation: rather than
taking the maximum of any child rel's width, we should weight the widths
proportionally to the number of rows expected from each child.  In hindsight
this is obviously correct because row width is really a proxy for the total
physical size of the relation.  Per discussion with Scott Carey (bug #4264).
2008-06-27 03:56:55 +00:00
Teodor Sigaev 5ff9899933 Fix bug "select lower('asd') = 'asd'" returns false with multibyte encoding
and non-C locale. Fix is just to use correct source's length for char2wchar
call.
2008-06-26 16:06:37 +00:00
Bruce Momjian 067f1e5fa8 Fix 'pg_ctl restart' to preserve command-line arguments. 2008-06-26 02:47:19 +00:00
Bruce Momjian a1183238be Use SYSTEMQUOTE as concatentation to strings, rather than %s printf
patterns, for clarity.
2008-06-26 01:35:45 +00:00
Tom Lane 5f6f840e93 Reduce the alignment requirement of type "name" from int to char, and arrange
to suppress zero-padding of "name" entries in indexes.

The alignment change is unlikely to save any space, but it is really needed
anyway to make the world safe for our widespread practice of passing plain
old C strings to functions that are declared as taking Name.  In the previous
coding, the C compiler was entitled to assume that a Name pointer was
word-aligned; but we were failing to guarantee that.  I think the reason
we'd not seen failures is that usually the only thing that gets done with
such a pointer is strcmp(), which is hard to optimize in a way that exploits
word-alignment.  Still, some enterprising compiler guy will probably think
of a way eventually, or we might change our code in a way that exposes
more-obvious optimization opportunities.

The padding change is accomplished in one-liner fashion by declaring the
"name" index opclasses to use storage type "cstring" in pg_opclass.h.
Normally btree and hash don't allow a nondefault storage type, because they
don't have any provisions for converting the input datum to another type.
However, because name and cstring are effectively the same thing except for
padding, no conversion is needed --- we only need index_form_tuple() to treat
the datum as being cstring not name, and this is sufficient.  This seems to
make for about a one-third reduction in the typical sizes of system catalog
indexes that involve "name" columns, of which we have many.

These two changes are only weakly related, but the alignment change makes
me feel safer that the padding change won't introduce problems, so I'm
committing them together.
2008-06-24 17:58:27 +00:00
Bruce Momjian f6ec7430f9 Merge duplicate upper/lower/initcap() routines in oracle_compat.c and
formatting.c to use common code;  remove duplicate functions and support
routines that are no longer needed.
2008-06-23 19:27:19 +00:00
Tom Lane eeee06919f Fix Gen_fmgrtab.sh to not rely on hard-wired knowledge of the column numbers
in pg_proc.  Also make it not emit duplicate extern declarations, and make it
a bit more bulletproof in some other small ways.  Likewise fix the equally
hard-wired, and utterly undocumented, knowledge in the MSVC build scripts.
For testing purposes and perhaps other uses in future, pull out that portion
of the MSVC scripts into a standalone perl script equivalent to
Gen_fmgrtab.sh, and make it generate actually identical output, rather than
just more-or-less-the-same output.

Motivated by looking at Pavel's variadic function patch.  Whether or not
that gets accepted, we can be sure that pg_proc's column set will change
again in the future; it's time to not have to deal with this gotcha.
2008-06-23 17:54:30 +00:00
Tom Lane dab421d2f0 Seems I was too optimistic in supposing that sinval's maxMsgNum could be
read and written without a lock.  The value itself is atomic, sure, but on
processors with weak memory ordering it's possible for a reader to see the
value change before it sees the associated message written into the buffer
array.  Fix by introducing a spinlock that's used just to read and write
maxMsgNum.  (We could do this with less overhead if we recognized a concept
of "memory access barrier"; is it worth introducing such a thing?  At the
moment probably not --- I can't measure any clear slowdown from adding the
spinlock, so this solution is probably fine.)  Per buildfarm results.
2008-06-20 00:24:53 +00:00
Tom Lane fad153ec45 Rewrite the sinval messaging mechanism to reduce contention and avoid
unnecessary cache resets.  The major changes are:

* When the queue overflows, we only issue a cache reset to the specific
backend or backends that still haven't read the oldest message, rather
than resetting everyone as in the original coding.

* When we observe backend(s) falling well behind, we signal SIGUSR1
to only one backend, the one that is furthest behind and doesn't already
have a signal outstanding for it.  When it finishes catching up, it will
in turn signal SIGUSR1 to the next-furthest-back guy, if there is one that
is far enough behind to justify a signal.  The PMSIGNAL_WAKEN_CHILDREN
mechanism is removed.

* We don't attempt to clean out dead messages after every message-receipt
operation; rather, we do it on the insertion side, and only when the queue
fullness passes certain thresholds.

* Split SInvalLock into SInvalReadLock and SInvalWriteLock so that readers
don't block writers nor vice versa (except during the infrequent queue
cleanout operations).

* Transfer multiple sinval messages for each acquisition of a read or
write lock.
2008-06-19 21:32:56 +00:00
Tom Lane 30dc388a0d Fix a few places that were non-multibyte-safe in tsearch configuration file
parsing.  Per bug #4253 from Giorgio Valoti.
2008-06-19 16:52:24 +00:00
Alvaro Herrera a3540b0f65 Improve our #include situation by moving pointer types away from the
corresponding struct definitions.  This allows other headers to avoid including
certain highly-loaded headers such as rel.h and relscan.h, instead using just
relcache.h, heapam.h or genam.h, which are more lightweight and thus cause less
unnecessary dependencies.
2008-06-19 00:46:06 +00:00
Tom Lane d1da215d32 Fix compiler warning introduced by recent patch. Tsk tsk. 2008-06-18 23:08:47 +00:00
Tom Lane fbeb9da22b Improve error reporting for problems in text search configuration files
by installing an error context subroutine that will provide the file name
and line number for all errors detected while reading a config file.
Some of the reader routines were already doing that in an ad-hoc way for
errors detected directly in the reader, but it didn't help for problems
detected in subroutines, such as encoding violations.

Back-patch to 8.3 because 8.3 is where people will be trying to debug
configuration files.
2008-06-18 20:55:42 +00:00
Bruce Momjian 9de09c087d Move wchar2char() and char2wchar() from tsearch into /mb to be easier to
use for other modules;  also move pnstrdup().

Clean up code slightly.
2008-06-18 18:42:54 +00:00
Tom Lane 86fdb32bd0 Remove freeBackends counter from the sinval shared memory area. We used to
use it to help enforce superuser_reserved_backends, but since 8.1 it's
just been dead weight.
2008-06-17 20:07:08 +00:00
Tom Lane b163baa89c Clean up some problems with redundant cross-type arithmetic operators. Add
int2-and-int8 implementations of the basic arithmetic operators +, -, *, /.
This doesn't really add any new functionality, but it avoids "operator is not
unique" failures that formerly occurred in these cases because the parser
couldn't decide whether to promote the int2 to int4 or int8.  We could
alternatively have removed the existing cross-type operators, but
experimentation shows that the cost of an additional type coercion expression
node is noticeable compared to such cheap operators; so let's not give up any
performance here.  On the other hand, I removed the int2-and-int4 modulo (%)
operators since they didn't seem as important from a performance standpoint.
Per a complaint last January from ykhuang.
2008-06-17 19:10:56 +00:00
Bruce Momjian 4274726d42 Add URL for introduction to multibyte programming in C. 2008-06-17 18:22:43 +00:00
Bruce Momjian dc69c0362f Move USE_WIDE_UPPER_LOWER define to c.h, and remove TS_USE_WIDE and use
USE_WIDE_UPPER_LOWER instead.
2008-06-17 16:09:06 +00:00
Tom Lane 2e835a4961 Fix the code that adds regclass constants to a plan's list of relation OIDs
that it depends on for replan-forcing purposes.  We need to consider plain OID
constants too, because eval_const_expressions folds a RelabelType atop a Const
to just a Const.  This change could result in OID values that aren't really
for tables getting added to the dependency list, but the worst-case
consequence would be occasional useless replans.  Per report from Gabriele
Messineo.
2008-06-17 14:51:32 +00:00
Tom Lane 906f27dd73 Make DROP INDEX lock the parent table before locking the index. This behavior
is necessary to avoid deadlock against ordinary queries, but we'd broken it
with recent changes that made the DROP machinery lock the index before
arriving at index_drop.  Per intermittent buildfarm failures.
2008-06-15 16:29:05 +00:00
Tom Lane 71ff461a18 Fix 64-bit problem in recent patch. 2008-06-15 01:41:37 +00:00
Tom Lane a0b012a1ab Rearrange ALTER TABLE syntax processing as per my recent proposal: the
grammar allows ALTER TABLE/INDEX/SEQUENCE/VIEW interchangeably for all
subforms of those commands, and then we sort out what's really legal
at execution time.  This allows the ALTER SEQUENCE/VIEW reference pages
to fully document all the ALTER forms available for sequences and views
respectively, and eliminates a longstanding cause of confusion for users.

The net effect is that the following forms are allowed that weren't before:
	ALTER SEQUENCE OWNER TO
	ALTER VIEW ALTER COLUMN SET/DROP DEFAULT
	ALTER VIEW OWNER TO
	ALTER VIEW SET SCHEMA
(There's no actual functionality gain here, but formerly you had to say
ALTER TABLE instead.)

Interestingly, the grammar tables actually get smaller, probably because
there are fewer special cases to keep track of.

I did not disallow using ALTER TABLE for these operations.  Perhaps we
should, but there's a backwards-compatibility issue if we do; in fact
it would break existing pg_dump scripts.  I did however tighten up
ALTER SEQUENCE and ALTER VIEW to reject non-sequences and non-views
in the new cases as well as a couple of cases where they didn't before.

The patch doesn't change pg_dump to use the new syntaxes, either.
2008-06-15 01:25:54 +00:00
Tom Lane 0cefb50f3c Refactor the handling of the various DropStmt variants so that when multiple
objects are specified, we drop them all in a single performMultipleDeletions
call.  This makes the RESTRICT/CASCADE checks more relaxed: it's not counted
as a cascade if one of the later objects has a dependency on an earlier one.
NOTICE messages about such cases go away, too.

In passing, fix the permissions check for DROP CONVERSION, which for some
reason was never made role-aware, and omitted the namespace-owner exemption
too.

Alex Hunsaker, with further fiddling by me.
2008-06-14 18:04:34 +00:00
Tom Lane 55a56845ed Improve the various elog messages in tuptoaster.c to report which TOAST table
the problem happened in.  These are all supposedly can't-happen cases, but
when they do happen it's useful to know where.

Back-patch to 8.3, but not further because the patch doesn't apply cleanly
further back.  Given the lack of response to my proposal of this, there
doesn't seem to be enough interest to justify much back-porting effort.
2008-06-13 02:59:47 +00:00
Heikki Linnakangas a213f1ee6c Refactor XLogOpenRelation() and XLogReadBuffer() in preparation for relation
forks. XLogOpenRelation() and the associated light-weight relation cache in
xlogutils.c is gone, and XLogReadBuffer() now takes a RelFileNode as argument,
instead of Relation.

For functions that still need a Relation struct during WAL replay, there's a
new function called CreateFakeRelcacheEntry() that returns a fake entry like
XLogOpenRelation() used to.
2008-06-12 09:12:31 +00:00
Tom Lane c4f2a0458d Improve reporting of dependencies in DROP to work like the scheme that we
devised for pg_shdepend, namely the individual dependencies are reported as
DETAIL lines rather than coming out as separate NOTICEs.  The client-side
report is capped at 100 lines, but the server log always gets a full report.
2008-06-11 21:53:49 +00:00
Bruce Momjian 70da495d84 Fix spelling mistake in postgresql.conf.
Greg Sabino Mullane
2008-06-11 15:44:52 +00:00
Heikki Linnakangas 96675bff1f Fix bug in the WAL recovery code to finish an incomplete split.
CacheInvalidateRelcache() crashes if called in WAL recovery, because the
invalidation infrastructure hasn't been initialized yet.

Back-patch to 8.2, where the bug was introduced.
2008-06-11 08:38:56 +00:00
Tom Lane 0b510ad920 Fix unportable (and incorrect anyway) usage of LL constant suffix that
recently snuck into cash.c.  Per report from Edmundo Robles Lopez.
2008-06-09 19:58:39 +00:00