Commit Graph

20230 Commits

Author SHA1 Message Date
Tom Lane 29c4ad9829 Support "x IS NOT NULL" clauses as indexscan conditions. This turns out
to be just a minor extension of the previous patch that made "x IS NULL"
indexable, because we can treat the IS NOT NULL condition as if it were
"x < NULL" or "x > NULL" (depending on the index's NULLS FIRST/LAST option),
just like IS NULL is treated like "x = NULL".  Aside from any possible
usefulness in its own right, this is an important improvement for
index-optimized MAX/MIN aggregates: it is now reliably possible to get
a column's min or max value cheaply, even when there are a lot of nulls
cluttering the interesting end of the index.
2010-01-01 21:53:49 +00:00
Magnus Hagander 15faca2596 Silence compiler warning on 64-bit windows build 2010-01-01 19:57:07 +00:00
Magnus Hagander f9c5d3e555 Detect a 64-bit build environment on Windows, and generate the appropriate
project files.

Based on the work of Tsutomu Yamada, but much refactored.
2010-01-01 17:34:25 +00:00
Magnus Hagander 5069e959b0 Delete solution cache file on clean, if it exists. 2010-01-01 16:58:53 +00:00
Peter Eisentraut 2c4d456d51 Update SQL features supported list 2010-01-01 16:54:48 +00:00
Magnus Hagander 741e4ad7de Make the win32 putenv() override update *all* present versions of the
MSVCRxx runtime, not just the current + Visual Studio 6 (MSVCRT). Clearly
there can be an almost unlimited number of runtimes loaded at the same
time.

Per report from Hiroshi Inoue
2010-01-01 14:57:16 +00:00
Tom Lane 2aff8c422e Suppress compiler warning, per buildfarm member narwhal 2009-12-31 22:11:10 +00:00
Tom Lane bd8a35655b Suppress compiler warning (pid_t isn't int everywhere) 2009-12-31 22:07:36 +00:00
Tom Lane b4594a66ba Add missing 'static' tag. 2009-12-31 21:47:12 +00:00
Tom Lane 85d02a6586 Redefine Datum as uintptr_t, instead of unsigned long.
This is more in keeping with modern practice, and is a first step towards
porting to Win64 (which has sizeof(pointer) > sizeof(long)).

Tsutomu Yamada, Magnus Hagander, Tom Lane
2009-12-31 19:41:37 +00:00
Peter Eisentraut 8abb011047 Update SQL features list for aggregate ORDER BY support 2009-12-31 14:51:16 +00:00
Peter Eisentraut c584d11bb3 Add information_schema.triggered_update_columns
This reflects the recently added support for triggers on columns.
2009-12-31 14:41:23 +00:00
Andrew Dunstan 31cf893088 enable examples to compile on Windows. patch from Hiroshi Saito. 2009-12-31 00:16:47 +00:00
Peter Eisentraut 3c505ef577 Fill in information schema column for trigger WHEN condition 2009-12-30 22:48:10 +00:00
Tom Lane e6df063cf2 Dept of second thoughts: recursive case in ANALYZE shouldn't emit a
pgstats message.  This might need to be done differently later, but
with the current logic that's what should happen.
2009-12-30 21:21:33 +00:00
Tom Lane 48c192c15e Revise pgstat's tracking of tuple changes to improve the reliability of
decisions about when to auto-analyze.

The previous code depended on n_live_tuples + n_dead_tuples - last_anl_tuples,
where all three of these numbers could be bad estimates from ANALYZE itself.
Even worse, in the presence of a steady flow of HOT updates and matching
HOT-tuple reclamations, auto-analyze might never trigger at all, even if all
three numbers are exactly right, because n_dead_tuples could hold steady.

To fix, replace last_anl_tuples with an accurately tracked count of the total
number of committed tuple inserts + updates + deletes since the last ANALYZE
on the table.  This can still be compared to the same threshold as before, but
it's much more trustworthy than the old computation.  Tracking this requires
one more intra-transaction counter per modified table within backends, but no
additional memory space in the stats collector.  There probably isn't any
measurable speed difference; if anything it might be a bit faster than before,
since I was able to eliminate some per-tuple arithmetic operations in favor of
adding sums once per (sub)transaction.

Also, simplify the logic around pgstat vacuum and analyze reporting messages
by not trying to fold VACUUM ANALYZE into a single pgstat message.

The original thought behind this patch was to allow scheduling of analyzes
on parent tables by artificially inflating their changes_since_analyze count.
I've left that for a separate patch since this change seems to stand on its
own merit.
2009-12-30 20:32:14 +00:00
Peter Eisentraut 6761cff376 Update SQL conformance: search conditions on triggers are supported 2009-12-30 19:37:47 +00:00
Magnus Hagander 50a4054d63 Exclude part of the product name string that is localized in Japanese versions
of MSVC when detecting MSVC version.

Hiroshi Inoue
2009-12-30 12:26:41 +00:00
Heikki Linnakangas ff1e1e45b9 Reset minRecoveryPoint at checkpoints, so that we don't uselessly update
it in the control file at crash recovery following an archive recovery.

Per Fujii Masao and subsequent discussion.
2009-12-30 08:37:21 +00:00
Tom Lane 4847d5956c Set errno to zero before invoking SSL_read or SSL_write. It appears that
at least in some Windows versions, these functions are capable of returning
a failure indication without setting errno.  That puts us into an infinite
loop if the previous value happened to be EINTR.  Per report from Brendan
Hill.

Back-patch to 8.2.  We could take it further back, but since this is only
known to be an issue on Windows and we don't support Windows before 8.2,
it does not seem worth the trouble.
2009-12-30 03:45:46 +00:00
Robert Haas 3d4b0ab29c Reject invalid input in int2vectorin.
Since the int2vector type is intended only for internal use, this patch doesn't
worry about prettifying the error messages, which has the fringe benefit of
avoiding creating additional translatable strings.  For a type intended to be
used by end-users, we would want to do better, but the approach taken here
seems like the correct trade-off for this case.

Caleb Welton
2009-12-30 01:29:22 +00:00
Tom Lane 540e69a061 Add an index on pg_inherits.inhparent, and use it to avoid seqscans in
find_inheritance_children().  This is a complete no-op in databases without
any inheritance.  In databases where there are just a few entries in
pg_inherits, it could conceivably be a small loss.  However, in databases with
many inheritance parents, it can be a big win.
2009-12-29 22:00:14 +00:00
Tom Lane 649b5ec7c8 Add the ability to store inheritance-tree statistics in pg_statistic,
and teach ANALYZE to compute such stats for tables that have subclasses.
Per my proposal of yesterday.

autovacuum still needs to be taught about running ANALYZE on parent tables
when their subclasses change, but the feature is useful even without that.
2009-12-29 20:11:45 +00:00
Heikki Linnakangas 84d723b6ce Previous fix for temporary file management broke returning a set from
PL/pgSQL function within an exception handler. Make sure we use the right
resource owner when we create the tuplestore to hold returned tuples.

Simplify tuplestore API so that the caller doesn't need to be in the right
memory context when calling tuplestore_put* functions. tuplestore.c
automatically switches to the memory context used when the tuplestore was
created. Tuplesort was already modified like this earlier. This patch also
removes the now useless MemoryContextSwitch calls from callers.

Report by Aleksei on pgsql-bugs on Dec 22 2009. Backpatch to 8.1, like
the previous patch that broke this.
2009-12-29 17:40:59 +00:00
Bruce Momjian 0d399d57d5 Remove PGDLLIMPORT used for binary upgrade; must be on the externs, per Tom. 2009-12-28 18:49:05 +00:00
Bruce Momjian 3687b2e0eb Add PGDLLIMPORT for binary_upgrade global variables so shared object
libraries can access them.
2009-12-28 18:39:03 +00:00
Bruce Momjian d2225da79f Remove non-ascii characters from source code. 2009-12-28 18:09:44 +00:00
Tom Lane 0b39231431 Avoid memory leak if pgstat_vacuum_stat is interrupted partway through.
The temporary hash tables made by pgstat_collect_oids should be allocated
in a short-term memory context, which is not the default behavior of
hash_create.  Noted while looking through hash_create calls in connection
with Robert Haas' recent complaint.

This is a pre-existing bug, but it doesn't seem important enough to
back-patch.  The hash table is not so large that it would matter unless this
happened many times within a session, which seems quite unlikely.
2009-12-27 19:40:07 +00:00
Tom Lane d4d1885e42 Remove a couple of unnecessary calls of CreateCacheMemoryContext. These
probably got there via blind copy-and-paste from one of the legitimate
callers, so rearrange and comment that code a bit to make it clearer that
this isn't a necessary prerequisite to hash_create.  Per observation
from Robert Haas.
2009-12-27 18:55:52 +00:00
Magnus Hagander c4371cdb8b Remove now unreferenced variable. 2009-12-27 16:11:28 +00:00
Magnus Hagander 8c940a30ae If the MSVCRT module is not found in the current binary, proceed to update
system and local environments anyway, instead of aborting. (This will
happen in a MSVC build with no or very few external libraries linked in)
2009-12-27 16:01:39 +00:00
Bruce Momjian e5b457c2ac Add backend and pg_dump code to allow preservation of pg_enum oids, for
use in binary upgrades.

Bump catalog version for detection by pg_migrator of new backend API.
2009-12-27 14:50:46 +00:00
Bruce Momjian 1fd9883ff4 Zero-label enums:
Allow enums to be created with zero labels, for use during binary upgrade.
2009-12-26 16:55:21 +00:00
Tom Lane f9845aca2b Fix brain fade in join-removal patch: a pushed-down clause in the outer join's
restrict list is not just something to ignore, it's actually grounds to
abandon the optimization entirely.  Per bug #5255 from Matteo Beccati.
2009-12-25 17:11:32 +00:00
Andrew Dunstan 56a4e19c01 Upgrade to latest ppport.h. Patch from Tim Bunce. 2009-12-25 00:24:59 +00:00
Bruce Momjian 0ef5910d6d Rename EnumValuesCreate() single-letter variable names to useful
variable names.
2009-12-24 22:17:58 +00:00
Bruce Momjian c44327afa4 Binary upgrade:
Modify pg_dump --binary-upgrade and add backend support routines to
support the preservation of pg_type oids when doing a binary upgrade.
This allows user-defined composite types and arrays to be binary
upgraded.
2009-12-24 22:09:24 +00:00
Tom Lane 668e37d138 Fix wrong WAL info value generated when gistContinueInsert() performs an
index page split.  This would result in index corruption, or even more likely
an error during WAL replay, if we were unlucky enough to crash during
end-of-recovery cleanup after having completed an incomplete GIST insertion.

Yoichi Hirai
2009-12-24 17:52:04 +00:00
Tom Lane d68e08d1fe Allow the index name to be omitted in CREATE INDEX, causing the system to
choose an index name the same as it would do for an unnamed index constraint.
(My recent changes to the index naming logic have helped to ensure that this
will be a reasonable choice.)  Per a suggestion from Peter.

A necessary side-effect is to promote CONCURRENTLY to type_func_name_keyword
status, ie, it can't be a table/column/index name anymore unless quoted.
This is not all bad, since we have heard more than once of people typing
CREATE INDEX CONCURRENTLY ON foo (...) and getting a normal index build of
an index named "concurrently", which was not what they wanted.  Now this
syntax will result in a concurrent build of an index with system-chosen
name; which they can rename afterwards if they want something else.
2009-12-23 17:41:45 +00:00
Tom Lane c176e12222 Remove code that attempted to rename index columns to keep them in sync with
their underlying table columns.  That code was not bright enough to cope with
collision situations (ie, new name conflicts with some other column of the
index).  Since there is no functional reason to do this at all, trying to
upgrade the logic to be bulletproof doesn't seem worth the trouble.

This change means that both the index name and the column names of an index
are set when it's created, and won't be automatically changed when the
underlying table columns are renamed.  Neatnik DBAs are still free to rename
them manually, of course.
2009-12-23 16:43:43 +00:00
Magnus Hagander df0cdd53d6 Add basic build support for Visual Studio 2008, without resorting to
generating the build files for 2005 and then converting them.
2009-12-23 13:27:04 +00:00
Heikki Linnakangas 4e766f2d17 Always pass catalog id to the options validator function specified in
CREATE FOREIGN DATA WRAPPER. Arguably it wasn't a bug because the
documentation said that it's passed the catalog ID or zero, but surely
we should provide it when it's known. And there isn't currently any
scenario where it's not known, and I can't imagine having one in the
future either, so better remove the "or zero" escape hatch and always
pass a valid catalog ID. Backpatch to 8.4.

Martin Pihlak
2009-12-23 12:23:59 +00:00
Bruce Momjian b683908064 Rename pg_dump.c TypeInfo variable tinfo to tyinfo, for clarity. 2009-12-23 04:10:50 +00:00
Tom Lane cfc5008a51 Adjust naming of indexes and their columns per recent discussion.
Index expression columns are now named after the FigureColname result for
their expressions, rather than always being "pg_expression_N".  Digits are
appended to this name if needed to make the column name unique within the
index.  (That happens for regular columns too, thus fixing the old problem
that CREATE INDEX fooi ON foo (f1, f1) fails.  Before exclusion indexes
there was no real reason to do such a thing, but now maybe there is.)

Default names for indexes and associated constraints now include the column
names of all their columns, not only the first one as in previous practice.
(Of course, this will be truncated as needed to fit in NAMEDATALEN.  Also,
pkey indexes retain the historical behavior of not naming specific columns
at all.)

An example of the results:

regression=# create table foo (f1 int, f2 text,
regression(# exclude (f1 with =, lower(f2) with =));
NOTICE:  CREATE TABLE / EXCLUDE will create implicit index "foo_f1_lower_exclusion" for table "foo"
CREATE TABLE
regression=# \d foo_f1_lower_exclusion
Index "public.foo_f1_lower_exclusion"
 Column |  Type   | Definition
--------+---------+------------
 f1     | integer | f1
 lower  | text    | lower(f2)
btree, for table "public.foo"
2009-12-23 02:35:25 +00:00
Tom Lane b7d6795445 Disallow comments on columns of relation types other than tables, views,
and composite types, which are the only relkinds for which pg_dump support
exists for dumping column comments.  There is no obvious usefulness for
comments on columns of sequences or toast tables; and while comments on
index columns might have some value, it's not worth the risk of compatibility
problems due to possible changes in the algorithm for assigning names to
index columns.  Per discussion.

In consequence, remove now-dead code for copying such comments in CREATE TABLE
LIKE.
2009-12-22 23:54:17 +00:00
Peter Eisentraut de0d75ea24 In CREATE SEQUENCE dump, put MINVALUE before MAXVALUE so it reads better. 2009-12-22 23:27:41 +00:00
Robert Haas c7e4be59ae More cleanups for the recent large object permissions patch.
Rewrite or adjust various comments for clarity.  Remove one bogus comment that
doesn't reflect what the code actually does.  Improve the description of the
lo_compat_privileges option.
2009-12-21 01:34:11 +00:00
Tom Lane cb05f5388d There is no good reason for the CREATE TABLE LIKE INCLUDING COMMENTS code to
have hard-wired knowledge of the rules for naming index columns.  It can
just look at the actual names in the source index, instead.  Do some minor
formatting cleanup too.
2009-12-20 18:28:14 +00:00
Peter Eisentraut baab7a0427 Translation updates 2009-12-19 20:23:26 +00:00
Itagaki Takahiro ce1489fa94 Add a whitespace to fix the query to dump large objects.
PL/pgSQL-by-default patch broke the code for 8.3 <= server_version < 8.5.
2009-12-19 04:13:30 +00:00
Tom Lane 4fca795de4 Bump catversion to reflect the fact that HS patch changed pg_proc
contents, and PG_CONTROL_VERSION to reflect the fact that it changed
pg_control contents.  (I see we did at least remember to change
XLOG_PAGE_MAGIC for the WAL contents changes.)
2009-12-19 04:08:32 +00:00
Tom Lane d37cda2c0c plpgsql patch broke the MSVC build, too. 2009-12-19 02:44:06 +00:00
Alvaro Herrera 1b612dfeb6 Update ecpg regression tests to plpgsql installed by default. 2009-12-19 02:38:51 +00:00
Simon Riggs efc16ea520 Allow read only connections during recovery, known as Hot Standby.
Enabled by recovery_connections = on (default) and forcing archive recovery using a recovery.conf. Recovery processing now emulates the original transactions as they are replayed, providing full locking and MVCC behaviour for read only queries. Recovery must enter consistent state before connections are allowed, so there is a delay, typically short, before connections succeed. Replay of recovering transactions can conflict and in some cases deadlock with queries during recovery; these result in query cancellation after max_standby_delay seconds have expired. Infrastructure changes have minor effects on normal running, though introduce four new types of WAL record.

New test mode "make standbycheck" allows regression tests of static command behaviour on a standby server while in recovery. Typical and extreme dynamic behaviours have been checked via code inspection and manual testing. Few port specific behaviours have been utilised, though primary testing has been on Linux only so far.

This commit is the basic patch. Additional changes will follow in this release to enhance some aspects of behaviour, notably improved handling of conflicts, deadlock detection and query cancellation. Changes to VACUUM FULL are also required.

Simon Riggs, with significant and lengthy review by Heikki Linnakangas, including streamlined redesign of snapshot creation and two-phase commit.

Important contributions from Florian Pflug, Mark Kirkwood, Merlin Moncure, Greg Stark, Gianni Ciolli, Gabriele Bartolini, Hannu Krosing, Robert Haas, Tatsuo Ishii, Hiroyuki Yamada plus support and feedback from many other community members.
2009-12-19 01:32:45 +00:00
Bruce Momjian 78a09145e0 binary migration: pg_migrator
Add comments about places where system oids have to be preserved for
binary migration.
2009-12-19 00:47:57 +00:00
Bruce Momjian 96c102fe27 Install server-side language PL/pgSQL by default. 2009-12-18 21:28:42 +00:00
Tom Lane be3a24de19 Force the TZ environment variable to be set during initdb. This is to
short-circuit the rather expensive identify_system_timezone() procedure,
which we have no real need for during initdb since nothing done here depends
on the timezone setting.  Since we launch quite a few standalone backends
during the initdb sequence, this adds up to a significant savings, and seems
worth doing to save developer time even though it will hardly matter to end
users.  Per my report today on pgsql-hackers.
2009-12-18 18:45:50 +00:00
Michael Meskes 36d192ad7d Reverting accidently commited changes. 2009-12-17 07:28:58 +00:00
Peter Eisentraut d6de43099a Don't unblock SIGQUIT in the SIGQUIT handler
This was possibly linked to a deadlock-like situation in glibc syslog code
invoked by the ereport call in quickdie().  In any case, a signal handler
should not unblock its own signal unless there is a specific reason to.
2009-12-16 23:05:00 +00:00
Peter Eisentraut b63b967a7e If there is no sigdelset(), define it as a macro.
This removes some duplicate code that recreated the identical workaround
when the newer signal API is missing.
2009-12-16 22:55:34 +00:00
Tom Lane 52fc0075ab Avoid a premature coercion failure in transformSetOperationTree() when
presented with an UNKNOWN-type Var, which can happen in cases where an
unknown literal appeared in a subquery.  While many such cases will fail
later on anyway in the planner, there are some cases where the planner is
able to flatten the query and replace the Var by the constant before it has
to coerce the union column to the final type.  I had added this check in 8.4
to provide earlier/better error detection, but it causes a regression for
some cases that worked OK before.  Fix by not making the check if the input
node is UNKNOWN type and not a Const or Param.  If it isn't going to work,
it will fail anyway at plan time, with the only real loss being inability to
provide an error cursor.  Per gripe from Britt Piehler.

In passing, rename a couple of variables to remove confusion from an
inner scope masking the same variable names in an outer scope.
2009-12-16 22:24:13 +00:00
Robert Haas ff499613d2 Several fixes for EXPLAIN (FORMAT YAML), plus one for EXPLAIN (FORMAT JSON).
ExplainSeparatePlans() was busted for both JSON and YAML output - the present
code is a holdover from the original version of my machine-readable explain
patch, which didn't have the grouping_stack machinery.  Also, fix an odd
distribution of labor between ExplainBeginGroup() and ExplainYAMLLineStarting()
when marking lists with "- ", with each providing one character.  This broke
the output format for multi-query statements.  Also, fix ExplainDummyGroup()
for the YAML output format.

Along the way, make the YAML format use escape_yaml() in situations where the
JSON format uses escape_json().  Right now, it doesn't matter because all the
values are known not to need escaping, but it seems safer this way.  Finally,
I added some comments to better explain what the YAML output format is doing.

Greg Sabino Mullane reported the issues with multi-query statements.
Analysis and remaining cleanups by me.
2009-12-16 22:16:16 +00:00
Michael Meskes d19669e5f9 Fixed auto-prepare to not try preparing statements that are not preparable. Bug
found and solved by Boszormenyi Zoltan <zb@cybertec.at>, some small adjustments
by me.
2009-12-16 10:15:07 +00:00
Peter Eisentraut dd4cd55c15 Python 3 support in PL/Python
Behaves more or less unchanged compared to Python 2, but the new language
variant is called plpython3u.  Documentation describing the naming scheme
is included.
2009-12-15 22:59:55 +00:00
Tom Lane 21d11e7ee2 Avoid unnecessary copying of source string when generating a cloned TParser.
For long source strings the copying results in O(N^2) behavior, and the
multiplier can be significant if wide-char conversion is involved.

Andres Freund, reviewed by Kevin Grittner.
2009-12-15 20:37:17 +00:00
Tom Lane a5495cd841 Add a hook to let loadable modules get control at ProcessUtility execution,
and use it to extend contrib/pg_stat_statements to track utility commands.

Itagaki Takahiro, reviewed by Euler Taveira de Oliveira.
2009-12-15 20:04:49 +00:00
Tom Lane 34d26872ed Support ORDER BY within aggregate function calls, at long last providing a
non-kluge method for controlling the order in which values are fed to an
aggregate function.  At the same time eliminate the old implementation
restriction that DISTINCT was only supported for single-argument aggregates.

Possibly release-notable behavioral change: formerly, agg(DISTINCT x)
dropped null values of x unconditionally.  Now, it does so only if the
agg transition function is strict; otherwise nulls are treated as DISTINCT
normally would, ie, you get one copy.

Andrew Gierth, reviewed by Hitoshi Harada
2009-12-15 17:57:48 +00:00
Robert Haas cddca5ec13 Add an EXPLAIN (BUFFERS) option to show buffer-usage statistics.
This patch also removes buffer-usage statistics from the track_counts
output, since this (or the global server statistics) is deemed to be a better
interface to this information.

Itagaki Takahiro, reviewed by Euler Taveira de Oliveira.
2009-12-15 04:57:48 +00:00
Itagaki Takahiro 6f1bf75d50 Fix pg_ctl initdb without options.
Passing NULL string to snprintf is avoided.
2009-12-15 00:17:50 +00:00
Tom Lane a620d5005d Fix a bug introduced when set-returning SQL functions were made inline-able:
we have to cope with the possibility that the declared result rowtype contains
dropped columns.  This fails in 8.4, as per bug #5240.

While at it, be more paranoid about inserting binary coercions when inlining.
The pre-8.4 code did not really need to worry about that because it could not
inline at all in any case where an added coercion could change the behavior
of the function's statement.  However, when inlining a SRF we allow sorting,
grouping, and set-ops such as UNION.  In these cases, modifying one of the
targetlist entries that the sort/group/setop depends on could conceivably
change the behavior of the function's statement --- so don't inline when
such a case applies.
2009-12-14 02:15:54 +00:00
Itagaki Takahiro 84f910a707 Additional fixes for large object access control.
Use pg_largeobject_metadata.oid instead of pg_largeobject.loid
to enumerate existing large objects in pg_dump, pg_restore, and
contrib modules.
2009-12-14 00:39:11 +00:00
Magnus Hagander 0182d6f646 Allow LDAP authentication to operate in search+bind mode, meaning it
does a search for the user in the directory first, and then binds with
the DN found for this user.

This allows for LDAP logins in scenarios where the DN of the user cannot
be determined simply by prefix and suffix, such as the case where different
users are located in different containers.

The old way of authentication can be significantly faster, so it's kept
as an option.

Robert Fleming and Magnus Hagander
2009-12-12 21:35:21 +00:00
Tom Lane a4e035b2f1 Fix integer-to-bit-string conversions to handle the first fractional byte
correctly when the output bit width is wider than the given integer by
something other than a multiple of 8 bits.

This has been wrong since I first wrote that code for 8.0 :-(.  Kudos to
Roman Kononov for being the first to notice, though I didn't use his
patch.  Per bug #5237.
2009-12-12 19:24:35 +00:00
Robert Haas 02490d4692 Export ExplainBeginOutput() and ExplainEndOutput() for auto_explain.
Without these functions, anyone outside of explain.c can't actually use
ExplainPrintPlan, because the ExplainState won't be initialized properly.
The user-visible result of this was a crash when using auto_explain with
the JSON output format.

Report by Euler Taveira de Oliveira.  Analysis by Tom Lane.  Patch by me.
2009-12-12 00:35:34 +00:00
Tom Lane d8e511fabb Ensure that the result tuple of an EvalPlanQual cycle gets materialized
before we zap the input tuple.  Otherwise, pass-by-reference columns of
the result slot are likely to contain just references to the input
tuple, leading to big trouble if the pfree'd space is reused.  Per
trouble report from Jaime Casanova.  This is a new bug in the recent
rewrite of EvalPlanQual, so nothing to back-patch.
2009-12-11 18:14:43 +00:00
Itagaki Takahiro f1325ce213 Add large object access control.
A new system catalog pg_largeobject_metadata manages
ownership and access privileges of large objects.

KaiGai Kohei, reviewed by Jaime Casanova.
2009-12-11 03:34:57 +00:00
Andrew Dunstan 324385d67f Add YAML to list of EXPLAIN formats. Greg Sabino Mullane, reviewed by Takahiro Itagaki. 2009-12-11 01:33:35 +00:00
Peter Eisentraut db7386187f PL/Python array support
Support arrays as parameters and return values of PL/Python functions.
2009-12-10 20:43:40 +00:00
Peter Eisentraut a37b001b80 Add init[db] option to pg_ctl
pg_ctl gets a new mode that runs initdb.  Adjust the documentation a bit to
not assume that initdb is the only way to run database cluster initialization.
But don't replace initdb as the canonical way.

Author: Zdenek Kotala <Zdenek.Kotala@Sun.COM>
2009-12-10 06:32:28 +00:00
Tom Lane 62aba76568 Prevent indirect security attacks via changing session-local state within
an allegedly immutable index function.  It was previously recognized that
we had to prevent such a function from executing SET/RESET ROLE/SESSION
AUTHORIZATION, or it could trivially obtain the privileges of the session
user.  However, since there is in general no privilege checking for changes
of session-local state, it is also possible for such a function to change
settings in a way that might subvert later operations in the same session.
Examples include changing search_path to cause an unexpected function to
be called, or replacing an existing prepared statement with another one
that will execute a function of the attacker's choosing.

The present patch secures VACUUM, ANALYZE, and CREATE INDEX/REINDEX against
these threats, which are the same places previously deemed to need protection
against the SET ROLE issue.  GUC changes are still allowed, since there are
many useful cases for that, but we prevent security problems by forcing a
rollback of any GUC change after completing the operation.  Other cases are
handled by throwing an error if any change is attempted; these include temp
table creation, closing a cursor, and creating or deleting a prepared
statement.  (In 7.4, the infrastructure to roll back GUC changes doesn't
exist, so we settle for rejecting changes of "search_path" in these contexts.)

Original report and patch by Gurjeet Singh, additional analysis by
Tom Lane.

Security: CVE-2009-4136
2009-12-09 21:57:51 +00:00
Magnus Hagander 7aeaa97de2 Add notes about updating disk and shared memory size information in the
documentation when doing new major release.
2009-12-09 17:03:30 +00:00
Magnus Hagander abf23ee86d Reject certificates with embedded NULLs in the commonName field. This stops
attacks where an attacker would put <attack>\0<propername> in the field and
trick the validation code that the certificate was for <attack>.

This is a very low risk attack since it reuqires the attacker to trick the
CA into issuing a certificate with an incorrect field, and the common
PostgreSQL deployments are with private CAs, and not external ones. Also,
default mode in 8.4 does not do any name validation, and is thus also not
vulnerable - but the higher security modes are.

Backpatch all the way. Even though versions 8.3.x and before didn't have
certificate name validation support, they still exposed this field for
the user to perform the validation in the application code, and there
is no way to detect this problem through that API.

Security: CVE-2009-4034
2009-12-09 06:37:06 +00:00
Tom Lane 65ed203910 Update time zone data files to tzdata release 2009s: DST law changes in
Antarctica, Argentina, Bangladesh, Fiji, Novokuznetsk, Pakistan, Palestine,
Samoa, Syria.  Also historical corrections for Hong Kong.
2009-12-09 00:35:32 +00:00
Tom Lane 0cb65564e5 Add exclusion constraints, which generalize the concept of uniqueness to
support any indexable commutative operator, not just equality.  Two rows
violate the exclusion constraint if "row1.col OP row2.col" is TRUE for
each of the columns in the constraint.

Jeff Davis, reviewed by Robert Haas
2009-12-07 05:22:23 +00:00
Tom Lane 8de7472b45 Don't use a duplicate OID for aclexplode(). 2009-12-06 02:55:54 +00:00
Peter Eisentraut 36f887c41c Speed up information schema privilege views
Instead of expensive cross joins to resolve the ACL, add table-returning
function aclexplode() that expands the ACL into a useful form, and join
against that.

Also, implement the role_*_grants views as a thin layer over the respective
*_privileges views instead of essentially repeating the same code twice.

fixes bug #4596

by Joachim Wieland, with cleanup by me
2009-12-05 21:43:36 +00:00
Heikki Linnakangas ab3148b712 Fix bug in temporary file management with subtransactions. A cursor opened
in a subtransaction stays open even if the subtransaction is aborted, so
any temporary files related to it must stay alive as well. With the patch,
we use ResourceOwners to track open temporary files and don't automatically
close them at subtransaction end (though in the normal case temporary files
are registered with the subtransaction resource owner and will therefore be
closed).

At end of top transaction, we still check that there's no temporary files
marked as close-at-end-of-transaction open, but that's now just a debugging
cross-check as the resource owner cleanup should've closed them already.
2009-12-03 11:03:29 +00:00
Bruce Momjian 8c1c2e38f6 thread-safety
Apply full patch to enable thread-safety by default, e.g. doc changes.
2009-12-02 14:07:26 +00:00
Tom Lane 59ed94ad0c Mark application_name as GUC_REPORT so that the value will be reported back
to the client by the server.  This might seem pretty pointless but apparently
it will help pgbouncer, and perhaps other connection poolers.  Anyway it's
practically free to do so for the normal use-case where appname is only set
in the startup packet --- we're just adding a few more bytes to the initial
ParameterStatus response packet.  Per comments from Marko Kreen.
2009-12-02 04:54:10 +00:00
Tom Lane 3dfcf8cc15 Instead of sending application_name as a SET command after the connection
is made, include it in the startup-packet options.  This makes it work more
like every other libpq connection option, in particular it now has the same
response to RESET ALL as the rest.  This also saves one network round trip
for new applications using application_name.  The cost is that if the server
is pre-8.5, it'll reject the startup packet altogether, forcing us to retry
the entire connection cycle.  But on balance we shouldn't be optimizing that
case in preference to the behavior with a new server, especially when doing
so creates visible behavioral oddities.  Per discussion.
2009-12-02 04:38:35 +00:00
Bruce Momjian b291c0fba8 psql -f -
Adjust psql -f - to behave like a normal file and honor the -1 flag.

Report from Robert Haas
2009-12-01 22:34:33 +00:00
Tom Lane 0d32342501 Teach the regular expression functions to do case-insensitive matching and
locale-dependent character classification properly when the database encoding
is UTF8.

The previous coding worked okay in single-byte encodings, or in any case for
ASCII characters, but failed entirely on multibyte characters.  The fix
assumes that the <wctype.h> functions use Unicode code points as the wchar
representation for Unicode, ie, wchar matches pg_wchar.

This is only a partial solution, since we're still stupid about non-ASCII
characters in multibyte encodings other than UTF8.  The practical effect
of that is limited, however, since those cases are generally Far Eastern
glyphs for which concepts like case-folding don't apply anyway.  Certainly
all or nearly all of the field reports of problems have been about UTF8.
A more general solution would require switching to the platform's wchar
representation for all regex operations; which is possible but would have
substantial disadvantages.  Let's try this and see if it's sufficient in
practice.
2009-12-01 21:00:24 +00:00
Bruce Momjian ef51395e24 Revert due to Tom's concerns:
Add ProcessUtility_hook() to handle all DDL to
contrib/pg_stat_statements.
2009-12-01 02:31:13 +00:00
Bruce Momjian d85cb27293 ProcessUtility_hook:
Add ProcessUtility_hook() to handle all DDL to contrib/pg_stat_statements.

Itagaki Takahiro
2009-12-01 01:08:46 +00:00
Tom Lane 908854209b Avoid core dump on empty thesaurus dictionary.
Per report from Robert Gravsjö.
2009-11-30 16:38:31 +00:00
Tom Lane 67ec9adaf1 Fix session-lifespan memory leak when a plperl function is redefined:
we have to tell Perl it can release its compiled copy of the function
text.  Noted by Alexey Klyukin.

Back-patch to 8.2 --- the problem exists further back, but this patch
won't work without modification, and it's probably not worth the trouble.
2009-11-29 21:02:16 +00:00
Peter Eisentraut 22032820f9 Error when a specified connection service is not found, instead of ignoring it 2009-11-29 20:14:53 +00:00
Tom Lane c19781ac3d Add some opr_sanity checks that the lengths of the various argument-info
arrays in a pg_proc entry match.  Seems like an easy mistake to make when
manually adjusting these values in a pg_proc.h entry.
2009-11-29 18:53:54 +00:00
Peter Eisentraut e527d5010b Remove prefix "ERROR:" from some messages, to make everything consistent 2009-11-29 18:53:44 +00:00
Tom Lane 0c61cff57a Make pg_stat_activity.application_name visible to all users, rather than
being hidden when current_query is.  Relocate it to a column position
more consistent with that behavior.  Per discussion.
2009-11-29 18:14:32 +00:00
Tom Lane 42b2907d12 Add support for anonymous code blocks (DO blocks) to PL/Perl.
Joshua Tolley, reviewed by Brendan Jurd and Tim Bunce
2009-11-29 03:02:27 +00:00
Tom Lane 8217cfbd99 Add support for an application_name parameter, which is displayed in
pg_stat_activity and recorded in log entries.

Dave Page, reviewed by Andres Freund
2009-11-28 23:38:08 +00:00
Bruce Momjian 017c47bf51 test_fsync:
Improve test descriptions displayed during test_fsync;  increase default
loops to 5k.
2009-11-28 15:04:54 +00:00
Tom Lane 1a95f12702 Eliminate a lot of list-management overhead within join_search_one_level
by adding a requirement that build_join_rel add new join RelOptInfos to the
appropriate list immediately at creation.  Per report from Robert Haas,
the list_concat_unique_ptr() calls that this change eliminates were taking
the lion's share of the runtime in larger join problems.  This doesn't do
anything to fix the fundamental combinatorial explosion in large join
problems, but it should push out the threshold of pain a bit further.

Note: because this changes the order in which joinrel lists are built,
it might result in changes in selected plans in cases where different
alternatives have exactly the same costs.  There is one example in the
regression tests.
2009-11-28 00:46:19 +00:00
Michael Meskes bb28e83636 Remove */ characters from declare cursor statements before putting them into a
comment.
2009-11-27 16:07:22 +00:00
Michael Meskes d7d5c6857b If no result is given NOTFOUND should be returned. Check for empty result
string too.
2009-11-27 13:32:17 +00:00
Michael Meskes 81a82a13b2 Added script to check if all rule re-definition in ecpg.addons are indeed used
in the build process. If not the build process will stop with an error message.
2009-11-27 10:00:40 +00:00
Michael Meskes 280744d461 Added missing files. 2009-11-26 15:55:57 +00:00
Michael Meskes c48d48d41f Forgot to add dynamic cursors to Changelog. 2009-11-26 15:40:43 +00:00
Michael Meskes fb9d62805d Synced addon rules after some renamings. 2009-11-26 15:39:26 +00:00
Michael Meskes a160c421a5 Added dynamic cursor names to ecpg. Almost the whole patch was done by
Boszormenyi Zoltan, with only a minor tweak or two from me.
2009-11-26 15:06:47 +00:00
Tom Lane da29cc8022 Simplify psql's new linestyle behavior to default to linestyle=ascii all
the time, rather than hoping we can tell whether the terminal supports
UTF8 characters.  Per discussion.
2009-11-25 20:26:31 +00:00
Michael Meskes 14cc1ed7ed Made function better readable. 2009-11-24 16:30:31 +00:00
Tom Lane 8d97c1dcfa Use diff's -w switch only on Windows, to avoid problems with inconsistent
newline representations.  Per buildfarm results and subsequent discussion.
Sync up a couple of other places that had their own policies.
2009-11-23 16:02:24 +00:00
Magnus Hagander fe4b8c42ad Add missing library to standalone libpq build on Win32.
Hiroshi Saito
2009-11-23 12:54:37 +00:00
Heikki Linnakangas cd87b6f8a5 Fix an old bug in multixact and two-phase commit. Prepared transactions can
be part of multixacts, so allocate a slot for each prepared transaction in
the "oldest member" array in multixact.c. On PREPARE TRANSACTION, transfer
the oldest member value from the current backends slot to the prepared xact
slot. Also save and recover the value from the 2pc state file.

The symptom of the bug was that after a transaction prepared, a shared lock
still held by the prepared transaction was sometimes ignored by other
transactions.

Fix back to 8.1, where both 2PC and multixact were introduced.
2009-11-23 09:58:36 +00:00
Tom Lane 1fe419279c Adjust expected-results spacing per buildfarm results. 2009-11-22 20:58:51 +00:00
Tom Lane ce3153fa93 Remove -w (--ignore-all-space) option from pg_regress's diff calls.
We have used -w for a long time as a means of reducing the reported diff
volume when one element of a result table isn't of the expected width.
However, most of the time the results just pass anyway, so this isn't as
important as it once was.  Meanwhile, the risk of missing potentially
significant deviations has gone up, particularly with psql's ability to
report error cursor positions.  So, let's switch over to space-sensitive
comparisons.  Per my proposal of yesterday.

(All the expected files that I can test here seem to be ready for this
already, but we'll see what the buildfarm thinks about others.)
2009-11-22 17:54:23 +00:00
Heikki Linnakangas 997a6a278f Remove superfluous curly brace, fixing compilation with OPTIMIZER_DEBUG.
Jan Urbanski
2009-11-22 14:54:31 +00:00
Tom Lane 1c499591f4 Ooops, forgot to check the libxml cases for psql output wrapping change. 2009-11-22 06:01:09 +00:00
Tom Lane 1753337cf5 Improve psql's tabular display of wrapped-around data by inserting markers
in the formerly-always-blank columns just to left and right of the data.
Different marking is used for a line break caused by a newline in the data
than for a straight wraparound.  A newline break is signaled by a "+" in the
right margin column in ASCII mode, or a carriage return arrow in UNICODE mode.
Wraparound is signaled by a dot in the right margin as well as the following
left margin in ASCII mode, or an ellipsis symbol in the same places in UNICODE
mode.  "\pset linestyle old-ascii" is added to make the previous behavior
available if anyone really wants it.

In passing, this commit also cleans up a few regression test files that
had unintended spacing differences from the current actual output.

Roger Leigh, reviewed by Gabrielle Roth and other members of PDXPUG.
2009-11-22 05:20:41 +00:00
Peter Eisentraut 93d3bac564 Ignore UTF-8-encoded Unicode byte-order mark at the beginning of a file if
the client encoding is UTF-8.

a limited version of a patch proposed by Itagaki Takahiro
2009-11-21 23:59:12 +00:00
Tom Lane e6c63bf660 Refactor ecpg grammar so that it uses the core grammar's unreserved_keyword
list, minus a few specific words that have to be treated specially.  This
replaces a hard-wired list of keywords that would have needed manual
maintenance, and was not getting it.  The 8.4 coding was already missing
these words, causing ecpg to incorrectly treat them as reserved words:
CALLED, CATALOG, DEFINER, ENUM, FOLLOWING, INVOKER, OPTIONS, PARTITION,
PRECEDING, RANGE, SECURITY, SERVER, UNBOUNDED, WRAPPER.  In HEAD we were
additionally missing COMMENTS, FUNCTIONS, SEQUENCES, TABLES.
Per gripe from Bosco Rama.
2009-11-21 05:44:05 +00:00
Tom Lane 7fc0f06221 Add a WHEN clause to CREATE TRIGGER, allowing a boolean expression to be
checked to determine whether the trigger should be fired.

For BEFORE triggers this is mostly a matter of spec compliance; but for AFTER
triggers it can provide a noticeable performance improvement, since queuing of
a deferred trigger event and re-fetching of the row(s) at end of statement can
be short-circuited if the trigger does not need to be fired.

Takahiro Itagaki, reviewed by KaiGai Kohei.
2009-11-20 20:38:12 +00:00
Peter Eisentraut 201a45c4fa Typo: dump -> restore
fixed in 8.4 and 8.5

Author: Guillaume Lelarge <guillaume@lelarge.info>
2009-11-19 22:05:48 +00:00
Tom Lane b1d55dca91 Fix memory leak in syslogger: logfile_rotate() would leak a copy of the
output filename if CSV logging was enabled and only one of the two possible
output files got rotated during a particular call (which would, in fact,
typically be the case during a size-based rotation).  This would amount to
about MAXPGPATH (1KB) per rotation, and it's been there since the CSV
code was put in, so it's surprising that nobody noticed it before.
Per bug #5196 from Thomas Poindessous.
2009-11-19 02:45:33 +00:00
Tom Lane c742b795dd Add a hook to CREATE/ALTER ROLE to allow an external module to check the
strength of database passwords, and create a sample implementation of
such a hook as a new contrib module "passwordcheck".

Laurenz Albe, reviewed by Takahiro Itagaki
2009-11-18 21:57:56 +00:00
Tom Lane 5e66a51c2e Provide a parenthesized-options syntax for VACUUM, analogous to that recently
adopted for EXPLAIN.  This will allow additional options to be implemented
in future without having to make them fully-reserved keywords.  The old syntax
remains available for existing options, however.

Itagaki Takahiro
2009-11-16 21:32:07 +00:00
Tom Lane 49ed392cd8 While doing the final setrefs.c pass over a plan tree, try to match up
non-Var sort/group expressions using ressortgroupref labels instead of
depending entirely on equal()-ity of the upper node's tlist expressions
to the lower node's.  This avoids emitting the wrong outputs in cases
where there are textually identical volatile sort/group expressions,
as for example
	select distinct random(),random() from generate_series(1,10);
Per report from Andrew Gierth.

Backpatch to 8.4.  Arguably this is wrong all the way back, but the only known
case where there's an observable problem is when using hash aggregation to
implement DISTINCT, which is new as of 8.4.  So for the moment I'll refrain
from backpatching further.
2009-11-16 18:04:40 +00:00
Peter Eisentraut 66363e8d6d Make text search parser accept underscores in XML attributes (bug #5075) 2009-11-15 13:57:01 +00:00
Tom Lane caf9c830d9 Improve planning of Materialize nodes inserted atop the inner input of a
mergejoin to shield it from doing mark/restore and refetches.  Put an explicit
flag in MergePath so we can centralize the logic that knows about this,
and add costing logic that considers using Materialize even when it's not
forced by the previously-existing considerations.  This is in response to
a discussion back in August that suggested that materializing an inner
indexscan can be helpful when the refetch percentage is high enough.
2009-11-15 02:45:35 +00:00
Magnus Hagander da8d684d39 Add inheritable ACE when creating a restricted token for execution on
Win32.

Also refactor the code around it to be more clear.

Jesse Morris
2009-11-14 15:39:36 +00:00
Tom Lane ef679ff6b7 Clean up a couple of bizarre code formatting choices in recent CREATE LIKE patch. 2009-11-13 23:49:23 +00:00
Tom Lane 82121aff12 Avoid assuming that enum CreateStmtLikeOption is unsigned. Zdenek Kotala 2009-11-13 23:44:19 +00:00
Tom Lane 6317609986 Add control knobs for plpgsql's variable resolution behavior, and make the
default be "throw error on conflict", as per discussions.  The GUC variable
is plpgsql.variable_conflict, with values "error", "use_variable",
"use_column".  The behavior can also be specified per-function by inserting
one of
	#variable_conflict error
	#variable_conflict use_variable
	#variable_conflict use_column
at the start of the function body.

The 8.5 release notes will need to mention using "use_variable" to retain
backward-compatible behavior, although we should encourage people to migrate
to the much less mistake-prone "error" setting.

Update the plpgsql documentation to match this and other recent changes.
2009-11-13 22:43:42 +00:00
Heikki Linnakangas 01038d4ad7 A better fix for the "ARRAY[...]::domain" problem. The previous patch worked,
but the transformed ArrayExpr claimed to have a return type of "domain",
even though the domain constraint was only checked by the enclosing
CoerceToDomain node. With this fix, the ArrayExpr is correctly labeled with
the base type of the domain. Per gripe by Tom Lane.
2009-11-13 19:48:20 +00:00
Heikki Linnakangas 942702a496 When you do "ARRAY[...]::domain", where domain is a domain over an array type,
we need to check domain constraints. We used to do it correctly, but 8.4
introduced a separate code path for the "ARRAY[]::arraytype" case to infer
the type of an empty ARRAY construct from the cast target, and forgot to take
domains into account.

Per report from Florian G. Pflug.
2009-11-13 16:09:10 +00:00
Teodor Sigaev 5e75f6790c Fix multicolumn GIN's wrong results with fastupdate enabled.
User-defined consistent functions believes the check array
contains at least one true element which was not a true for
scanning pending list.

Per report from Yury Don <yura@vpcit.ru>
2009-11-13 11:17:04 +00:00
Tom Lane 0894c6b838 The recent patch to log changes in postgresql.conf settings dumped core
if the initial value of a string variable was NULL, which is entirely
possible.  Noted while experimenting with custom_variable_classes.
2009-11-12 18:20:23 +00:00
Tom Lane 8539a0e00a Check for C/POSIX before assuming that nl_langinfo or win32_langinfo
will work.  Per buildfarm results.
2009-11-12 03:37:15 +00:00
Tom Lane 8f8a5df694 Make initdb behave sanely when the selected locale has codeset "US-ASCII".
Per discussion, this should result in defaulting to SQL_ASCII encoding.
The original coding could not support that because it conflated selection
of SQL_ASCII encoding with not being able to determine the encoding.
Adjust pg_get_encoding_from_locale()'s API to distinguish these cases,
and fix callers appropriately.  Only initdb actually changes behavior,
since the other callers were perfectly content to consider these cases
equivalent.

Per bug #5178 from Boh Yap.  Not going to bother back-patching, since
no one has complained before and there's an easy workaround (namely,
specify the encoding you want).
2009-11-12 02:46:16 +00:00
Tom Lane 19d802767d Remove pg_parse_string_token() --- not needed anymore. 2009-11-12 01:13:12 +00:00
Tom Lane 2dee828cac Remove plpgsql's separate lexer (finally!), in favor of using the core lexer
directly.  This was a lot of trouble, but should be worth it in terms of
not having to keep the plpgsql lexer in step with core anymore.  In addition
the handling of keywords is significantly better-structured, allowing us to
de-reserve a number of words that plpgsql formerly treated as reserved.
2009-11-12 00:13:00 +00:00
Peter Eisentraut 60cd1f1829 In psql \du, separate the role attributes by comma instead of newline,
for an arguably more pleasant display.
2009-11-11 21:07:41 +00:00
Alvaro Herrera e9984c47e9 Change "name" nonterminal in cursor-related productions to cursor_name.
This is a preparatory patch for allowing a dynamic cursor name be used in the
ECPG grammar.

Author: Zoltan Boszormenyi
2009-11-11 20:31:26 +00:00
Alvaro Herrera 2ea179f361 Support optional FROM/IN in FETCH and MOVE
The main motivation for this is that it's required for Informix compatibility
in ECPG.

This patch makes the ECPG and core grammars a bit closer to one another for
these productions.

Author: Zoltan Boszormenyi
2009-11-11 19:25:42 +00:00
Tom Lane 90bfe99963 Do not build psql's flex module on its own, but instead include it in
mainloop.c.  This ensures that postgres_fe.h is read before including
any system headers, which is necessary to avoid problems on some platforms
where we make nondefault selections of feature macros for stdio.h or
other headers.  We have had this policy for flex modules in the backend
for many years, but for some reason it was not applied to psql.
Per trouble report from Alexandra Roy and diagnosis by Albe Laurenz.
2009-11-10 23:12:13 +00:00
Tom Lane 21e3edd6ca Revert the temporary patch to work around Snow Leopard readdir() bug.
Apple has fixed that bug in 10.6.2, and we should encourage users to
update to that version rather than trusting this cosmetic patch.
As was recently noted by Stephen Tyler, this patch was only masking
the problem in the context of DROP TABLESPACE, but the failure could
occur in other places such as pg_xlog cleanup.
2009-11-10 18:53:38 +00:00
Bruce Momjian 089f4b921c interval_abs():
Add C comment about why there is no interval_abs():  it is unclear what
value to return:

    http://archives.postgresql.org/pgsql-general/2009-10/msg01031.php
    http://archives.postgresql.org/pgsql-general/2009-11/msg00041.php
2009-11-10 18:41:24 +00:00
Alvaro Herrera e7ec022266 Fix longstanding problems in VACUUM caused by untimely interruptions
In VACUUM FULL, an interrupt after the initial transaction has been recorded
as committed can cause postmaster to restart with the following error message:
PANIC: cannot abort transaction NNNN, it was already committed
This problem has been reported many times.

In lazy VACUUM, an interrupt after the table has been truncated by
lazy_truncate_heap causes other backends' relcache to still point to the
removed pages; this can cause future INSERT and UPDATE queries to error out
with the following error message:
could not read block XX of relation 1663/NNN/MMMM: read only 0 of 8192 bytes
The window to this race condition is extremely narrow, but it has been seen in
the wild involving a cancelled autovacuum process.

The solution for both problems is to inhibit interrupts in both operations
until after the respective transactions have been committed.  It's not a
complete solution, because the transaction could theoretically be aborted by
some other error, but at least fixes the most common causes of both problems.
2009-11-10 18:00:06 +00:00
Tom Lane 73a2f6c653 More incremental refactoring in plpgsql: get rid of gram.y dependencies on
yytext.  This is a necessary change if we're going to have a lexer interface
layer that does lookahead, since yytext won't necessarily be in step with
what the grammar thinks is the current token.  yylval and yylloc should
be the only side-variables that we need to manage when doing lookahead.
2009-11-10 02:13:13 +00:00
Tom Lane 10bcfa189b Re-refactor the core scanner's API, in order to get out from under the problem
of different parsers having different YYSTYPE unions that they want to use
with it.  I defined a new union core_YYSTYPE that is just the (very short)
list of semantic values returned by the core scanner.  I had originally
worried that this would require an extra interface layer, but actually we can
have parser.c's base_yylex (formerly filtered_base_yylex) take care of that at
no extra cost.  Names associated with the core scanner are now "core_yy_foo",
with "base_yy_foo" being used in the core Bison parser and the parser.c
interface layer.

This solves the last serious stumbling block to eliminating plpgsql's separate
lexer.  One restriction that will still be present is that plpgsql and the
core will have to agree on the token numbers assigned to tokens that can be
returned by the core lexer.  Since Bison doesn't seem willing to accept
external assignments of those numbers, we'll have to live with decreeing that
core and plpgsql grammars declare these tokens first and in the same order.
2009-11-09 18:38:48 +00:00
Tom Lane 2ace38d226 Fix WHERE CURRENT OF to work as designed within plpgsql. The argument
can be the name of a plpgsql cursor variable, which formerly was converted
to $N before the core parser saw it, but that's no longer the case.
Deal with plain name references to plpgsql variables, and add a regression
test case that exposes the failure.
2009-11-09 02:36:59 +00:00
Tom Lane 39bd3fd1db Modernize plpgsql's handling of parse locations, making it look a lot more
like the core parser's code.  In particular, track locations at the character
rather than line level during parsing, allowing many more parse-time error
conditions to be reported with precise error pointers rather than just
"near line N".

Also, exploit the fact that we no longer need to substitute $N for variable
references by making extracted SQL queries and expressions be exact copies
of subranges of the function text, rather than having random whitespace
changes within them.  This makes it possible to directly map parse error
positions from the core parser onto positions in the function text, which
lets us report them without the previous kluge of showing the intermediate
internal-query form.  (Later it might be good to do that for core
parse-analysis errors too, but this patch is just touching plpgsql's
lexer/parser, not what happens at runtime.)

In passing, make plpgsql's lexer use palloc not malloc.

These changes make plpgsql's parse-time error reports noticeably nicer
(as illustrated by the regression test changes), and will also simplify
the planned removal of plpgsql's separate lexer by reducing the impedance
mismatch between what it does and what the core lexer does.
2009-11-09 00:26:55 +00:00
Tom Lane fb60af4127 Remove ancient text file containing plpgsql installation instructions.
This was long ago superseded by the standard build process and main
SGML documentation.
2009-11-07 17:21:34 +00:00
Tom Lane f2b7692e75 Rearrange plpgsql parsing to simplify and speed it up a bit.
* Pull the responsibility for %TYPE and %ROWTYPE out of the scanner,
letting read_datatype manage it instead.

* Avoid unnecessary scanner-driven lookups of plpgsql variables in
places where it's not needed, which is actually most of the time;
we do not need it in DECLARE sections nor in text that is a SQL
query or expression.

* Rationalize the set of token types returned by the scanner:
distinguishing T_SCALAR, T_RECORD, T_ROW seems to complicate the grammar
in more places than it simplifies it, so merge these into one
token type T_DATUM; but split T_ERROR into T_DBLWORD and T_TRIPWORD
for clarity and simplicity of later processing.

Some of this will need to be revisited again when we try to make
plpgsql use the core scanner, but this patch gets some of the bigger
stumbling blocks out of the way.
2009-11-07 00:52:26 +00:00
Andrew Dunstan b79f49c780 Keep track of language's trusted flag in InlineCodeBlock. Needed to support DO blocks for languages that have both trusted and untrusted variants. 2009-11-06 21:57:57 +00:00
Tom Lane 0772f1e53d Change plpgsql from using textual substitution to insert variable references
into SQL expressions, to using the newly added parser callback hooks.

This allows us to do the substitutions in a more semantically-aware way:
a variable reference will only be recognized where it can validly go,
ie, a place where a column value or parameter would be legal, instead of
the former behavior that would replace any textual match including
table names and column aliases (leading to syntax errors later on).
A release-note-worthy fine point is that plpgsql variable names that match
fully-reserved words will now need to be quoted.

This commit preserves the former behavior that variable references take
precedence over any possible match to a column name.  The infrastructure
is in place to support the reverse precedence or throwing an error on
ambiguity, but those behaviors aren't accessible yet.

Most of the code changes here are associated with making the namespace
data structure persist so that it can be consulted at runtime, instead
of throwing it away at the end of initial function parsing.

The plpgsql scanner is still doing name lookups, but that behavior is
now irrelevant for SQL expressions.  A future commit will deal with
removing unnecessary lookups.
2009-11-06 18:37:55 +00:00
Tom Lane 593f4b854a Don't treat NEW and OLD as reserved words anymore. For the purposes of rules
it works just as well to have them be ordinary identifiers, and this gets rid
of a number of ugly special cases.  Plus we aren't interfering with non-rule
usage of these names.

catversion bump because the names change internally in stored rules.
2009-11-05 23:24:27 +00:00
Peter Eisentraut 45d7e04fce reenable -> re-enable
Pointed out by Debian's lintian.
2009-11-05 20:13:06 +00:00
Tom Lane c29ae527e9 Remove plpgsql's RENAME declaration, which has bizarre and mostly nonfunctional
behavior, and is so little used that no one has been interested in fixing it.
To ensure that possible uses are covered, remove the ALIAS declaration's
arbitrary restriction that only $n identifiers can be aliased.

(We could alternatively make RENAME act just like ALIAS, but per discussion
having two different ways to do the same thing is probably more confusing than
helpful.)
2009-11-05 16:58:36 +00:00
Tom Lane 8e79277699 Allow binary-coercible cases in ri_HashCompareOp; there are some such cases
that are not handled by find_coercion_pathway, notably composite->RECORD.
Now that 8.4 supports composites as primary keys, it's worth dealing with
this case.
2009-11-05 04:38:29 +00:00
Tom Lane 6bef82b38a Rename some encoding conversion modules to keep pathnames in our source
tarballs under 100 characters.  This should avoid failures with certain
untarring tools (WinZip and Midnight Commander have been mentioned as
likely suspects).  Per my proposal of yesterday.
catversion bumped since the initial contents of pg_proc change.
2009-11-04 23:47:04 +00:00
Tom Lane 9ab6c3033e Make expression locations for LIKE and SIMILAR TO constructs uniformly point
at the first keyword of the expression, rather than drawing a rather
artificial distinction between the ESCAPE subclause and the rest.
Per gripe from Gokulakannan Somasundaram and subsequent discusssion.
2009-11-04 23:15:08 +00:00
Tom Lane 9bedd128d6 Add support for invoking parser callback hooks via SPI and in cached plans.
As proof of concept, modify plpgsql to use the hooks.  plpgsql is still
inserting $n symbols textually, but the "back end" of the parsing process now
goes through the ParamRef hook instead of using a fixed parameter-type array,
and then execution only fetches actually-referenced parameters, using a hook
added to ParamListInfo.

Although there's a lot left to be done in plpgsql, this already cures the
"if (TG_OP = 'INSERT' and NEW.foo ...)"  problem, as illustrated by the
changed regression test.
2009-11-04 22:26:08 +00:00
Heikki Linnakangas 91ce16a903 Allow rewriting ALTER TABLE to skip WAL logging.
Itagaki Takahiro, with small changes by me and Simon.
2009-11-04 12:24:23 +00:00
Peter Eisentraut a4d03bbcda Build bzip2 tarball in dist target as well 2009-11-03 21:28:10 +00:00
Peter Eisentraut 16cd34a435 Fix regression tests for psql \d view patch 2009-11-03 14:52:10 +00:00
Peter Eisentraut 2e3b16c8ba Improve PL/Python elog output
When the elog functions (plpy.info etc.) get a single argument, just print
that argument instead of printing the single-member tuple like ('foo',).
2009-11-03 11:05:03 +00:00
Peter Eisentraut 2fe1b4dd65 In psql, show view definition only with \d+, not with \d
The rationale is that view definitions tend to be long and obscure the
main information about the view.
2009-11-03 10:34:47 +00:00
Peter Eisentraut 9e41114676 Fix obscure segfault condition in PL/Python
In PLy_output(), when the elog() call in the TRY branch throws an exception
(this can happen when a statement timeout kicks in, for example), the
PyErr_SetString() call in the CATCH branch can cause a segfault, because the
Py_XDECREF(so) call before it releases memory that is still used by the sv
variable that PyErr_SetString() uses as argument, because sv points into
memory owned by so.

Backpatched back to 8.0, where this code was introduced.

I also threw in a couple of volatile declarations for variables that are used
before and after the TRY.  I don't think they caused the crash that I
observed, but they could become issues.
2009-11-03 09:35:18 +00:00
Tom Lane 7d535ebe5b Dept of second thoughts: after studying index_getnext() a bit more I realize
that it can scribble on scan->xs_ctup.t_self while following HOT chains,
so we can't rely on that to stay valid between hashgettuple() calls.
Introduce a private variable in HashScanOpaque, instead.
2009-11-01 22:30:54 +00:00
Tom Lane c4afdca4c2 Fix two serious bugs introduced into hash indexes by the 8.4 patch that made
hash indexes keep entries sorted by hash value.  First, the original plans for
concurrency assumed that insertions would happen only at the end of a page,
which is no longer true; this could cause scans to transiently fail to find
index entries in the presence of concurrent insertions.  We can compensate
by teaching scans to re-find their position after re-acquiring read locks.
Second, neither the bucket split nor the bucket compaction logic had been
fixed to preserve hashvalue ordering, so application of either of those
processes could lead to permanent corruption of an index, in the sense
that searches might fail to find entries that are present.

This patch fixes the split and compaction logic to preserve hashvalue
ordering, but it cannot do anything about pre-existing corruption.  We will
need to recommend reindexing all hash indexes in the 8.4.2 release notes.

To buy back the performance loss hereby induced in split and compaction,
fix them to use PageIndexMultiDelete instead of retail PageIndexDelete
operations.  We might later want to do something with qsort'ing the
page contents rather than doing a binary search for each insertion,
but that seemed more invasive than I cared to risk in a back-patch.

Per bug #5157 from Jeff Janes and subsequent investigation.
2009-11-01 21:25:25 +00:00
Tom Lane ef59fa0453 Ensure the previous Perl interpreter selection is restored upon exit from
plperl_call_handler, in both the normal and error-exit paths.  Per report
from Alexey Klyukin.
2009-10-31 18:11:59 +00:00
Tom Lane fb5d05805b Implement parser hooks for processing ColumnRef and ParamRef nodes, as per my
recent proposal.  As proof of concept, remove knowledge of Params from the
core parser, arranging for them to be handled entirely by parser hook
functions.  It turns out we need an additional hook for that --- I had
forgotten about the code that handles inferring a parameter's type from
context.

This is a preliminary step towards letting plpgsql handle its variables
through parser hooks.  Additional work remains to be done to expose the
facility through SPI, but I think this is all the changes needed in the core
parser.
2009-10-31 01:41:31 +00:00
Tom Lane 8442317beb Make the overflow guards in ExecChooseHashTableSize be more protective.
The original coding ensured nbuckets and nbatch didn't exceed INT_MAX,
which while not insane on its own terms did nothing to protect subsequent
code like "palloc(nbatch * sizeof(BufFile *))".  Since enormous join size
estimates might well be planner error rather than reality, it seems best
to constrain the initial sizes to be not more than work_mem/sizeof(pointer),
thus ensuring the allocated arrays don't exceed work_mem.  We will allow
nbatch to get bigger than that during subsequent ExecHashIncreaseNumBatches
calls, but we should still guard against integer overflow in those palloc
requests.  Per bug #5145 from Bernt Marius Johnsen.

Although the given test case only seems to fail back to 8.2, previous
releases have variants of this issue, so patch all supported branches.
2009-10-30 20:58:45 +00:00
Tom Lane 77c666fe42 Un-break EXPLAIN for Append plans. I messed this up a few days ago while
adding the ModifyTable node type --- I had been thinking ModifyTable should
replace Append as a special case in push_plan(), but actually both of them
have to be special-cased.
2009-10-28 18:51:56 +00:00
Tom Lane be6899f139 Fix \df to re-allow regexp special characters in the function name pattern.
This has always worked, up until somebody's thinko here:
http://archives.postgresql.org/pgsql-committers/2009-04/msg00233.php
Per bug #5143 from Piotr Wolinski.
2009-10-28 18:09:44 +00:00
Tom Lane cbcd1701f1 Fix AcquireRewriteLocks to be sure that it acquires the right lock strength
when FOR UPDATE is propagated down into a sub-select expanded from a view.
Similar bug to parser's isLockedRel issue that I fixed yesterday; likewise
seems not quite worth the effort to back-patch.
2009-10-28 17:36:50 +00:00
Tom Lane 46e3a16b05 When FOR UPDATE/SHARE is used with LIMIT, put the LockRows plan node
underneath the Limit node, not atop it.  This fixes the old problem that such
a query might unexpectedly return fewer rows than the LIMIT says, due to
LockRows discarding updated rows.

There is a related problem that LockRows might destroy the sort ordering
produced by earlier steps; but fixing that by pushing LockRows below Sort
would create serious performance problems that are unjustified in many
real-world applications, as well as potential deadlock problems from locking
many more rows than expected.  Instead, keep the present semantics of applying
FOR UPDATE after ORDER BY within a single query level; but allow the user to
specify the other way by writing FOR UPDATE in a sub-select.  To make that
work, track whether FOR UPDATE appeared explicitly in sub-selects or got
pushed down from the parent, and don't flatten a sub-select that contained an
explicit FOR UPDATE.
2009-10-28 14:55:47 +00:00
Tom Lane 44956c52c5 Fix AfterTriggerSaveEvent to use a test and elog, not just Assert, to check
that it's called within an AfterTriggerBeginQuery/AfterTriggerEndQuery pair.
The RI cascade triggers suppress that overhead on the assumption that they
are always run non-deferred, so it's possible to violate the condition if
someone mistakenly changes pg_trigger to mark such a trigger deferred.
We don't really care about supporting that, but throwing an error instead
of crashing seems desirable.  Per report from Marcelo Costa.
2009-10-27 20:14:27 +00:00
Tom Lane 61e5328208 Make FOR UPDATE/SHARE in the primary query not propagate into WITH queries;
for example in
  WITH w AS (SELECT * FROM foo) SELECT * FROM w, bar ... FOR UPDATE
the FOR UPDATE will now affect bar but not foo.  This is more useful and
consistent than the original 8.4 behavior, which tried to propagate FOR UPDATE
into the WITH query but always failed due to assorted implementation
restrictions.  Even though we are in process of removing those restrictions,
it seems correct on philosophical grounds to not let the outer query's
FOR UPDATE affect the WITH query.

In passing, fix isLockedRel which frequently got things wrong in
nested-subquery cases: "FOR UPDATE OF foo" applies to an alias foo in the
current query level, not subqueries.  This has been broken for a long time,
but it doesn't seem worth back-patching further than 8.4 because the actual
consequences are minimal.  At worst the parser would sometimes get
RowShareLock on a relation when it should be AccessShareLock or vice versa.
That would only make a difference if someone were using ExclusiveLock
concurrently, which no standard operation does, and anyway FOR UPDATE
doesn't result in visible changes so it's not clear that the someone would
notice any problem.  Between that and the fact that FOR UPDATE barely works
with subqueries at all in existing releases, I'm not excited about worrying
about it.
2009-10-27 17:11:18 +00:00
Peter Eisentraut f1c5247563 Simplify a few makefile rules since install-sh can now install multiple
files in one run.
2009-10-26 21:33:01 +00:00
Peter Eisentraut 3ceae4795b Check errors in for loop 2009-10-26 21:11:22 +00:00
Heikki Linnakangas 2078e384a3 Fix range check in date_recv that tried to limit accepted values to only
those accepted by date_in(). I confused julian day numbers and number of
days since the postgres epoch 2000-01-01 in the original patch.

I just noticed that it's still easy to get such out-of-range values into
the database using to_date or +- operators, but this patch doesn't do
anything about those functions.

Per report from James Pye.
2009-10-26 16:13:11 +00:00
Tom Lane 9f2ee8f287 Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases.  We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries.  If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row.  The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.

Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested.  To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param.  Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.

This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE.  This is needed to avoid the
duplicate-output-tuple problem.  It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 02:26:45 +00:00
Peter Eisentraut 76d8883c8e When querying a table with child tables, do not check permissions on the
child tables.  This was found to be useless and confusing in virtually all
cases, and also contrary to the SQL standard.
2009-10-23 05:24:52 +00:00
Tom Lane ab61df9e52 Remove regex_flavor GUC, so that regular expressions are always "advanced"
style by default.  Per discussion, there seems to be hardly anything that
really relies on being able to change the regex flavor, so the ability to
select it via embedded options ought to be enough for any stragglers.
Also, if we didn't remove the GUC, we'd really be morally obligated to
mark the regex functions non-immutable, which'd possibly create performance
issues.
2009-10-21 20:38:58 +00:00
Tom Lane 289e2905c8 Remove add_missing_from GUC and associated parser support for "implicit RTEs".
Per recent discussion, add_missing_from has been deprecated for long enough to
consider removing, and it's getting in the way of planned parser refactoring.
The system now always behaves as though add_missing_from were OFF.
2009-10-21 20:22:38 +00:00
Peter Eisentraut ef8df75e67 Translations update for 8.5alpha2 2009-10-20 18:23:27 +00:00
Peter Eisentraut b1e71e8f35 Update translation updating procedure. This pertains to some changes I
made to automatically exclude translations below the 80% minimum.
2009-10-20 18:22:19 +00:00
Magnus Hagander c7b5e851eb Fix typo in previous release as reported by Itagaki Takahiro, but missed
by me.
2009-10-17 05:14:52 +00:00
Magnus Hagander 748771379b Write to the Windows eventlog in UTF16, converting the message encoding
as necessary.

Itagaki Takahiro with some changes from me
2009-10-17 00:24:51 +00:00
Tom Lane 76c09dbe8d Rewrite pam_passwd_conv_proc to be more robust: avoid assuming that the
pam_message array contains exactly one PAM_PROMPT_ECHO_OFF message.
Instead, deal with however many messages there are, and don't throw error
for PAM_ERROR_MSG and PAM_TEXT_INFO messages.  This logic is borrowed from
openssh 5.2p1, which hopefully has seen more real-world PAM usage than we
have.  Per bug #5121 from Ryan Douglas, which turned out to be caused by
the conv_proc being called with zero messages.  Apparently that is normal
behavior given the combination of Linux pam_krb5 with MS Active Directory
as the domain controller.

Patch all the way back, since this code has been essentially untouched
since 7.4.  (Surprising we've not heard complaints before.)
2009-10-16 22:08:36 +00:00
Heikki Linnakangas c02350dc2c FREEZE and VERBOSE options were in wrong order in the VACUUM command that
vacuumdb produces. Per report by Thom Brown.
2009-10-16 10:38:25 +00:00
Tom Lane 198483a2e4 First committed version of plpython_unicode_0.out did not actually contain the
required \200 bytes.  Let's see if this commit works, or if CVS is messing it up.
2009-10-15 23:39:13 +00:00
Michael Meskes d9623859e8 Made ECPG more robust against applications freeing strings, based on
patch send in by Boszormenyi Zoltan <zb@cybertec.at>.
2009-10-15 10:20:15 +00:00
Alvaro Herrera 314288fc04 Avoid using trivial usernames in foreign_data regression test.
Author: Martin Pihlak
2009-10-14 23:36:43 +00:00
Tom Lane b2734a0d79 Support SQL-compliant triggers on columns, ie fire only if certain columns
are named in the UPDATE's SET list.

Note: the schema of pg_trigger has not actually changed; we've just started
to use a column that was there all along.  catversion bumped anyway so that
this commit is included in the history of potentially interesting changes
to system catalog contents.

Itagaki Takahiro
2009-10-14 22:14:25 +00:00
Heikki Linnakangas be922e8555 Rename the new MAX_AUTH_TOKEN_LENGTH #define to PG_MAX_AUTH_MAX_TOKEN_LENGTH,
to make it more obvious that it's a PostgreSQL internal limit, not something
that comes from system header files.
2009-10-14 22:09:46 +00:00
Peter Eisentraut ea2467d78b Add alternative expected file for unicode test for client encoding not UTF8 2009-10-14 21:42:58 +00:00
Heikki Linnakangas e2a41957a4 Raise the maximum authentication token (Kerberos ticket) size in GSSAPI
and SSPI athentication methods. While the old 2000 byte limit was more than
enough for Unix Kerberos implementations, tickets issued by Windows Domain
Controllers can be much larger.

Ian Turner
2009-10-14 07:27:13 +00:00
Tom Lane 42ec8ad628 Add "\pset linestyle ascii/unicode" option to psql, allowing our traditional
ASCII-art style of table output to be upgraded to use Unicode box drawing
characters if desired.  By default, psql will use the Unicode characters
whenever client_encoding is UTF8.

The patch forces linestyle=ascii in pg_regress usage, ensuring we don't
break the regression tests in Unicode locales.

Roger Leigh
2009-10-13 21:04:01 +00:00
Tom Lane b140711643 Fix ts_stat's failure on empty tsvector.
Also insert a couple of Asserts that check for stack overflow.
Bogus coding appears to be new in 8.4 --- older releases had a much
simpler algorithm here.  Per bug #5111.
2009-10-13 14:33:14 +00:00
Alvaro Herrera 201e5b282b Add new PGC_S_DATABASE_USER enum value to several places missed by my patch
last week.

Per note and patch from Jeff Davis.
2009-10-13 14:18:40 +00:00
Tom Lane 8d54c2482b Code review for LIKE INCLUDING patch --- clean up some cosmetic and not
so cosmetic stuff.
2009-10-13 00:53:08 +00:00
Tom Lane 5ec1341136 Use plurals (TABLES, FUNCTIONS, etc) in ALTER DEFAULT PRIVILEGES. We have
the keywords as a consequence of the GRANT ALL patch, so we might as well
use them and make the ALTER commands read more naturally.
2009-10-12 23:41:45 +00:00
Tom Lane 11ca04b4b7 Support GRANT/REVOKE ON ALL TABLES/SEQUENCES/FUNCTIONS IN SCHEMA.
Petr Jelinek
2009-10-12 20:39:42 +00:00
Andrew Dunstan faa1afc6c1 CREATE LIKE INCLUDING COMMENTS and STORAGE, and INCLUDING ALL shortcut. Itagaki Takahiro. 2009-10-12 19:49:24 +00:00
Tom Lane 0adaf4cb31 Move the handling of SELECT FOR UPDATE locking and rechecking out of
execMain.c and into a new plan node type LockRows.  Like the recent change
to put table updating into a ModifyTable plan node, this increases planning
flexibility by allowing the operations to occur below the top level of the
plan tree.  It's necessary in any case to restore the previous behavior of
having FOR UPDATE locking occur before ModifyTable does.

This partially refactors EvalPlanQual to allow multiple rows-under-test
to be inserted into the EPQ machinery before starting an EPQ test query.
That isn't sufficient to fix EPQ's general bogosity in the face of plans
that return multiple rows per test row, though.  Since this patch is
mostly about getting some plan node infrastructure in place and not about
fixing ten-year-old bugs, I will leave EPQ improvements for another day.

Another behavioral change that we could now think about is doing FOR UPDATE
before LIMIT, but that too seems like it should be treated as a followon
patch.
2009-10-12 18:10:51 +00:00
Tom Lane 05d249717d Improve similar_escape() in two different ways:
* Stop escaping ? and {.  As of SQL:2008, SIMILAR TO is defined to have
POSIX-compatible interpretation of ? as well as {m,n} and related constructs,
so we should allow these things through to our regex engine.

* Escape ^ and $.  It appears that our regex engine will treat ^^ at the
beginning of the string the same as ^, and similarly for $$ at the end of
the string, which meant that SIMILAR TO was effectively ignoring ^ at the
start of the pattern and $ at the end.  Since these are not supposed to be
metacharacters, this is a bug.

The second part of this is arguably a back-patchable bug fix, but I'm
hesitant to do that because it might break applications that are expecting
something like "col SIMILAR TO '^foo$'" to work like a POSIX pattern.
Seems safer to only change it at a major version boundary.

Per discussion of an example from Doug Gorley.
2009-10-10 03:50:15 +00:00
Tom Lane 8a5849b7ff Split the processing of INSERT/UPDATE/DELETE operations out of execMain.c.
They are now handled by a new plan node type called ModifyTable, which is
placed at the top of the plan tree.  In itself this change doesn't do much,
except perhaps make the handling of RETURNING lists and inherited UPDATEs a
tad less klugy.  But it is necessary preparation for the intended extension of
allowing RETURNING queries inside WITH.

Marko Tiikkaja
2009-10-10 01:43:50 +00:00
Peter Eisentraut b865d27582 Use pg_get_triggerdef in pg_dump
Add a variant of pg_get_triggerdef with a second argument "pretty" that
causes the output to be formatted in the way pg_dump used to do.  Use this
variant in pg_dump with server versions >= 8.5.

This insulates pg_dump from most future trigger feature additions, such as
the upcoming column triggers patch.

Author: Itagaki Takahiro <itagaki.takahiro@oss.ntt.co.jp>
2009-10-09 21:02:56 +00:00
Tom Lane c970292a94 Remove very ancient tuple-counting infrastructure (IncrRetrieved() and
friends).  This code has all been ifdef'd out for many years, and doesn't
seem to have any prospect of becoming any more useful in the future.
EXPLAIN ANALYZE is what people use in practice, and I think if we did want
process-wide counters we'd be more likely to put in dtrace events for that
than try to resurrect this code.  Get rid of it so as to have one less detail
to worry about while refactoring execMain.c.
2009-10-08 22:34:57 +00:00
Alvaro Herrera 474f825574 Add the new psql command \drds to the psql docs, help and tab completion.
I also thank Bernd Helmle for the documentation help on the previous settings
patch, which I forgot on the commit message.
2009-10-08 16:34:01 +00:00
Heikki Linnakangas eab94d8182 Fix off-by-one bug in bitncmp(): When comparing a number of bits divisible by
8, bitncmp() may dereference a pointer one byte out of bounds.

Chris Mikkelson (bug #5101)
2009-10-08 04:46:21 +00:00
Tom Lane 717fa274d1 Support use of function argument names to identify which actual arguments
match which function parameters.  The syntax uses AS, for example
	funcname(value AS arg1, anothervalue AS arg2)

Pavel Stehule
2009-10-08 02:39:25 +00:00
Alvaro Herrera 2eda8dfb52 Make it possibly to specify GUC params per user and per database.
Create a new catalog pg_db_role_setting where they are now stored, and better
encapsulate the code that deals with settings into its realm.  The old
datconfig and rolconfig columns are removed.

psql has gained a \drds command to display the settings.

Backwards compatibility warning: while the backwards-compatible system views
still have the config columns, they no longer completely represent the
configuration for a user or database.

Catalog version bumped.
2009-10-07 22:14:26 +00:00
Alvaro Herrera 07cefdfb7a Fix snapshot management, take two.
Partially revert the previous patch I installed and replace it with a more
general fix: any time a snapshot is pushed as Active, we need to ensure that it
will not be modified in the future.  This means that if the same snapshot is
used as CurrentSnapshot, it needs to be copied separately.  This affects
serializable transactions only, because CurrentSnapshot has already been copied
by RegisterSnapshot and so PushActiveSnapshot does not think it needs another
copy.  However, CommandCounterIncrement would modify CurrentSnapshot, whereas
ActiveSnapshots must not have their command counters incremented.

I say "partially" because the regression test I added for the previous bug
has been kept.

(This restores 8.3 behavior, because before snapmgr.c existed, any snapshot set
as Active was copied.)

Per bug report from Stuart Bishop in
6bc73d4c0910042358k3d1adff3qa36f8df75198ecea@mail.gmail.com
2009-10-07 16:27:18 +00:00
Tom Lane e0c433c4a3 Change CREATE TABLE so that column default expressions coming from different
inheritance parent tables are compared using equal(), instead of doing
strcmp() on the nodeToString representation.  The old implementation was
always a tad cheesy, and it finally fails completely as of 8.4, now that the
node tree might contain syntax location information.  equal() knows it's
supposed to ignore those fields, but strcmp() hardly can.  Per recent
report from Scott Ribe.
2009-10-06 00:55:26 +00:00
Tom Lane 249724cb01 Create an ALTER DEFAULT PRIVILEGES command, which allows users to adjust
the privileges that will be applied to subsequently-created objects.

Such adjustments are always per owning role, and can be restricted to objects
created in particular schemas too.  A notable benefit is that users can
override the traditional default privilege settings, eg, the PUBLIC EXECUTE
privilege traditionally granted by default for functions.

Petr Jelinek
2009-10-05 19:24:49 +00:00
Tom Lane 35a173ab33 Fix assorted memory leaks in pg_hba.conf parsing. Over a sufficiently
large number of SIGHUP cycles, these would have run the postmaster out
of memory.  Noted while testing memory-leak scenario in postgresql.conf
configuration-change-printing patch.
2009-10-03 20:04:39 +00:00
Tom Lane 54d60bbd07 Fix a couple of issues in recent patch to print updates to postgresql.conf
settings: avoid calling superuser() in contexts where it's not defined,
don't leak the transient copies of GetConfigOption output, and avoid the
whole exercise in postmaster child processes.

I found that actually no current caller of GetConfigOption has any use for
its internal check of GUC_SUPERUSER_ONLY.  But rather than just remove
that entirely, it seemed better to add a parameter indicating whether to
enforce the check.

Per report from Simon and subsequent testing.
2009-10-03 18:04:57 +00:00
Tom Lane 66a8417f4e Fix an oversight in an 8.3-era patch: pgstat_initstats should allow stats
to be collected for sequences.

Report and fix by Akira Kurosawa
2009-10-02 22:49:50 +00:00
Tom Lane e66d714386 Make sure that GIN fast-insert and regular code paths enforce the same
tuple size limit.  Improve the error message for index-tuple-too-large
so that it includes the actual size, the limit, and the index name.
Sync with the btree occurrences of the same error.

Back-patch to 8.4 because it appears that the out-of-sync problem
is occurring in the field.

Teodor and Tom
2009-10-02 21:14:04 +00:00
Tom Lane d691cb9141 Fix erroneous handling of shared dependencies (ie dependencies on roles)
in CREATE OR REPLACE FUNCTION.  The original code would update pg_shdepend
as if a new function was being created, even if it wasn't, with two bad
consequences: pg_shdepend might record the wrong owner for the function,
and any dependencies for roles mentioned in the function's ACL would be lost.
The fix is very easy: just don't touch pg_shdepend at all when doing a
function replacement.

Also update the CREATE FUNCTION reference page, which never explained
exactly what changes and doesn't change in a function replacement.
In passing, fix the CREATE VIEW reference page similarly; there's no
code bug there, but the docs didn't say what happens.
2009-10-02 18:13:04 +00:00
Alvaro Herrera caa4cfa369 Ensure that a cursor has an immutable snapshot throughout its lifespan.
The old coding was using a regular snapshot, referenced elsewhere, that was
subject to having its command counter updated.  Fix by creating a private copy
of the snapshot exclusively for the cursor.

Backpatch to 8.4, which is when the bug was introduced during the snapshot
management rewrite.
2009-10-02 17:57:30 +00:00
Michael Meskes fabf75cffc Applied patch by Boszormenyi Zoltan <zb@cybertec.at> to fix memory leak in decimal handling. 2009-10-01 18:03:54 +00:00
Tom Lane f3aec2c7f5 Support "samehost" and "samenet" specifications in pg_hba.conf,
by enumerating the machine's IP interfaces to look for a match.

Stef Walter
2009-10-01 01:58:58 +00:00
Tom Lane 960d7ff022 Allow MOVE FORWARD n, MOVE BACKWARD n, MOVE FORWARD ALL, MOVE BACKWARD ALL
in plpgsql.  Clean up a couple of corner cases in the MOVE/FETCH syntax.

Pavel Stehule
2009-09-29 20:05:29 +00:00
Tom Lane 25549edb26 Fix equivclass.c's not-quite-right strategy for handling X=X clauses.
The original coding correctly noted that these aren't just redundancies
(they're effectively X IS NOT NULL, assuming = is strict).  However, they
got treated that way if X happened to be in a single-member EquivalenceClass
already, which could happen if there was an ORDER BY X clause, for instance.
The simplest and most reliable solution seems to be to not try to process
such clauses through the EquivalenceClass machinery; just throw them back
for traditional processing.  The amount of work that'd be needed to be
smarter than that seems out of proportion to the benefit.

Per bug #5084 from Bernt Marius Johnsen, and analysis by Andrew Gierth.
2009-09-29 01:20:34 +00:00
Andrew Dunstan 176c3c8db9 Convert a perl array to a postgres array when returned by Set Returning Functions as well as non SRFs. Backpatch to 8.1 where these facilities were introduced. with a little help from Abhijit Menon-Sen. 2009-09-28 17:31:12 +00:00
Michael Meskes 2ad57ee276 Added some explanation about how the parser is generated, taken from an email by
Zoltan Boszormenyi <zb@cybertec.at>.
2009-09-28 08:50:57 +00:00
Tom Lane 421d7d8edb Remove no-longer-needed ExecCountSlots infrastructure. 2009-09-27 21:10:53 +00:00
Tom Lane f92e8a4b5e Replace the array-style TupleTable data structure with a simple List of
TupleTableSlot nodes.  This eliminates the need to count in advance
how many Slots will be needed, which seems more than worth the small
increase in the amount of palloc traffic during executor startup.

The ExecCountSlots infrastructure is now all dead code, but I'll remove it
in a separate commit for clarity.

Per a comment from Robert Haas.
2009-09-27 20:09:58 +00:00
Tom Lane 61be11ff08 Make libpq reject non-numeric and out-of-range port numbers with a suitable
error message, rather than blundering on and failing with something opaque.

Sam Mason
2009-09-27 03:43:10 +00:00
Tom Lane 799ac99201 Sync psql's scanner with recent changes in backend scanner's flex rules.
Marko Kreen, Tom Lane
2009-09-27 03:27:24 +00:00
Tom Lane 3686bcb9c9 Ooops, fix to Genbki.pm for ROWTYPE_OID wasn't quite right. Also, make a few
spacing tweaks so it produces exactly the same output as genbki.sh.
2009-09-27 02:14:04 +00:00
Tom Lane 12d8fae4cd Simplify the bootstrap (BKI) code by getting rid of a useless table of all
the strings seen during the bootstrap run.  There might have been some
actual point to doing that, many years ago, but as far as I can see the only
value now is to conserve a bit of memory.  Even if we cared about wasting
a megabyte or so during the initdb run, it'd be far more effective to
arrange to release memory at the end of each BKI command, instead of
intentionally hanging onto strings that might never be used again.
Not maintaining the table probably makes it faster too; but the main point
of this patch is to get rid of a couple hundred lines of unnecessary and
rather crufty code.
2009-09-27 01:32:11 +00:00
Tom Lane 23cf415a65 Hmm, seems a lot of the buildfarm is running versions of awk that
don't have gensub().  Use sub() instead, tedious though it be.
2009-09-26 23:22:48 +00:00
Tom Lane ca70c3cfda Revert my ill-considered change that made formrdesc not insert the correct
relation rowtype OID into the relcache entries it builds.  This ensures
that catcache copies of the relation tupdescs will be fully correct.
While the deficiency doesn't seem to have any effect in the current
sources, we have been bitten by not-quite-right catcache tupdescs before,
so it seems like a good idea to maintain the rule that they should be right.
2009-09-26 23:08:22 +00:00
Tom Lane 4985635230 Extend the BKI infrastructure to allow system catalogs to be given
hand-assigned rowtype OIDs, even when they are not "bootstrapped" catalogs
that have handmade type rows in pg_type.h.  Give pg_database such an OID.
Restore the availability of C macros for the rowtype OIDs of the bootstrapped
catalogs.  (These macros are now in the individual catalogs' .h files,
though, not in pg_type.h.)

This commit doesn't do anything especially useful by itself, but it's
necessary infrastructure for reverting some ill-considered changes in
relcache.c.
2009-09-26 22:42:03 +00:00
Tom Lane c2e228d44e Fix RelationCacheInitializePhase2 (Phase3, in HEAD) to cope with the
possibility of shared-inval messages causing a relcache flush while it tries
to fill in missing data in preloaded relcache entries.  There are actually
two distinct failure modes here:

1. The flush could delete the next-to-be-processed cache entry, causing
the subsequent hash_seq_search calls to go off into the weeds.  This is
the problem reported by Michael Brown, and I believe it also accounts
for bug #5074.  The simplest fix is to restart the hashtable scan after
we've read any new data from the catalogs.  It appears that pre-8.4
branches have not suffered from this failure, because by chance there were
no other catalogs sharing the same hash chains with the catalogs that
RelationCacheInitializePhase2 had work to do for.  However that's obviously
pretty fragile, and it seems possible that derivative versions with
additional system catalogs might be vulnerable, so I'm back-patching this
part of the fix anyway.

2. The flush could delete the *current* cache entry, in which case the
pointer to the newly-loaded data would end up being stored into an
already-deleted Relation struct.  As long as it was still deleted, the only
consequence would be some leaked space in CacheMemoryContext.  But it seems
possible that the Relation struct could already have been recycled, in
which case this represents a hard-to-reproduce clobber of cached data
structures, with unforeseeable consequences.  The fix here is to pin the
entry while we work on it.

In passing, also change RelationCacheInitializePhase2 to Assert that
formrdesc() set up the relation's cached TupleDesc (rd_att) with the
correct type OID and hasoids values.  This is more appropriate than
silently updating the values, because the original tupdesc might already
have been copied into the catcache.  However this part of the patch is
not in HEAD because it fails due to some questionable recent changes in
formrdesc :-(.  That will be cleaned up in a subsequent patch.
2009-09-26 18:24:49 +00:00
Peter Eisentraut d39a84a612 Prevent isolated second surrogate in U& syntax 2009-09-25 21:13:06 +00:00
Peter Eisentraut ada0116e56 Remove backup states from Unicode escapes patch 2009-09-25 20:51:37 +00:00
Peter Eisentraut c2bb0378cf Unicode escapes in E'...' strings
Author: Marko Kreen <markokr@gmail.com>
2009-09-22 23:52:53 +00:00
Tom Lane 9048b73184 Implement the DO statement to support execution of PL code without having
to create a function for it.

Procedural languages now have an additional entry point, namely a function
to execute an inline code block.  This seemed a better design than trying
to hide the transient-ness of the code from the PL.  As of this patch, only
plpgsql has an inline handler, but probably people will soon write handlers
for the other standard PLs.

In passing, remove the long-dead LANCOMPILER option of CREATE LANGUAGE.

Petr Jelinek
2009-09-22 23:43:43 +00:00
Tom Lane d5a43ffde0 Fix crash if a DROP is attempted on an internally-dependent object.
Introduced in 8.4 rewrite of dependency.c.
Per bug #5072 from Amit Khandekar.
2009-09-22 15:46:35 +00:00
Peter Eisentraut 02faeb4ac8 Surrogate pair support for U& string and identifier syntax
This is mainly to make the functionality consistent with the proposed \u
escape syntax.
2009-09-21 22:22:07 +00:00
Bruce Momjian c6bc0feb00 fsync test files
Prevent creation of 16GB files during fsync testing; only create 16MB
files;  backpatch to 8.4.X.
2009-09-21 20:20:56 +00:00