we need to be careful to reset rc to PLPGSQL_RC_OK, depending on how
the loop's logic is structured. If we continue a loop but it then
exits without executing the loop's body again, we want to return
PLPGSQL_RC_OK to our caller. Enhance the regression tests to catch
this problem. Per report from Michael Fuhr.
It never leaked memory before PG 8.0, so none of the callers are
expecting this. Cleanest fix seems to be to make it allocate the needed
memory in estate->eval_econtext, where it will be cleaned up by
the next exec_eval_cleanup. Per report from Bill Rugolsky.
*fail*, to test that plpython didn't allow untrusted operations.
When we changed plpython to plpythonu because python didn't actually have
a secure sandbox mode, someone (probably me :-() misinterpreted the tests
as checking whether Python's file I/O works. Which is a stupid thing for
us to be testing. Remove it so we don't clutter the filesystem with
random temporary files.
copying/converting the new value, which meant that it failed badly on
"var := var" if var is of pass-by-reference type. Fix this and a similar
hazard in exec_move_row(); not sure that the latter can manifest before
8.0, but patch it all the way back anyway. Per report from Dave Chapeskie.
> against rc1. It simply checks with GetDatabaseEncoding() if the current
> database is in UTF-8, and if so, sets the UTF-8 flag on the arguments
> that are passed to perl. This means that it isn't necessary to
> utf8::upgrade() every string, as perl has no way of knowing offhand
> that a string is UTF-8 -- but postgres does, because the database
> encoding is specified, so it makes sense to turn the flag on. You
> should also be able to properly manipulate UTF-8 strings now from
> plperl as opposed to plperlu, because otherwise you'd have to use
> encoding 'utf8' which was not allowed. It could also eliminate some
> unexpected bugs if you assume that perl knows the string is unicode.
It
> is enabled only for perl 5.6 and higher, so earlier versions will not
> be affected.
>
> I have been assured by crab that the patch is quite harmless and will
> not break anything. It would be great to see it in 8 final! :-)
David Kamholz
instead of just scalar variables. Add regression tests and update the
documentation. Along the way, remove some redundant error checking
code from exec_stmt_perform().
Original patch from Pavel Stehule, reworked by Neil Conway.
These contain the SQLSTATE and error message of the current exception,
respectively. They are scope-local variables that are only defined
in exception handlers (so attempting to reference them outside an
exception handler is an error). Update the regression tests and the
documentation.
Also, do some minor related cleanup: export an unpack_sql_state()
function from the backend and use it to unpack a SQLSTATE into a
string, and add a free_var() function to pl_exec.c
Original patch from Pavel Stehule, review by Neil Conway.
This allows the result of executing a SELECT to be assigned to a row
variable, record variable, or list of scalars. Docs and regression tests
updated. Per Pavel Stehule, improvements and cleanup by Neil Conway.
1. Rename spi_return_next to return_next.
2. Add a new test for return_next.
3. Update the expected output.
4. Update the documentation.
Abhijit Menon-Sen
>
> > The second issue is where plperl returns a large result set.
I have attached the following seven patches to address this problem:
1. Trivial. Replaces some errant spaces with tabs.
2. Trivial. Fixes the spelling of Jan's name, and gets rid of many
inane, useless, annoying, and often misleading comments. Here's
a sample: "plperl_init_all() - Initialize all".
(I have tried to add some useful comments here and there, and will
continue to do so now and again.)
3. Trivial. Splits up some long lines.
4. Converts SRFs in PL/Perl to use a Tuplestore and SFRM_Materialize
to return the result set, based on the PL/PgSQL model.
There are two major consequences: result sets will spill to disk when
they can no longer fit in work_mem; and "select foo_srf()" no longer
works. (I didn't lose sleep over the latter, since that form is not
valid in PL/PgSQL, and it's not documented in PL/Perl.)
5. Trivial, but important. Fixes use of "undef" instead of undef. This
would cause empty functions to fail in bizarre ways. I suspect that
there's still another (old) bug here. I'll investigate further.
6. Moves the majority of (4) out into a new plperl_return_next()
function, to make it possible to expose the functionality to
Perl; cleans up some of the code besides.
7. Add an spi_return_next function for use in Perl code.
If you want to apply the patches and try them out, 8-composite.diff is
what you should use. (Note: my patches depend upon Andrew's use-strict
and %_SHARED patches being applied.)
Here's something to try:
create or replace function foo() returns setof record as $$
$i = 0;
for ("World", "PostgreSQL", "PL/Perl") {
spi_return_next({f1=>++$i, f2=>'Hello', f3=>$_});
}
return;
$$ language plperl;
select * from foo() as (f1 integer, f2 text, f3 text);
(Many thanks to Andrews Dunstan and Supernews for their help.)
Abhijit Menon-Sen
spotted by Qingqing Zhou. The HASH_ENTER action now automatically
fails with elog(ERROR) on out-of-memory --- which incidentally lets
us eliminate duplicate error checks in quite a bunch of places. If
you really need the old return-NULL-on-out-of-memory behavior, you
can ask for HASH_ENTER_NULL. But there is now an Assert in that path
checking that you aren't hoping to get that behavior in a palloc-based
hash table.
Along the way, remove the old HASH_FIND_SAVE/HASH_REMOVE_SAVED actions,
which were not being used anywhere anymore, and were surely too ugly
and unsafe to want to see revived again.
for testing PLs and contrib_regression for testing contrib, instead of
overwriting the core system's regression database as formerly done.
Andrew Dunstan
which is neither needed by nor related to that header. Remove the bogus
inclusion and instead include the header in those C files that actually
need it. Also fix unnecessary inclusions and bad inclusion order in
tsearch2 files.
to produce when running the executor. This is consistent with the internal
executor APIs (such as ExecutorRun), which also use a long for this purpose.
It also allows FETCH_ALL to be passed -- since FETCH_ALL is defined as
LONG_MAX, this wouldn't have worked on platforms where int and long are of
different sizes. Per report from Tzahi Fadida.
only one argument. (Per recent discussion, the option to accept multiple
arguments is pretty useless for user-defined types, and would be a likely
source of security holes if it was used.) Simplify call sites of
output/send functions to not bother passing more than one argument.
indexes. Replace all heap_openr and index_openr calls by heap_open
and index_open. Remove runtime lookups of catalog OID numbers in
various places. Remove relcache's support for looking up system
catalogs by name. Bulky but mostly very boring patch ...
output parameters or VOID or a set. There seems no particular reason to
insist on a RETURN in these cases, since the function return value is
determined by other elements anyway. Per recent discussion.
OPENed on non-SELECT commands such as EXPLAIN or SHOW (anything that
returns tuples is allowed). This flexibility already existed for
bound cursors, but OPEN was artificially restricting what it would
take. Per a gripe some months back.
change saves a great deal of space in pg_proc and its primary index,
and it eliminates the former requirement that INDEX_MAX_KEYS and
FUNC_MAX_ARGS have the same value. INDEX_MAX_KEYS is still embedded
in the on-disk representation (because it affects index tuple header
size), but FUNC_MAX_ARGS is not. I believe it would now be possible
to increase FUNC_MAX_ARGS at little cost, but haven't experimented yet.
There are still a lot of vestigial references to FUNC_MAX_ARGS, which
I will clean up in a separate pass. However, getting rid of it
altogether would require changing the FunctionCallInfoData struct,
and I'm not sure I want to buy into that.
Document use of macros for pg_printf functions.
Bump major versions of all interfaces to handle movement of get_progname
from libpq to libpgport in 8.0, and probably other libpgport changes in 8.1.
and parsing work in PL/PgSQL:
- memory management is now done via palloc(). The compiled representation
of each function now has its own memory context. Therefore, the storage
consumed by a function can be reclaimed via MemoryContextDelete().
During compilation, the CurrentMemoryContext is the function's memory
context. This means that a palloc() is sufficient to allocate memory
that will have the same lifetime as the function itself. As a result,
code invoked during compilation should be careful to pfree() temporary
allocations to avoid leaking memory. Since a lot of the code in the
backend is not careful about releasing palloc'ed memory, that means
we should switch into a temporary memory context before invoking
backend functions. A temporary context appropriate for such allocations
is `compile_tmp_cxt'.
- The ability to use palloc() allows us to simply a lot of the code in
the parser. Rather than representing lists of elements via ad hoc
linked lists or arrays, we can use the List type. Rather than doing
malloc followed by memset(0), we can just use palloc0().
- We now check that the user has supplied the right number of parameters
to a RAISE statement. Supplying either too few or too many results in
an error (at runtime).
- PL/PgSQL's parser needs to accept arbitrary SQL statements. Since we
do not want to duplicate the SQL grammar in the PL/PgSQL grammar, this
means we need to be quite lax in what the PL/PgSQL grammar considers
a "SQL statement". This can lead to misleading behavior if there is a
syntax error in the function definition, since we assume a malformed
PL/PgSQL construct is a SQL statement. Furthermore, these errors were
only detected at runtime (when we tried to execute the alleged "SQL
statement" via SPI).
To rectify this, the patch changes the parser to invoke the main SQL
parser when it sees a string it believes to be a SQL expression. This
means that synctically-invalid SQL will be rejected during the
compilation of the PL/PgSQL function. This is only done when compiling
for "validation" purposes (i.e. at CREATE FUNCTION time), so it should
not impose a runtime overhead.
- Fixes for the various buffer overruns I've patched in stable branches
in the past few weeks. I've rewritten code where I thought it was
warranted (unlike the patches applied to older branches, which were
minimally invasive).
- Various other minor changes and cleanups.
- Updates to the regression tests.
initially NULL. For 8.0 we changed the main executor to have this
behavior in an UPDATE of an array column, but plpgsql's equivalent case
was overlooked. Per report from Sven Willenberger.
MemoryContextAllocZero back to MemoryContextAlloc, same as it was in 7.4.
The zeroing is unnecessary since all the meaningful fields are filled in
just below. I had made it do that out of neatnik-ism, but some testing
with an example provided by Pavel Stehule showed that the zeroing was
accounting for about 5% of the runtime in a compute-intensive plpgsql
function. That seems a bit high of a price for neatnik-ism...
advancing ActiveSnapshot when we are inside a volatile function.
Per example from Gaetano Mendola. Add a regression test to catch
similar problems in future.
several reports of users being confused when they attempt to use ELSEIF
and run into trouble due to PL/PgSQL's lax parser. The parser will be
improved for 8.1, but we can fix most of the problem by allowing ELSEIF
for now.
of an inheritance child table is binary-compatible with the rowtype of
its parent, invent an expression node type that does the conversion
correctly. Fixes the new bug exhibited by Kris Shannon as well as a
lot of old bugs that would only show up when using multiple inheritance
or after altering the parent table.
data returned from Perl. Consolidate multiple bits of code to convert
a Perl hash to a tuple, and drive the conversion off the keys present
in the hash rather than the tuple column names, so we detect error if
the hash contains keys it shouldn't. (This means keys not in the hash
will silently default to NULL, which seems ok to me.) Fix a bunch of
reference-count leaks too.
subtransactions quite right either: the ReleaseCurrentSubTransaction
call should occur inside the PG_TRY, so that the proper path is taken
if an error occurs during subtransaction commit. This assumes that
AbortSubTransaction can cope with the state left behind if
CommitSubTransaction fails partway through, but we were already
requiring that.
operations are now run as subtransactions, so that errors in them
can be reported as ordinary Perl or Tcl errors and caught by the
normal error handling convention of those languages. Also do some
minor code cleanup in pltcl.c: extract a large chunk of duplicated
code in pltcl_SPI_execute and pltcl_SPI_execute_plan into a shared
subroutine.
rather than longjmp'ing clear out of Perl and thereby leaving Perl in
a broken state. Also some minor prettification of error messages.
Still need to do something with spi_exec_query() error handling.
for the languages even when not installed in a standard directory.
pltcl may need this treatment as well, but we don't have the right path
conveniently available, so I'll leave it alone as long as there aren't
actual reports of trouble.
may expand the Perl stack, therefore we must SPAGAIN to reload the local
stack pointer after calling it. Also a couple other marginal readability
improvements.
some of the bugs exposed thereby. The remaining 'might be used uninitialized'
warnings look like live bugs, but I am not familiar enough with Perl/C hacking
to tell how to fix them.
We don't really want to start a new SPI connection, just keep using the old
one; otherwise we have memory management problems as illustrated by
John Kennedy's bug report of today. This requires a bit of a hack to
ensure the SPI stack state is properly restored, but then again what we
were doing before was a hack too, strictly speaking. Add a regression
test to cover this case.
1. Two minor cleanups:
- We don't need to call hv_exists+hv_fetch; we should just check the
return value of hv_fetch.
- newSVpv("undef",0) is the string "undef", not a real undef.
2. This should fix the bug Andrew Dunstan described in a recent -hackers
post. It replaces three bogus "eval_pv(key, 0)" calls with newSVpv,
and eliminates another redundant hv_exists+hv_fetch pair.
3. plperl_build_tuple_argument builds up a string of Perl code to create
a hash representing the tuple. This patch creates the hash directly.
4. Another minor cleanup: replace a couple of av_store()s with av_push.
5. Analogous to #3 for plperl_trigger_build_args. This patch removes the
static sv_add_tuple_value function, which does much the same as two
other utility functions defined later, and merges the functionality
into plperl_hash_from_tuple.
I have tested the patches to the best of my limited ability, but I would
appreciate it very much if someone else could review and test them too.
(Thanks to Andrew and David Fetter for their help with some testing.)
Abhijit Menon-Sen
argument, leading to label matching failures at run-time. Per report from
Patrick Fiche. Also, fix it so that an unrecognized label argument draws
a more useful error message than 'syntax error'.
-L spec rather than assuming libpython is in the standard search path
(this returns to the way 7.4 did it). But check the distutils output
to see if it looks like Python has built a shared library, and if so
link with that instead of the probably-not-shared library found in
configdir.
- replace some function signatures of the form "some_type foo()" with
"some_type foo(void)"
- replace a few instances of a literal 0 being used as a NULL pointer;
there are more instances of this in the code, but I just fixed a few
- in src/backend/utils/mb/wstrncmp.c, replace K&R style function
declarations with ANSI style, remove use of 'register' keyword
- remove an "extern" modifier that was applied to a function definition
(rather than a declaration)
to unreserved keyword, use ereport not elog, assign a separate error code
for 'could not obtain lock' so that applications will be able to detect
that case cleanly.
as per recent discussions. Invent SubTransactionIds that are managed like
CommandIds (ie, counter is reset at start of each top transaction), and
use these instead of TransactionIds to keep track of subtransaction status
in those modules that need it. This means that a subtransaction does not
need an XID unless it actually inserts/modifies rows in the database.
Accordingly, don't assign it an XID nor take a lock on the XID until it
tries to do that. This saves a lot of overhead for subtransactions that
are only used for error recovery (eg plpgsql exceptions). Also, arrange
to release a subtransaction's XID lock as soon as the subtransaction
exits, in both the commit and abort cases. This avoids holding many
unique locks after a long series of subtransactions. The price is some
additional overhead in XactLockTableWait, but that seems acceptable.
Finally, restructure the state machine in xact.c to have a more orthogonal
set of states for subtransactions.
mode see a fresh snapshot for each command in the function, rather than
using the latest interactive command's snapshot. Also, suppress fresh
snapshots as well as CommandCounterIncrement inside STABLE and IMMUTABLE
functions, instead using the snapshot taken for the most closely nested
regular query. (This behavior is only sane for read-only functions, so
the patch also enforces that such functions contain only SELECT commands.)
As per my proposal of 6-Sep-2004; I note that I floated essentially the
same proposal on 19-Jun-2002, but that discussion tailed off without any
action. Since 8.0 seems like the right place to be taking possibly
nontrivial backwards compatibility hits, let's get it done now.
executed. Previously, the DECLARE would succeed but subsequent FETCHes
would fail since the parameter values supplied to DECLARE were not
propagated to the portal created for the cursor.
In support of this, add type Oids to ParamListInfo entries, which seems
like a good idea anyway since code that extracts a value can double-check
that it got the type of value it was expecting.
Oliver Jowett, with minor editorialization by Tom Lane.
number of active subtransaction XIDs in each backend's PGPROC entry,
and use this to avoid expensive probes into pg_subtrans during
TransactionIdIsInProgress. Extend EOXactCallback API to allow add-on
modules to get control at subxact start/end. (This is deliberately
not compatible with the former API, since any uses of that API probably
need manual review anyway.) Add basic reference documentation for
SAVEPOINT and related commands. Minor other cleanups to check off some
of the open issues for subtransactions.
Alvaro Herrera and Tom Lane.
more nearly Oracle-equivalent. Allow matching by category as well as
specific error code. Document the set of available condition names
(or more accurately, synchronize it with the existing documentation). In
passing, update errcodes.sgml to include codes added during 7.5 development.
Create a shared function to convert a SPI error code into a string
(replacing near-duplicate code in several PLs), and use it anywhere
that a SPI function call error is reported.
There are still some things that need refinement; in particular I fear
that the recognized set of error condition names probably has little in
common with what Oracle recognizes. But it's a start.
possible to trap an error inside a function rather than letting it
propagate out to PostgresMain. You still have to use AbortCurrentTransaction
to clean up, but at least the error handling itself will cooperate.
* perl_useshrplib gets set to "yes" and not to "true". I assume it's set
to "true" on unix, so I left both.
* Need to translate backslashes into slashes
* The linker config coming out of perl was for MSVC and not for mingw
Magnus Hagander
currently unapplied regarding spi_internal.c, makes some additional
fixes relating to return types, and also contains the fix for
preventing the use of insecure versions of Safe.pm.
There is one remaing return case that does not appear to work, namely
return of a composite directly in a select, i.e. if foo returns some
composite type, 'select * from foo()' works but 'select foo()' doesn't.
We will either fix that or document it as a limitation.
The function plperl_func_handler is a mess - I will try to get it
cleaned up (and split up) in a subsequent patch, time permitting.
Also, reiterating previous advice - this changes slightly the API for
spi_exec_query - the returned object has either 2 or 3 members: 'status'
(string) and 'proceesed' (int,- number of rows) and, if rows are
returned, 'rows' (array of tuple hashes).
Andrew Dunstan
FOR loops are giving weird syntax errors. Restructure parsing of FOR
loops so that the integer-loop-vs-query-loop decision is driven off
the presence of '..' between IN and LOOP, rather than the presence
of a matching record/row variable name. Hopefully this will make the
behavior a bit more transparent.
plperlNG. Review and minor cleanup/improvements by Joe Conway.
Summary of new functionality:
- Shared data space and namespace. There is a new global variable %_SHARED
that functions can use to store and save data between invocations of a
function, or between different functions. Also, all trusted plperl function
now share a common Safe container (this is an optimization, also), which
they can use for storing non-lexical variables, functions, etc.
- Triggers are now supported
- Records can now be returned (as a hash reference)
- Sets of records can now be returned (as a reference to an array of hash
references).
- New function spi_exec_query() provided for performing db functions or
getting data from db.
- Optimization for counting hash keys (Abhijit Menon-Sen)
- Allow return of 'record' and 'setof record'
As a side effect, cause subscripts in INSERT targetlists to do something
more or less sensible; previously we evaluated such subscripts and then
effectively ignored them. Another side effect is that UPDATE-ing an
element or slice of an array value that is NULL now produces a non-null
result, namely an array containing just the assigned-to positions.
of a composite type to get that type's OID as their second parameter,
in place of typelem which is useless. The actual changes are mostly
centralized in getTypeInputInfo and siblings, but I had to fix a few
places that were fetching pg_type.typelem for themselves instead of
using the lsyscache.c routines. Also, I renamed all the related variables
from 'typelem' to 'typioparam' to discourage people from assuming that
they necessarily contain array element types.
into SQL expressions. At present this only works usefully for variables
of named rowtypes, not RECORD variables, since the SQL parser can't infer
anything about datatypes from a RECORD Param. Still, it's a step forward.
scalar and composite (rowtype) cases a little better. This commit is
just a code-beautification operation and shouldn't make any real
difference in behavior, but it's an important preliminary step for
trying to improve plgsql's handling of rowtypes.
In the past, we used a 'Lispy' linked list implementation: a "list" was
merely a pointer to the head node of the list. The problem with that
design is that it makes lappend() and length() linear time. This patch
fixes that problem (and others) by maintaining a count of the list
length and a pointer to the tail node along with each head node pointer.
A "list" is now a pointer to a structure containing some meta-data
about the list; the head and tail pointers in that structure refer
to ListCell structures that maintain the actual linked list of nodes.
The function names of the list API have also been changed to, I hope,
be more logically consistent. By default, the old function names are
still available; they will be disabled-by-default once the rest of
the tree has been updated to use the new API names.
rather than allowing them only in a few special cases as before. In
particular you can now pass a ROW() construct to a function that accepts
a rowtype parameter. Internal generation of RowExprs fixes a number of
corner cases that used to not work very well, such as referencing the
whole-row result of a JOIN or subquery. This represents a further step in
the work I started a month or so back to make rowtype values into
first-class citizens.
conversion of basic ASCII letters. Remove all uses of strcasecmp and
strncasecmp in favor of new functions pg_strcasecmp and pg_strncasecmp;
remove most but not all direct uses of toupper and tolower in favor of
pg_toupper and pg_tolower. These functions use the same notions of
case folding already developed for identifier case conversion. I left
the straight locale-based folding in place for situations where we are
just manipulating user data and not trying to match it to built-in
strings --- for example, the SQL upper() function is still locale
dependent. Perhaps this will prove not to be what's wanted, but at
the moment we can initdb and pass regression tests in Turkish locale.
results with tuples as ordinary varlena Datums. This commit does not
in itself do much for us, except eliminate the horrid memory leak
associated with evaluation of whole-row variables. However, it lays the
groundwork for allowing composite types as table columns, and perhaps
some other useful features as well. Per my proposal of a few days ago.
a whole row or record variable into a SQL function. Eventually this case
should be made to actually work, but for now this is better than what it
did before.
is done at creation time for plpgsql functions. Improve createlang and
droplang to support adding/dropping validators for PLs. Initial steps
towards producing a syntax error position from plpgsql syntax errors
(this part is a work in progress, and will change depending on outcome
of current discussions).
so that the 'val' is computed only once, per recent discussion. The
speedup is not much when 'val' is just a simple variable, but could be
significant for larger expressions. More importantly this avoids issues
with multiple evaluations of a volatile 'val', and it allows the CASE
expression to be reverse-listed in its original form by ruleutils.c.
exposed thereby. AFAICT these would not lead to any worse problems than
junk emitted on the backend's stdout, but we should have the option to
catch possible worse errors in future.
problem, per previous discussion. Make some additional changes to
centralize the knowledge of just how identifier downcasing is done,
in hopes of simplifying any future tweaking in this area.
Make btree index creation and initial validation of foreign-key constraints
use maintenance_work_mem rather than work_mem as their memory limit.
Add some code to guc.c to allow these variables to be referenced by their
old names in SHOW and SET commands, for backwards compatibility.
pointer type when it is not necessary to do so.
For future reference, casting NULL to a pointer type is only necessary
when (a) invoking a function AND either (b) the function has no prototype
OR (c) the function is a varargs function.
parameters to be declared with names. pg_proc has a column to store
names, and CREATE FUNCTION can insert data into it, but that's all as
yet. I need to do more work on the pg_dump and plpgsql portions of the
patch before committing those, but I thought I'd get the bulky changes
in before the tree drifts under me.
initdb forced due to pg_proc change.
to be less dangerous, and often faster as well. ExprState trees are
not kept across transaction boundaries; this eliminates problems with
resource leakage in failed transactions. But by keeping them in a
per-transaction EState, we can safely arrange for a single ExprState
to be shared by all the expression evaluations done in a given plpgsql
function call. (Formerly it seemed necessary to create and destroy an
ExprState for each exec_eval_simple_expr() call.) This saves time in
any scenario where a plpgsql function executes more than one expression.
Seems to be about as fast as 7.3 for simple cases, and significantly
faster for functions that do a lot of calculations.
method. Fix a number of places where shared libraries were linked without
mentioning all the libraries they depend on; the Darwin and AIX ports
are known to require this, and it doesn't seem to hurt any other supported
platforms. (Hence, remove code in pl/tcl makefile that tried to avoid
mentioning other libs if not needed.)
used as trigger on different relations. I am not convinced that Tcl
actually has to have this, but it seems a good idea to make it be
parallel to the other PLs that definitely do need it.
relation, when the same function is used as a trigger on more than
one relation. This avoids crashes due to differing rowtypes for
different relations. Per bug report from Lance Thomas, 7-Feb-03.
the trigger is attached to in the hashkey. This ensures that we will
create separate compiled trees for each table the trigger is used with,
avoiding possible datatype-mismatch problems if the tables have different
rowtypes. This is essentially the same bug recently identified in plpython
--- though plpgsql doesn't seem as prone to crash when the rowtype changes
underneath it. But failing robustly is no substitute for just working.
>>that you cannot change the value, similar to the argument variables:
>
> Perhaps you shouldn't mark it isconst; then it would actually have some
> usefulness (you could use it directly as a temporary variable to hold
> the intended result). I can't see much value in aliasing it if it's
> const, either.
OK; the only change in this version is "isconst = false;". Now you can
use $0 as a result placeholder if desired. E.g.:
create or replace function tmp(anyelement, anyelement) returns anyarray as '
declare
v_ret alias for $0;
v_el1 alias for $1;
v_el2 alias for $2;
begin
v_ret := ARRAY[v_el1, v_el2];
return v_ret;
end;
' language 'plpgsql';
create table f(f1 text, f2 text, f3 int, f4 int);
insert into f values ('a','b',1,2);
insert into f values ('z','x',3,4);
select tmp(f1,f2) from f;
select tmp(f3,f4) from f;
Joe Conway
'scalar op ALL (array)', where the operator is applied between the
lefthand scalar and each element of the array. The operator must
yield boolean; the result of the construct is the OR or AND of the
per-element results, respectively.
Original coding by Joe Conway, after an idea of Peter's. Rewritten
by Tom to keep the implementation strictly separate from subqueries.
> rexec and making it an untrusted language. Last time I looked, it didn't
> look particularly difficult. I've set aside some time next week, so stay
> tuned.
Attached is a patch that removes all of the RExec code from plpython from
the current PostgreSQL CVS. In addition, plpython needs to be changed to an
untrusted language in createlang. Please let me know if there are any
problems.
Kevin Jacobs
character in identifiers. The first change eliminates the current need
to put spaces around parameter references, as in "x<=$2". The second
change improves compatibility with Oracle and some other RDBMSes. This
was discussed and agreed to back in January, but did not get done.
yy_fatal_error() call results in elog(ERROR) not exit(). This was
already fixed in the main lexer and plpgsql, but extend same technique
to all the other dot-l files. Also, on review of the possible calls
to yy_fatal_error(), it seems safe to use elog(ERROR) not elog(FATAL).
avoids 'input buffer overflow' failure on long literals, improves
performance, gives the right answer for line position in functions
containing multiline literals, suppresses annoying compiler warnings,
and generally is so much better I wonder why we didn't do it before.
rewritten and the protocol is changed, but most elog calls are still
elog calls. Also, we need to contemplate mechanisms for controlling
all this functionality --- eg, how much stuff should appear in the
postmaster log? And what API should libpq expose for it?
expressions, ARRAY(sub-SELECT) expressions, some array functions.
Polymorphic functions using ANYARRAY/ANYELEMENT argument and return
types. Some regression tests in place, documentation is lacking.
Joe Conway, with some kibitzing from Tom Lane.
include output that vary depending on the python build one is
running. Basically, the order of keys in a dictionary is
non-deterministic, and that part of the test fails for me regularly.
I rewrote the test to work around this problem, and include a patch
file with that change and the change to the expected otuput as well.
Mike Meyer
(materialization into a tuple store) discussed on pgsql-hackers earlier.
I've updated the documentation and the regression tests.
Notes on the implementation:
- I needed to change the tuple store API slightly -- it assumes that it
won't be used to hold data across transaction boundaries, so the temp
files that it uses for on-disk storage are automatically reclaimed at
end-of-transaction. I added a flag to tuplestore_begin_heap() to control
this behavior. Is changing the tuple store API in this fashion OK?
- in order to store executor results in a tuple store, I added a new
CommandDest. This works well for the most part, with one exception: the
current DestFunction API doesn't provide enough information to allow the
Executor to store results into an arbitrary tuple store (where the
particular tuple store to use is chosen by the call site of
ExecutorRun). To workaround this, I've temporarily hacked up a solution
that works, but is not ideal: since the receiveTuple DestFunction is
passed the portal name, we can use that to lookup the Portal data
structure for the cursor and then use that to get at the tuple store the
Portal is using. This unnecessarily ties the Portal code with the
tupleReceiver code, but it works...
The proper fix for this is probably to change the DestFunction API --
Tom suggested passing the full QueryDesc to the receiveTuple function.
In that case, callers of ExecutorRun could "subclass" QueryDesc to add
any additional fields that their particular CommandDest needed to get
access to. This approach would work, but I'd like to think about it for
a little bit longer before deciding which route to go. In the mean time,
the code works fine, so I don't think a fix is urgent.
- (semi-related) I added a NO SCROLL keyword to DECLARE CURSOR, and
adjusted the behavior of SCROLL in accordance with the discussion on
-hackers.
- (unrelated) Cleaned up some SGML markup in sql.sgml, copy.sgml
Neil Conway
x[42] := whatever;
The facility is pretty primitive because it doesn't do array slicing and
it has the same semantics as array update in SQL (array must already
be non-null, etc). But it's a start.
entire contents of the subplan into the tuplestore before we can return
any tuples. Instead, the tuplestore holds what we've already read, and
we fetch additional rows from the subplan as needed. Random access to
the previously-read rows works with the tuplestore, and doesn't affect
the state of the partially-read subplan. This is a step towards fixing
the problems with cursors over complex queries --- we don't want to
stick in Materialize nodes if they'll prevent quick startup for a cursor.
startup, not in the parser; this allows ALTER DOMAIN to work correctly
with domain constraint operations stored in rules. Rod Taylor;
code review by Tom Lane.
that's selecting into a RECORD variable returns zero rows, make it
assign an all-nulls row to the RECORD; this is consistent with what
happens when the SELECT INTO target is not a RECORD. In support of
this, tweak the SPI code so that a valid tuple descriptor is returned
even when a SPI select returns no rows.
a per-query memory context created by CreateExecutorState --- and destroyed
by FreeExecutorState. This provides a final solution to the longstanding
problem of memory leaked by various ExecEndNode calls.
execution state trees, and ExecEvalExpr takes an expression state tree
not an expression plan tree. The plan tree is now read-only as far as
the executor is concerned. Next step is to begin actually exploiting
this property.
so that all executable expression nodes inherit from a common supertype
Expr. This is somewhat of an exercise in code purity rather than any
real functional advance, but getting rid of the extra Oper or Func node
formerly used in each operator or function call should provide at least
a little space and speed improvement.
initdb forced by changes in stored-rules representation.
to plan nodes, not vice-versa. All executor state nodes now inherit from
struct PlanState. Copying of plan trees has been simplified by not
storing a list of SubPlans in Plan nodes (eliminating duplicate links).
The executor still needs such a list, but it can build it during
ExecutorStart since it has to scan the plan tree anyway.
No initdb forced since no stored-on-disk structures changed, but you
will need a full recompile because of node-numbering changes.
logic, dissuade planner from thinking that 'x IS DISTINCT FROM 42' may
be optimized into 'x = 42' (!!), cause dependency on = operator to be
recorded correctly, minor other improvements.
-hackers a couple days ago.
Notes/caveats:
- added regression tests for the new functionality, all
regression tests pass on my machine
- added pg_dump support
- updated PL/PgSQL to support per-statement triggers; didn't
look at the other procedural languages.
- there's (even) more code duplication in trigger.c than there
was previously. Any suggestions on how to refactor the
ExecXXXTriggers() functions to reuse more code would be
welcome -- I took a brief look at it, but couldn't see an
easy way to do it (there are several subtly-different
versions of the code in question)
- updated the documentation. I also took the liberty of
removing a big chunk of duplicated syntax documentation in
the Programmer's Guide on triggers, and moving that
information to the CREATE TRIGGER reference page.
- I also included some spelling fixes and similar small
cleanups I noticed while making the changes. If you'd like
me to split those into a separate patch, let me know.
Neil Conway
PL/PgSQL. Previously, it had been bundled together with the assign
statement implementation, for some reason that wasn't clear to me
(they certainly don't share any code with one another). So I separated
them and made PERFORM a statement like any other. No changes in
functionality.
Along the way, I added some regression tests for PERFORM, added a
bunch more SGML tags to the PL/PgSQL docs, and removed an obsolete
comment relating to the implementation of RETURN NEXT.
Neil Conway
(usually bison output files), not as standalone files. This hack
works around flex's insistence on including <stdio.h> before we are
able to include postgres.h; postgres.h will already be read before
the compiler starts to read the flex output file. Needed for largefile
support on some platforms.
SPI_prepare: they all save the prepared plan into topCxt, and so the
procCxt copy that's actually returned by SPI_prepare ought to be freed.
Diagnosis and plpython fix by Nigel Andrews, followup for other PLs
by Tom Lane.
1) pltcl:
Add SPI_freetuptable() calls to avoid memory leaks (Me + Neil Conway)
Change sprintf()s to snprintf()s (Neil Conway)
Remove header files included elsewhere (Neil Conway)
2)plpython:
Add SPI_freetuptable() calls to avoid memory leaks
Cosemtic change to remove a compiler warning
Notes:
I have tested pltcl.c for
a) the original leak problem reported for the repeated call of spi_exec
in a TCL fragment
and
b) the subsequent report resulting from the use of spi_exec -array
in a TCL
fragment.
The plpython.c patch is exactly the same as that applied to make
revision 1.23,
the plpython_schema.sql and feature.expected sections of the patch are
also the
same as last submited, applied and subsequently reversed out. It remains
untested by me (other than via make check). However, this should be safe
provided PyString_FromString() _copies_ the given string to make a
PyObject.
Nigel J. Andrews
let's say this patch superscedes the previous one.
I have also attached a patch addressing the similar memory leak problem in
plpython. This includes a slight adjustment of the tests in the source
directory. The patch also includes a cosmetic change to remove a compiler
warning although I think the change makes the code look worse though.
BTW, by my reckoning the memory leak would occur with prepared plans and
without. If that is not the case then I've been barking up the wrong tree.
Nigel J. Andrews
with OPAQUE. CREATE LANGUAGE, CREATE TRIGGER, and CREATE TYPE will all
accept references to functions declared with OPAQUE --- but they will
issue a NOTICE, and will modify the function entries in pg_proc to have
the preferred type-safe argument or result types instead of OPAQUE.
Per recent pghackers discussions.
>
>>::sigh:: Is it me or does it look like all
>>of pl/pgsql is schema un-aware (ie, all of the declarations). -sc
>
>
> Yeah. The group of routines parse_word, parse_dblword, etc that are
> called by the lexer certainly all need work. There are some
> definitional issues to think about, too --- plpgsql presently relies on
> the number of names to give it some idea of what to look for, and those
> rules are probably all toast now. Please come up with a sketch of what
> you think the behavior should be before you start hacking code.
Attached is a diff -c format proposal to fix this. I've also attached a short
test script. Seems to work OK and passes all regression tests.
Here's a breakdown of how I understand plpgsql's "Special word rules" -- I
think it illustrates the behavior reasonably well. New functions added by this
patch are plpgsql_parse_tripwordtype and plpgsql_parse_dblwordrowtype:
Joe Conway
Eliminate the mysterious games that the Cygwin build plays with the linker
flag variables. DLLLIBS is gone, use SHLIB_LINK like everyone else.
Detect cygipc in configure, after the linker flags are set up, otherwise
configure might not work at all.
Make sure everything is covered by make clean.
Fix the build of the new conversion procedure modules.
Add new DLLIMPORT markers where required.
Finally, the compiler complains if we use an explicit
-I/usr/local/include, so don't do that. Curiously, -L/usr/local/lib is
still necessary.
should be pretty safe in practice, but it's probably better to be safe
than sorry.
I was actually looking for cases where NAMEDATALEN is assumed to be
32, but only found one. That's fixed too, as well as a few bits of
code cleanup.
Neil Conway
value '-2' is used to indicate a variable-width type whose width is
computed as strlen(datum)+1. Everything that looks at typlen is updated
except for array support, which Joe Conway is working on; at the moment
it wouldn't work to try to create an array of cstring.
with OPAQUE, as per recent pghackers discussion. I still want to do some
more work on the 'cstring' pseudo-type, but I'm going to commit the bulk
of the changes now before the tree starts shifting under me ...
FOUND is set whenever a SELECT INTO returns > 0 rows, *or* when an
INSERT, UPDATE, or DELETE affects > 0 rows. We implemented the first
part of this behavior, but not the second.
I also improved the documentation on the various situations in which
FOUND can be set (excluding inside FOR loops, which I still need to
think about), and added some regression tests for this behavior.
Neil Conway
> There's no longer a separate call to heap_storage_create in that routine
> --- the right place to make the test is now in the storage_create
> boolean parameter being passed to heap_create. A simple change, but
> it passeth patch's understanding ...
Thanks.
Attached is a patch against cvs tip as of 8:30 PM PST or so. Turned out
that even after fixing the failed hunks, there was a new spot in
bufmgr.c which needed to be fixed (related to temp relations;
RelationUpdateNumberOfBlocks). But thankfully the regression test code
caught it :-)
Joe Conway
correctly, truncate to NAMEDATALEN where needed, allow whitespace
around dots in qualified identifiers. Get rid of T_RECFIELD and
T_TGARGV token categories, which weren't accomplishing anything
except to create room for sins of omission in the grammar, ie,
places that should have allowed them and didn't. Fix a few other
bugs en passant.
whitespaces in identifers of any kind(table names,attribute
names,variables ...) in Pl/pgSQL procedural language.Explicit definition
of bug can be found in Re: [HACKERS] Bug of PL/pgSQL parser
TODO item completed:
o -Fix PL/PgSQL to handle quoted mixed-case identifiers
eutm
code review by Tom Lane. Remaining issues: functions that take or
return tuple types are likely to break if one drops (or adds!)
a column in the table defining the type. Need to think about what
to do here.
Along the way: some code review for recent COPY changes; mark system
columns attnotnull = true where appropriate, per discussion a month ago.
bitmap, if present).
Per Tom Lane's suggestion the information whether a tuple has an oid
or not is carried in the tuple descriptor. For debugging reasons
tdhasoid is of type char, not bool. There are predefined values for
WITHOID, WITHOUTOID and UNDEFOID.
This patch has been generated against a cvs snapshot from last week
and I don't expect it to apply cleanly to current sources. While I
post it here for public review, I'm working on a new version against a
current snapshot. (There's been heavy activity recently; hope to
catch up some day ...)
This is a long patch; if it is too hard to swallow, I can provide it
in smaller pieces:
Part 1: Accessor macros
Part 2: tdhasoid in TupDesc
Part 3: Regression test
Part 4: Parameter withoid to heap_addheader
Part 5: Eliminate t_oid from HeapTupleHeader
Part 2 is the most hairy part because of changes in the executor and
even in the parser; the other parts are straightforward.
Up to part 4 the patched postmaster stays binary compatible to
databases created with an unpatched version. Part 5 is small (100
lines) and finally breaks compatibility.
Manfred Koizar
HeapTupleHeaderData in setter and getter macros called
HeapTupleHeaderGetXmin, HeapTupleHeaderSetXmin etc.
It also introduces a "virtual" field xvac by defining
HeapTupleHeaderGetXvac and HeapTupleHeaderSetXvac. Xvac is used by
VACUUM, in fact it is stored in t_cmin.
Manfred Koizar
system, not Tcl-provided one.
Make sure export file, if any, is cleaned.
Tcl configuration is now read directly in configure and recorded in
Makefile.global. This eliminates some duplicate efforts and allows
for easier hand-editing of the results, if necessary.
As proof of concept, provide an alternate implementation based on POSIX
semaphores. Also push the SysV shared-memory implementation into a
separate file so that it can be replaced conveniently.
handled as special productions. This is needed to keep us honest about
user-schema type names that happen to coincide with system type names.
Per pghackers discussion 24-Apr. To avoid bloating the keyword list
too much, I removed the translations for datetime, timespan, and lztext,
all of which were slated for destruction several versions back anyway.
in schemas other than the system namespace; however, there's no search
path yet, and not all operations work yet on tables outside the system
namespace.
o Change all current CVS messages of NOTICE to WARNING. We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.
o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.
o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.
o Remove INFO from the client_min_messages options and add NOTICE.
Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.
Regression passed.
now just below FATAL in server_min_messages. Added more text to
highlight ordering difference between it and client_min_messages.
---------------------------------------------------------------------------
REALLYFATAL => PANIC
STOP => PANIC
New INFO level the prints to client by default
New LOG level the prints to server log by default
Cause VACUUM information to print only to the client
NOTICE => INFO where purely information messages are sent
DEBUG => LOG for purely server status messages
DEBUG removed, kept as backward compatible
DEBUG5, DEBUG4, DEBUG3, DEBUG2, DEBUG1 added
DebugLvl removed in favor of new DEBUG[1-5] symbols
New server_min_messages GUC parameter with values:
DEBUG[5-1], INFO, NOTICE, ERROR, LOG, FATAL, PANIC
New client_min_messages GUC parameter with values:
DEBUG[5-1], LOG, INFO, NOTICE, ERROR, FATAL, PANIC
Server startup now logged with LOG instead of DEBUG
Remove debug_level GUC parameter
elog() numbers now start at 10
Add test to print error message if older elog() values are passed to elog()
Bootstrap mode now has a -d that requires an argument, like postmaster
The attached patch enables plperl to build under Cygwin. It is
basically yet another BE_DLLLIBS patch with a perl MakeMaker twist. I
tried the patch under Red Hat 7.1 Linux too and I did not observe any
ill effects.
Jason Tishler
to give more useful error messages. Stephen Szabo's example of this
morning ('loop' used as a variable name inside a subselect) works
correctly now, and a FOR that is misinterpreted as an integer FOR will
draw 'missing .. at end of SQL expression', which is at least
marginally helpful.
Enabling this feature adds very light overhead of 1 select from pg_class on
first using of pl/tcl in backend if unknown suppport is really unused.
But pl/tcl with this support has very improved functionality.
Patch includes changes to documentation.
NOTICE added about error location (same method already used by plpgsql
executor). Add checking of pg_proc row xmin/cmin to ensure that
plpgsql functions will be recompiled after they've been modified by
CREATE OR REPLACE FUNCTION.
rather than having its own somewhat half-baked notion of what a type
declaration looks like. This is necessary now to ensure that plpgsql
will think a 'timestamp' variable has the same semantics as 'timestamp'
does in the main SQL grammar; and it should avoid divergences in future.
lookup info in the relcache for index access method support functions.
This makes a huge difference for dynamically loaded support functions,
and should save a few cycles even for built-in ones. Also tweak dfmgr.c
so that load_external_function is called only once, not twice, when
doing fmgr_info for a dynamically loaded function. All per performance
gripe from Teodor Sigaev, 5-Oct-01.
which contains stack trace.
One problem, after this patch errors will generate multiline ERROR
messages. Is it acceptable or do I need split it and generate multiple
singleline messages?
Vsevolod Lobko
> > > > > and --enable-unicode-convertion if it ought to work correctly
> > > > > with Tcl/Tk >= 8.1 (client or server side).
> > > > >
> > > > > - PL/Tcl needs to be changed to use pg_do_encoding_conversion
> > > > > if it runs on a Tcl version >= 8.1 .
> > >
> > > > I'll do pl/tcl part in the next version of patch. Using this approach we
> > > > can eliminate overhead for databases in UNICODE.
> > >
> > > Any progress on this? I'd prefer to get rid of this --enable-pltcl-utf
> > > option before release.
> >
> > Done
> >
> > Next version removes --enable-pltcl-utf switch and enables embedded
> > utf conversion of pgsql if tcl version >=8.1 and --enable-unicode-conversion
under libdir, for a cleaner separation in the installation layout
and compatibility with binary packaging standards. Point backend's
default search location there. The contrib modules are also
installed in the said location, giving them the benefit of the
default search path as well. No changes in user interface
nevertheless.
