1999-01-27 17:15:22 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
2002-10-15 01:49:20 +02:00
|
|
|
* spi_priv.h
|
1999-01-27 17:15:22 +01:00
|
|
|
* Server Programming Interface private declarations
|
|
|
|
*
|
2019-01-02 18:44:25 +01:00
|
|
|
* Portions Copyright (c) 1996-2019, PostgreSQL Global Development Group
|
2002-10-15 01:49:20 +02:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
*
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/include/executor/spi_priv.h
|
1999-01-27 17:15:22 +01:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#ifndef SPI_PRIV_H
|
|
|
|
#define SPI_PRIV_H
|
|
|
|
|
1999-07-14 03:20:30 +02:00
|
|
|
#include "executor/spi.h"
|
2017-04-01 06:17:18 +02:00
|
|
|
#include "utils/queryenvironment.h"
|
1999-01-27 17:15:22 +01:00
|
|
|
|
2002-10-15 01:49:20 +02:00
|
|
|
|
2007-03-16 00:12:07 +01:00
|
|
|
#define _SPI_PLAN_MAGIC 569278163
|
|
|
|
|
1999-01-27 17:15:22 +01:00
|
|
|
typedef struct
|
|
|
|
{
|
2005-10-01 20:43:19 +02:00
|
|
|
/* current results */
|
Widen query numbers-of-tuples-processed counters to uint64.
This patch widens SPI_processed, EState's es_processed field, PortalData's
portalPos field, FuncCallContext's call_cntr and max_calls fields,
ExecutorRun's count argument, PortalRunFetch's result, and the max number
of rows in a SPITupleTable to uint64, and deals with (I hope) all the
ensuing fallout. Some of these values were declared uint32 before, and
others "long".
I also removed PortalData's posOverflow field, since that logic seems
pretty useless given that portalPos is now always 64 bits.
The user-visible results are that command tags for SELECT etc will
correctly report tuple counts larger than 4G, as will plpgsql's GET
GET DIAGNOSTICS ... ROW_COUNT command. Queries processing more tuples
than that are still not exactly the norm, but they're becoming more
common.
Most values associated with FETCH/MOVE distances, such as PortalRun's count
argument and the count argument of most SPI functions that have one, remain
declared as "long". It's not clear whether it would be worth promoting
those to int64; but it would definitely be a large dollop of additional
API churn on top of this, and it would only help 32-bit platforms which
seem relatively less likely to see any benefit.
Andreas Scherbaum, reviewed by Christian Ullrich, additional hacking by me
2016-03-12 22:05:10 +01:00
|
|
|
uint64 processed; /* by Executor */
|
Prevent leakage of SPI tuple tables during subtransaction abort.
plpgsql often just remembers SPI-result tuple tables in local variables,
and has no mechanism for freeing them if an ereport(ERROR) causes an escape
out of the execution function whose local variable it is. In the original
coding, that wasn't a problem because the tuple table would be cleaned up
when the function's SPI context went away during transaction abort.
However, once plpgsql grew the ability to trap exceptions, repeated
trapping of errors within a function could result in significant
intra-function-call memory leakage, as illustrated in bug #8279 from
Chad Wagner.
We could fix this locally in plpgsql with a bunch of PG_TRY/PG_CATCH
coding, but that would be tedious, probably slow, and prone to bugs of
omission; moreover it would do nothing for similar risks elsewhere.
What seems like a better plan is to make SPI itself responsible for
freeing tuple tables at subtransaction abort. This patch attacks the
problem that way, keeping a list of live tuple tables within each SPI
function context. Currently, such freeing is automatic for tuple tables
made within the failed subtransaction. We might later add a SPI call to
mark a tuple table as not to be freed this way, allowing callers to opt
out; but until someone exhibits a clear use-case for such behavior, it
doesn't seem worth bothering.
A very useful side-effect of this change is that SPI_freetuptable() can
now defend itself against bad calls, such as duplicate free requests;
this should make things more robust in many places. (In particular,
this reduces the risks involved if a third-party extension contains
now-redundant SPI_freetuptable() calls in error cleanup code.)
Even though the leakage problem is of long standing, it seems imprudent
to back-patch this into stable branches, since it does represent an API
semantics change for SPI users. We'll patch this in 9.3, but live with
the leakage in older branches.
2013-07-25 22:45:43 +02:00
|
|
|
SPITupleTable *tuptable; /* tuptable currently being built */
|
2005-10-01 20:43:19 +02:00
|
|
|
|
2017-10-07 01:18:58 +02:00
|
|
|
/* subtransaction in which current Executor call was started */
|
|
|
|
SubTransactionId execSubid;
|
|
|
|
|
Prevent leakage of SPI tuple tables during subtransaction abort.
plpgsql often just remembers SPI-result tuple tables in local variables,
and has no mechanism for freeing them if an ereport(ERROR) causes an escape
out of the execution function whose local variable it is. In the original
coding, that wasn't a problem because the tuple table would be cleaned up
when the function's SPI context went away during transaction abort.
However, once plpgsql grew the ability to trap exceptions, repeated
trapping of errors within a function could result in significant
intra-function-call memory leakage, as illustrated in bug #8279 from
Chad Wagner.
We could fix this locally in plpgsql with a bunch of PG_TRY/PG_CATCH
coding, but that would be tedious, probably slow, and prone to bugs of
omission; moreover it would do nothing for similar risks elsewhere.
What seems like a better plan is to make SPI itself responsible for
freeing tuple tables at subtransaction abort. This patch attacks the
problem that way, keeping a list of live tuple tables within each SPI
function context. Currently, such freeing is automatic for tuple tables
made within the failed subtransaction. We might later add a SPI call to
mark a tuple table as not to be freed this way, allowing callers to opt
out; but until someone exhibits a clear use-case for such behavior, it
doesn't seem worth bothering.
A very useful side-effect of this change is that SPI_freetuptable() can
now defend itself against bad calls, such as duplicate free requests;
this should make things more robust in many places. (In particular,
this reduces the risks involved if a third-party extension contains
now-redundant SPI_freetuptable() calls in error cleanup code.)
Even though the leakage problem is of long standing, it seems imprudent
to back-patch this into stable branches, since it does represent an API
semantics change for SPI users. We'll patch this in 9.3, but live with
the leakage in older branches.
2013-07-25 22:45:43 +02:00
|
|
|
/* resources of this execution context */
|
|
|
|
slist_head tuptables; /* list of all live SPITupleTables */
|
2000-06-28 05:33:33 +02:00
|
|
|
MemoryContext procCxt; /* procedure context */
|
|
|
|
MemoryContext execCxt; /* executor context */
|
2007-03-16 00:12:07 +01:00
|
|
|
MemoryContext savedcxt; /* context of SPI_connect's caller */
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
SubTransactionId connectSubid; /* ID of connecting subtransaction */
|
2017-04-01 06:17:18 +02:00
|
|
|
QueryEnvironment *queryEnv; /* query environment setup for SPI level */
|
Transaction control in PL procedures
In each of the supplied procedural languages (PL/pgSQL, PL/Perl,
PL/Python, PL/Tcl), add language-specific commit and rollback
functions/commands to control transactions in procedures in that
language. Add similar underlying functions to SPI. Some additional
cleanup so that transaction commit or abort doesn't blow away data
structures still used by the procedure call. Add execution context
tracking to CALL and DO statements so that transaction control commands
can only be issued in top-level procedure and block calls, not function
calls or other procedure or block calls.
- SPI
Add a new function SPI_connect_ext() that is like SPI_connect() but
allows passing option flags. The only option flag right now is
SPI_OPT_NONATOMIC. A nonatomic SPI connection can execute transaction
control commands, otherwise it's not allowed. This is meant to be
passed down from CALL and DO statements which themselves know in which
context they are called. A nonatomic SPI connection uses different
memory management. A normal SPI connection allocates its memory in
TopTransactionContext. For nonatomic connections we use PortalContext
instead. As the comment in SPI_connect_ext() (previously SPI_connect())
indicates, one could potentially use PortalContext in all cases, but it
seems safest to leave the existing uses alone, because this stuff is
complicated enough already.
SPI also gets new functions SPI_start_transaction(), SPI_commit(), and
SPI_rollback(), which can be used by PLs to implement their transaction
control logic.
- portalmem.c
Some adjustments were made in the code that cleans up portals at
transaction abort. The portal code could already handle a command
*committing* a transaction and continuing (e.g., VACUUM), but it was not
quite prepared for a command *aborting* a transaction and continuing.
In AtAbort_Portals(), remove the code that marks an active portal as
failed. As the comment there already predicted, this doesn't work if
the running command wants to keep running after transaction abort. And
it's actually not necessary, because pquery.c is careful to run all
portal code in a PG_TRY block and explicitly runs MarkPortalFailed() if
there is an exception. So the code in AtAbort_Portals() is never used
anyway.
In AtAbort_Portals() and AtCleanup_Portals(), we need to be careful not
to clean up active portals too much. This mirrors similar code in
PreCommit_Portals().
- PL/Perl
Gets new functions spi_commit() and spi_rollback()
- PL/pgSQL
Gets new commands COMMIT and ROLLBACK.
Update the PL/SQL porting example in the documentation to reflect that
transactions are now possible in procedures.
- PL/Python
Gets new functions plpy.commit and plpy.rollback.
- PL/Tcl
Gets new commands commit and rollback.
Reviewed-by: Andrew Dunstan <andrew.dunstan@2ndquadrant.com>
2018-01-22 14:30:16 +01:00
|
|
|
|
|
|
|
/* transaction management support */
|
2018-04-26 20:47:16 +02:00
|
|
|
bool atomic; /* atomic execution context, does not allow
|
|
|
|
* transactions */
|
|
|
|
bool internal_xact; /* SPI-managed transaction boundary, skip
|
|
|
|
* cleanup */
|
2018-09-08 02:09:57 +02:00
|
|
|
|
|
|
|
/* saved values of API global variables for previous nesting level */
|
|
|
|
uint64 outer_processed;
|
|
|
|
SPITupleTable *outer_tuptable;
|
|
|
|
int outer_result;
|
1999-01-27 17:15:22 +01:00
|
|
|
} _SPI_connection;
|
|
|
|
|
2007-03-16 00:12:07 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* SPI plans have three states: saved, unsaved, or temporary.
|
2007-03-16 00:12:07 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Ordinarily, the _SPI_plan struct itself as well as the argtypes array
|
|
|
|
* are in a dedicated memory context identified by plancxt (which can be
|
|
|
|
* really small). All the other subsidiary state is in plancache entries
|
|
|
|
* identified by plancache_list (note: the list cells themselves are in
|
|
|
|
* plancxt).
|
2007-03-16 00:12:07 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* In an unsaved plan, the plancxt as well as the plancache entries' contexts
|
|
|
|
* are children of the SPI procedure context, so they'll all disappear at
|
|
|
|
* function exit. plancache.c also knows that the plancache entries are
|
|
|
|
* "unsaved", so it doesn't link them into its global list; hence they do
|
|
|
|
* not respond to inval events. This is OK since we are presumably holding
|
|
|
|
* adequate locks to prevent other backends from messing with the tables.
|
2007-03-16 00:12:07 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* For a saved plan, the plancxt is made a child of CacheMemoryContext
|
2014-05-06 18:12:18 +02:00
|
|
|
* since it should persist until explicitly destroyed. Likewise, the
|
2011-09-16 06:42:53 +02:00
|
|
|
* plancache entries will be under CacheMemoryContext since we tell
|
|
|
|
* plancache.c to save them. We rely on plancache.c to keep the cache
|
|
|
|
* entries up-to-date as needed in the face of invalidation events.
|
|
|
|
*
|
|
|
|
* There are also "temporary" SPI plans, in which the _SPI_plan struct is
|
|
|
|
* not even palloc'd but just exists in some function's local variable.
|
|
|
|
* The plancache entries are unsaved and exist under the SPI executor context,
|
|
|
|
* while additional data such as argtypes and list cells is loose in the SPI
|
|
|
|
* executor context. Such plans can be identified by having plancxt == NULL.
|
2007-03-16 00:12:07 +01:00
|
|
|
*
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* We can also have "one-shot" SPI plans (which are typically temporary,
|
|
|
|
* as described above). These are meant to be executed once and discarded,
|
|
|
|
* and various optimizations are made on the assumption of single use.
|
|
|
|
* Note in particular that the CachedPlanSources within such an SPI plan
|
|
|
|
* are not "complete" until execution.
|
|
|
|
*
|
2007-03-16 00:12:07 +01:00
|
|
|
* Note: if the original query string contained only whitespace and comments,
|
|
|
|
* the plancache_list will be NIL and so there is no place to store the
|
|
|
|
* query string. We don't care about that, but we do care about the
|
|
|
|
* argument type array, which is why it's seemingly-redundantly stored.
|
|
|
|
*/
|
|
|
|
typedef struct _SPI_plan
|
1999-01-27 17:15:22 +01:00
|
|
|
{
|
2007-03-16 00:12:07 +01:00
|
|
|
int magic; /* should equal _SPI_PLAN_MAGIC */
|
|
|
|
bool saved; /* saved or unsaved plan? */
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
bool oneshot; /* one-shot plan? */
|
2018-03-24 15:05:06 +01:00
|
|
|
bool no_snapshots; /* let the caller handle the snapshots */
|
2007-11-15 22:14:46 +01:00
|
|
|
List *plancache_list; /* one CachedPlanSource per parsetree */
|
2007-03-16 00:12:07 +01:00
|
|
|
MemoryContext plancxt; /* Context containing _SPI_plan and data */
|
2007-11-15 22:14:46 +01:00
|
|
|
int cursor_options; /* Cursor options used for planning */
|
2007-03-16 00:12:07 +01:00
|
|
|
int nargs; /* number of plan arguments */
|
|
|
|
Oid *argtypes; /* Argument types (NULL if nargs is 0) */
|
2009-11-04 23:26:08 +01:00
|
|
|
ParserSetupHook parserSetup; /* alternative parameter spec method */
|
2010-02-26 03:01:40 +01:00
|
|
|
void *parserSetupArg;
|
1999-01-27 17:15:22 +01:00
|
|
|
} _SPI_plan;
|
|
|
|
|
Phase 2 of pgindent updates.
Change pg_bsd_indent to follow upstream rules for placement of comments
to the right of code, and remove pgindent hack that caused comments
following #endif to not obey the general rule.
Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using
the published version of pg_bsd_indent, but a hacked-up version that
tried to minimize the amount of movement of comments to the right of
code. The situation of interest is where such a comment has to be
moved to the right of its default placement at column 33 because there's
code there. BSD indent has always moved right in units of tab stops
in such cases --- but in the previous incarnation, indent was working
in 8-space tab stops, while now it knows we use 4-space tabs. So the
net result is that in about half the cases, such comments are placed
one tab stop left of before. This is better all around: it leaves
more room on the line for comment text, and it means that in such
cases the comment uniformly starts at the next 4-space tab stop after
the code, rather than sometimes one and sometimes two tabs after.
Also, ensure that comments following #endif are indented the same
as comments following other preprocessor commands such as #else.
That inconsistency turns out to have been self-inflicted damage
from a poorly-thought-through post-indent "fixup" in pgindent.
This patch is much less interesting than the first round of indent
changes, but also bulkier, so I thought it best to separate the effects.
Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org
Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
|
|
|
#endif /* SPI_PRIV_H */
|