rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Centralize the logic for protective copying of utility statements. 

In the "simple Query" code path, it's fine for parse analysis or
execution of a utility statement to scribble on the statement's node
tree, since that'll just be thrown away afterwards.  However it's
not fine if the node tree is in the plan cache, as then it'd be
corrupted for subsequent executions.  Up to now we've dealt with
that by having individual utility-statement functions apply
copyObject() if they were going to modify the tree.  But that's
prone to errors of omission.  Bug #17053 from Charles Samborski
shows that CREATE/ALTER DOMAIN didn't get this memo, and can
crash if executed repeatedly from plan cache.

In the back branches, we'll just apply a narrow band-aid for that,
but in HEAD it seems prudent to have a more principled fix that
will close off the possibility of other similar bugs in future.
Hence, let's hoist the responsibility for doing copyObject up into
ProcessUtility from its children, thus ensuring that it happens for
all utility statement types.

Also, modify ProcessUtility's API so that its callers can tell it
whether a copy step is necessary.  It turns out that in all cases,
the immediate caller knows whether the node tree is transient, so
this doesn't involve a huge amount of code thrashing.  In this way,
while we lose a little bit in the execute-from-cache code path due
to sometimes copying node trees that wouldn't be mutated anyway,
we gain something in the simple-Query code path by not copying
throwaway node trees.  Statements that are complex enough to be
expensive to copy are almost certainly ones that would have to be
copied anyway, so the loss in the cache code path shouldn't be much.

(Note that this whole problem applies only to utility statements.
Optimizable statements don't have the issue because we long ago made
the executor treat Plan trees as read-only.  Perhaps someday we will
make utility statement execution act likewise, but I'm not holding
my breath.)

Discussion: https://postgr.es/m/931771.1623893989@sss.pgh.pa.us
Discussion: https://postgr.es/m/17053-3ca3f501bbc212b4@postgresql.org
This commit is contained in:
Tom Lane 2021-06-18 11:22:58 -04:00
parent 0a4efdc7eb
commit 7c337b6b52
19 changed files with 56 additions and 91 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
contrib/pg_stat_statements/pg_stat_statements.c
View File

@ -320,6 +320,7 @@ static void pgss_ExecutorRun(QueryDesc *queryDesc,
static void pgss_ExecutorFinish(QueryDesc *queryDesc);
static void pgss_ExecutorEnd(QueryDesc *queryDesc);
static void pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context, ParamListInfo params,
QueryEnvironment *queryEnv,
DestReceiver *dest, QueryCompletion *qc);
@ -1069,6 +1070,7 @@ pgss_ExecutorEnd(QueryDesc *queryDesc)
*/
static void
pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context,
ParamListInfo params, QueryEnvironment *queryEnv,
DestReceiver *dest, QueryCompletion *qc)
@ -1126,11 +1128,11 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
PG_TRY();
{
if (prev_ProcessUtility)
prev_ProcessUtility(pstmt, queryString,
prev_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
else
standard_ProcessUtility(pstmt, queryString,
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
}
@ -1176,11 +1178,11 @@ pgss_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
else
{
if (prev_ProcessUtility)
prev_ProcessUtility(pstmt, queryString,
prev_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
else
standard_ProcessUtility(pstmt, queryString,
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
}

5
contrib/sepgsql/hooks.c
View File

@ -313,6 +313,7 @@ sepgsql_exec_check_perms(List *rangeTabls, bool abort)
static void
sepgsql_utility_command(PlannedStmt *pstmt,
const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context,
ParamListInfo params,
QueryEnvironment *queryEnv,
@ -378,11 +379,11 @@ sepgsql_utility_command(PlannedStmt *pstmt,
}
if (next_ProcessUtility_hook)
(*next_ProcessUtility_hook) (pstmt, queryString,
(*next_ProcessUtility_hook) (pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
else
standard_ProcessUtility(pstmt, queryString,
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
}

8
src/backend/commands/copyto.c
View File

@ -438,14 +438,8 @@ BeginCopyTo(ParseState *pstate,
/*
* Run parse analysis and rewrite. Note this also acquires sufficient
* locks on the source table(s).
*
* Because the parser and planner tend to scribble on their input, we
* make a preliminary copy of the source querytree. This prevents
* problems in the case that the COPY is in a portal or plpgsql
* function and is executed repeatedly. (See also the same hack in
* DECLARE CURSOR and PREPARE.) XXX FIXME someday.
*/
rewritten = pg_analyze_and_rewrite(copyObject(raw_query),
rewritten = pg_analyze_and_rewrite(raw_query,
pstate->p_sourcetext, NULL, 0,
NULL);

8
src/backend/commands/createas.c
View File

@ -299,14 +299,8 @@ ExecCreateTableAs(ParseState *pstate, CreateTableAsStmt *stmt,
* rewriter. We do not do AcquireRewriteLocks: we assume the query
* either came straight from the parser, or suitable locks were
* acquired by plancache.c.
*
* Because the rewriter and planner tend to scribble on the input, we
* make a preliminary copy of the source querytree. This prevents
* problems in the case that CTAS is in a portal or plpgsql function
* and is executed repeatedly. (See also the same hack in EXPLAIN and
* PREPARE.)
*/
rewritten = QueryRewrite(copyObject(query));
rewritten = QueryRewrite(query);
/* SELECT should never rewrite to more or less than one SELECT query */
if (list_length(rewritten) != 1)

14
src/backend/commands/explain.c
View File

@ -256,14 +256,8 @@ ExplainQuery(ParseState *pstate, ExplainStmt *stmt,
* rewriter. We do not do AcquireRewriteLocks: we assume the query either
* came straight from the parser, or suitable locks were acquired by
* plancache.c.
*
* Because the rewriter and planner tend to scribble on the input, we make
* a preliminary copy of the source querytree. This prevents problems in
* the case that the EXPLAIN is in a portal or plpgsql function and is
* executed repeatedly. (See also the same hack in DECLARE CURSOR and
* PREPARE.) XXX FIXME someday.
*/
rewritten = QueryRewrite(castNode(Query, copyObject(stmt->query)));
rewritten = QueryRewrite(castNode(Query, stmt->query));
/* emit opening boilerplate */
ExplainBeginOutput(es);
@ -427,7 +421,8 @@ ExplainOneQuery(Query *query, int cursorOptions,
* "into" is NULL unless we are explaining the contents of a CreateTableAsStmt.
*
* This is exported because it's called back from prepare.c in the
* EXPLAIN EXECUTE case.
* EXPLAIN EXECUTE case. In that case, we'll be dealing with a statement
* that's in the plan cache, so we have to ensure we don't modify it.
*/
void
ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
@ -441,8 +436,7 @@ ExplainOneUtility(Node *utilityStmt, IntoClause *into, ExplainState *es,
{
/*
* We have to rewrite the contained SELECT and then pass it back to
* ExplainOneQuery. It's probably not really necessary to copy the
* contained parsetree another time, but let's be safe.
* ExplainOneQuery. Copy to be safe in the EXPLAIN EXECUTE case.
*/
CreateTableAsStmt *ctas = (CreateTableAsStmt *) utilityStmt;
List *rewritten;

1
src/backend/commands/extension.c
View File

@ -786,6 +786,7 @@ execute_sql_string(const char *sql)
ProcessUtility(stmt,
sql,
false,
PROCESS_UTILITY_QUERY,
NULL,
NULL,

3
src/backend/commands/foreigncmds.c
View File

@ -1570,8 +1570,7 @@ ImportForeignSchema(ImportForeignSchemaStmt *stmt)
pstmt->stmt_len = rs->stmt_len;
/* Execute statement */
ProcessUtility(pstmt,
cmd,
ProcessUtility(pstmt, cmd, false,
PROCESS_UTILITY_SUBCOMMAND, NULL, NULL,
None_Receiver, NULL);

8
src/backend/commands/policy.c
View File

@ -747,12 +747,12 @@ CreatePolicy(CreatePolicyStmt *stmt)
addNSItemToQuery(with_check_pstate, nsitem, false, true, true);
qual = transformWhereClause(qual_pstate,
copyObject(stmt->qual),
stmt->qual,
EXPR_KIND_POLICY,
"POLICY");
with_check_qual = transformWhereClause(with_check_pstate,
copyObject(stmt->with_check),
stmt->with_check,
EXPR_KIND_POLICY,
"POLICY");
@ -922,7 +922,7 @@ AlterPolicy(AlterPolicyStmt *stmt)
addNSItemToQuery(qual_pstate, nsitem, false, true, true);
qual = transformWhereClause(qual_pstate, copyObject(stmt->qual),
qual = transformWhereClause(qual_pstate, stmt->qual,
EXPR_KIND_POLICY,
"POLICY");
@ -946,7 +946,7 @@ AlterPolicy(AlterPolicyStmt *stmt)
addNSItemToQuery(with_check_pstate, nsitem, false, true, true);
with_check_qual = transformWhereClause(with_check_pstate,
copyObject(stmt->with_check),
stmt->with_check,
EXPR_KIND_POLICY,
"POLICY");

8
src/backend/commands/portalcmds.c
View File

@ -76,14 +76,8 @@ PerformCursorOpen(ParseState *pstate, DeclareCursorStmt *cstmt, ParamListInfo pa
* rewriter. We do not do AcquireRewriteLocks: we assume the query either
* came straight from the parser, or suitable locks were acquired by
* plancache.c.
*
* Because the rewriter and planner tend to scribble on the input, we make
* a preliminary copy of the source querytree. This prevents problems in
* the case that the DECLARE CURSOR is in a portal or plpgsql function and
* is executed repeatedly. (See also the same hack in EXPLAIN and
* PREPARE.) XXX FIXME someday.
*/
rewritten = QueryRewrite((Query *) copyObject(query));
rewritten = QueryRewrite(query);
/* SELECT should never rewrite to more or less than one query */
if (list_length(rewritten) != 1)

5
src/backend/commands/prepare.c
View File

@ -78,12 +78,9 @@ PrepareQuery(ParseState *pstate, PrepareStmt *stmt,
/*
* Need to wrap the contained statement in a RawStmt node to pass it to
* parse analysis.
*
* Because parse analysis scribbles on the raw querytree, we must make a
* copy to ensure we don't modify the passed-in tree. FIXME someday.
*/
rawstmt = makeNode(RawStmt);
rawstmt->stmt = (Node *) copyObject(stmt->query);
rawstmt->stmt = stmt->query;
rawstmt->stmt_location = stmt_location;
rawstmt->stmt_len = stmt_len;

1
src/backend/commands/schemacmds.c
View File

@ -191,6 +191,7 @@ CreateSchemaCommand(CreateSchemaStmt *stmt, const char *queryString,
/* do this step */
ProcessUtility(wrapper,
queryString,
false,
PROCESS_UTILITY_SUBCOMMAND,
NULL,
NULL,

3
src/backend/commands/tablecmds.c
View File

@ -4408,8 +4408,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
* Copy the original subcommand for each table. This avoids conflicts
* when different child tables need to make different parse
* transformations (for example, the same column may have different column
* numbers in different children). It also ensures that we don't corrupt
* the original parse tree, in case it is saved in plancache.
* numbers in different children).
*/
cmd = copyObject(cmd);

5
src/backend/commands/view.c
View File

@ -417,12 +417,9 @@ DefineView(ViewStmt *stmt, const char *queryString,
/*
* Run parse analysis to convert the raw parse tree to a Query. Note this
* also acquires sufficient locks on the source table(s).
*
* Since parse analysis scribbles on its input, copy the raw parse tree;
* this ensures we don't corrupt a prepared statement, for example.
*/
rawstmt = makeNode(RawStmt);
rawstmt->stmt = (Node *) copyObject(stmt->query);
rawstmt->stmt = stmt->query;
rawstmt->stmt_location = stmt_location;
rawstmt->stmt_len = stmt_len;

1
src/backend/executor/functions.c
View File

@ -886,6 +886,7 @@ postquel_getnext(execution_state *es, SQLFunctionCachePtr fcache)
{
ProcessUtility(es->qd->plannedstmt,
fcache->src,
false,
PROCESS_UTILITY_QUERY,
es->qd->params,
es->qd->queryEnv,

1
src/backend/executor/spi.c
View File

@ -2545,6 +2545,7 @@ _SPI_execute_plan(SPIPlanPtr plan, ParamListInfo paramLI,
InitializeQueryCompletion(&qc);
ProcessUtility(stmt,
plansource->query_string,
true, /* protect plancache's node tree */
context,
paramLI,
_SPI_current->queryEnv,

36
src/backend/parser/parse_utilcmd.c
View File

@ -11,10 +11,6 @@
* Hence these functions are now called at the start of execution of their
* respective utility commands.
*
* NOTE: in general we must avoid scribbling on the passed-in raw parse
* tree, since it might be in a plan cache. The simplest solution is
* a quick copyObject() call before manipulating the query tree.
*
*
* Portions Copyright (c) 1996-2021, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
@ -177,12 +173,6 @@ transformCreateStmt(CreateStmt *stmt, const char *queryString)
Oid existing_relid;
ParseCallbackState pcbstate;
/*
* We must not scribble on the passed-in CreateStmt, so copy it. (This is
* overkill, but easy.)
*/
stmt = copyObject(stmt);
/* Set up pstate */
pstate = make_parsestate(NULL);
pstate->p_sourcetext = queryString;
@ -2824,12 +2814,6 @@ transformIndexStmt(Oid relid, IndexStmt *stmt, const char *queryString)
if (stmt->transformed)
return stmt;
/*
* We must not scribble on the passed-in IndexStmt, so copy it. (This is
* overkill, but easy.)
*/
stmt = copyObject(stmt);
/* Set up pstate */
pstate = make_parsestate(NULL);
pstate->p_sourcetext = queryString;
@ -2925,12 +2909,6 @@ transformStatsStmt(Oid relid, CreateStatsStmt *stmt, const char *queryString)
if (stmt->transformed)
return stmt;
/*
* We must not scribble on the passed-in CreateStatsStmt, so copy it.
* (This is overkill, but easy.)
*/
stmt = copyObject(stmt);
/* Set up pstate */
pstate = make_parsestate(NULL);
pstate->p_sourcetext = queryString;
@ -2993,9 +2971,6 @@ transformStatsStmt(Oid relid, CreateStatsStmt *stmt, const char *queryString)
*
* actions and whereClause are output parameters that receive the
* transformed results.
*
* Note that we must not scribble on the passed-in RuleStmt, so we do
* copyObject() on the actions and WHERE clause.
*/
void
transformRuleStmt(RuleStmt *stmt, const char *queryString,
@ -3070,7 +3045,7 @@ transformRuleStmt(RuleStmt *stmt, const char *queryString,
/* take care of the where clause */
*whereClause = transformWhereClause(pstate,
(Node *) copyObject(stmt->whereClause),
stmt->whereClause,
EXPR_KIND_WHERE,
"WHERE");
/* we have to fix its collations too */
@ -3142,8 +3117,7 @@ transformRuleStmt(RuleStmt *stmt, const char *queryString,
addNSItemToQuery(sub_pstate, newnsitem, false, true, false);
/* Transform the rule action statement */
top_subqry = transformStmt(sub_pstate,
(Node *) copyObject(action));
top_subqry = transformStmt(sub_pstate, action);
/*
* We cannot support utility-statement actions (eg NOTIFY) with
@ -3325,12 +3299,6 @@ transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
AlterTableCmd *newcmd;
ParseNamespaceItem *nsitem;
/*
* We must not scribble on the passed-in AlterTableStmt, so copy it. (This
* is overkill, but easy.)
*/
stmt = copyObject(stmt);
/* Caller is responsible for locking the relation */
rel = relation_open(relid, NoLock);
tupdesc = RelationGetDescr(rel);

1
src/backend/tcop/pquery.c
View File

@ -1146,6 +1146,7 @@ PortalRunUtility(Portal portal, PlannedStmt *pstmt,
ProcessUtility(pstmt,
portal->sourceText,
(portal->cplan != NULL), /* protect tree if in plancache */
isTopLevel ? PROCESS_UTILITY_TOPLEVEL : PROCESS_UTILITY_QUERY,
portal->portalParams,
portal->queryEnv,

23
src/backend/tcop/utility.c
View File

@ -476,6 +476,7 @@ CheckRestrictedOperation(const char *cmdname)
*
* pstmt: PlannedStmt wrapper for the utility statement
* queryString: original source text of command
* readOnlyTree: if true, pstmt's node tree must not be modified
* context: identifies source of statement (toplevel client command,
* non-toplevel client command, subcommand of a larger utility command)
* params: parameters to use during execution
@ -501,6 +502,7 @@ CheckRestrictedOperation(const char *cmdname)
void
ProcessUtility(PlannedStmt *pstmt,
const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context,
ParamListInfo params,
QueryEnvironment *queryEnv,
@ -518,11 +520,11 @@ ProcessUtility(PlannedStmt *pstmt,
* call standard_ProcessUtility().
*/
if (ProcessUtility_hook)
(*ProcessUtility_hook) (pstmt, queryString,
(*ProcessUtility_hook) (pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
else
standard_ProcessUtility(pstmt, queryString,
standard_ProcessUtility(pstmt, queryString, readOnlyTree,
context, params, queryEnv,
dest, qc);
}
@ -541,13 +543,14 @@ ProcessUtility(PlannedStmt *pstmt,
void
standard_ProcessUtility(PlannedStmt *pstmt,
const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context,
ParamListInfo params,
QueryEnvironment *queryEnv,
DestReceiver *dest,
QueryCompletion *qc)
{
Node *parsetree = pstmt->utilityStmt;
Node *parsetree;
bool isTopLevel = (context == PROCESS_UTILITY_TOPLEVEL);
bool isAtomicContext = (!(context == PROCESS_UTILITY_TOPLEVEL || context == PROCESS_UTILITY_QUERY_NONATOMIC) || IsTransactionBlock());
ParseState *pstate;
@ -556,6 +559,18 @@ standard_ProcessUtility(PlannedStmt *pstmt,
/* This can recurse, so check for excessive recursion */
check_stack_depth();
/*
* If the given node tree is read-only, make a copy to ensure that parse
* transformations don't damage the original tree. This could be
* refactored to avoid making unnecessary copies in more cases, but it's
* not clear that it's worth a great deal of trouble over. Statements
* that are complex enough to be expensive to copy are exactly the ones
* we'd need to copy, so that only marginal savings seem possible.
*/
if (readOnlyTree)
pstmt = copyObject(pstmt);
parsetree = pstmt->utilityStmt;
/* Prohibit read/write commands in read-only states. */
readonly_flags = ClassifyUtilityCommandAsReadOnly(parsetree);
if (readonly_flags != COMMAND_IS_STRICTLY_READ_ONLY &&
@ -1211,6 +1226,7 @@ ProcessUtilitySlow(ParseState *pstate,
ProcessUtility(wrapper,
queryString,
false,
PROCESS_UTILITY_SUBCOMMAND,
params,
NULL,
@ -1918,6 +1934,7 @@ ProcessUtilityForAlterTable(Node *stmt, AlterTableUtilityContext *context)
ProcessUtility(wrapper,
context->queryString,
false,
PROCESS_UTILITY_SUBCOMMAND,
context->params,
context->queryEnv,

6
src/include/tcop/utility.h
View File

@ -69,17 +69,21 @@ typedef struct AlterTableUtilityContext
/* Hook for plugins to get control in ProcessUtility() */
typedef void (*ProcessUtility_hook_type) (PlannedStmt *pstmt,
const char *queryString, ProcessUtilityContext context,
const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context,
ParamListInfo params,
QueryEnvironment *queryEnv,
DestReceiver *dest, QueryCompletion *qc);
extern PGDLLIMPORT ProcessUtility_hook_type ProcessUtility_hook;
extern void ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context, ParamListInfo params,
QueryEnvironment *queryEnv,
DestReceiver *dest, QueryCompletion *qc);
extern void standard_ProcessUtility(PlannedStmt *pstmt, const char *queryString,
bool readOnlyTree,
ProcessUtilityContext context, ParamListInfo params,
QueryEnvironment *queryEnv,
DestReceiver *dest, QueryCompletion *qc);