2007-03-13 01:33:44 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* plancache.c
|
|
|
|
* Plan cache management.
|
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* The plan cache manager has two principal responsibilities: deciding when
|
|
|
|
* to use a generic plan versus a custom (parameter-value-specific) plan,
|
|
|
|
* and tracking whether cached plans need to be invalidated because of schema
|
|
|
|
* changes in the objects they depend on.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* The logic for choosing generic or custom plans is in choose_custom_plan,
|
|
|
|
* which see for comments.
|
|
|
|
*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Cache invalidation is driven off sinval events. Any CachedPlanSource
|
2011-09-16 06:42:53 +02:00
|
|
|
* that matches the event is marked invalid, as is its generic CachedPlan
|
|
|
|
* if it has one. When (and if) the next demand for a cached plan occurs,
|
|
|
|
* parse analysis and rewrite is repeated to build a new valid query tree,
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
* and then planning is performed as normal. We also force re-analysis and
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
* re-planning if the active search_path is different from the previous time
|
|
|
|
* or, if RLS is involved, if the user changes or the RLS environment changes.
|
2011-09-16 06:42:53 +02:00
|
|
|
*
|
|
|
|
* Note that if the sinval was a result of user DDL actions, parse analysis
|
|
|
|
* could throw an error, for example if a column referenced by the query is
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
* no longer present. Another possibility is for the query's output tupdesc
|
|
|
|
* to change (for instance "SELECT *" might expand differently than before).
|
|
|
|
* The creator of a cached plan can specify whether it is allowable for the
|
|
|
|
* query to change output tupdesc on replan --- if so, it's up to the
|
2007-03-13 01:33:44 +01:00
|
|
|
* caller to notice changes and cope with them.
|
|
|
|
*
|
2008-09-09 20:58:09 +02:00
|
|
|
* Currently, we track exactly the dependencies of plans on relations and
|
2014-05-06 18:12:18 +02:00
|
|
|
* user-defined functions. On relcache invalidation events or pg_proc
|
2008-09-09 20:58:09 +02:00
|
|
|
* syscache invalidation events, we invalidate just those plans that depend
|
|
|
|
* on the particular object being modified. (Note: this scheme assumes
|
|
|
|
* that any table modification that requires replanning will generate a
|
|
|
|
* relcache inval event.) We also watch for inval events on certain other
|
|
|
|
* system catalogs, such as pg_namespace; but for them, our response is
|
|
|
|
* just to invalidate all plans. We expect updates on those catalogs to
|
|
|
|
* be infrequent enough that more-detailed tracking is not worth the effort.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
|
|
|
*
|
2016-01-02 19:33:40 +01:00
|
|
|
* Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
|
2007-03-13 01:33:44 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/backend/utils/cache/plancache.c
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#include "postgres.h"
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
#include <limits.h>
|
|
|
|
|
2007-09-20 19:56:33 +02:00
|
|
|
#include "access/transam.h"
|
2007-03-23 20:53:52 +01:00
|
|
|
#include "catalog/namespace.h"
|
2007-03-13 01:33:44 +01:00
|
|
|
#include "executor/executor.h"
|
2009-07-14 17:37:50 +02:00
|
|
|
#include "executor/spi.h"
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
#include "miscadmin.h"
|
2008-08-26 00:42:34 +02:00
|
|
|
#include "nodes/nodeFuncs.h"
|
2013-08-24 21:14:17 +02:00
|
|
|
#include "optimizer/cost.h"
|
2008-09-09 20:58:09 +02:00
|
|
|
#include "optimizer/planmain.h"
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
#include "optimizer/prep.h"
|
2011-09-25 23:33:32 +02:00
|
|
|
#include "parser/analyze.h"
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
#include "parser/parsetree.h"
|
2007-03-13 01:33:44 +01:00
|
|
|
#include "storage/lmgr.h"
|
|
|
|
#include "tcop/pquery.h"
|
|
|
|
#include "tcop/utility.h"
|
|
|
|
#include "utils/inval.h"
|
|
|
|
#include "utils/memutils.h"
|
2012-08-29 00:02:07 +02:00
|
|
|
#include "utils/resowner_private.h"
|
Fix column-privilege leak in error-message paths
While building error messages to return to the user,
BuildIndexValueDescription, ExecBuildSlotValueDescription and
ri_ReportViolation would happily include the entire key or entire row in
the result returned to the user, even if the user didn't have access to
view all of the columns being included.
Instead, include only those columns which the user is providing or which
the user has select rights on. If the user does not have any rights
to view the table or any of the columns involved then no detail is
provided and a NULL value is returned from BuildIndexValueDescription
and ExecBuildSlotValueDescription. Note that, for key cases, the user
must have access to all of the columns for the key to be shown; a
partial key will not be returned.
Further, in master only, do not return any data for cases where row
security is enabled on the relation and row security should be applied
for the user. This required a bit of refactoring and moving of things
around related to RLS- note the addition of utils/misc/rls.c.
Back-patch all the way, as column-level privileges are now in all
supported versions.
This has been assigned CVE-2014-8161, but since the issue and the patch
have already been publicized on pgsql-hackers, there's no point in trying
to hide this commit.
2015-01-12 23:04:11 +01:00
|
|
|
#include "utils/rls.h"
|
2008-03-26 19:48:59 +01:00
|
|
|
#include "utils/snapmgr.h"
|
2008-09-09 20:58:09 +02:00
|
|
|
#include "utils/syscache.h"
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
/*
|
|
|
|
* We must skip "overhead" operations that involve database access when the
|
|
|
|
* cached plan's subject statement is a transaction control command.
|
|
|
|
*/
|
|
|
|
#define IsTransactionStmtPlan(plansource) \
|
|
|
|
((plansource)->raw_parse_tree && \
|
|
|
|
IsA((plansource)->raw_parse_tree, TransactionStmt))
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* This is the head of the backend's list of "saved" CachedPlanSources (i.e.,
|
|
|
|
* those that are in long-lived storage and are examined for sinval events).
|
|
|
|
* We thread the structs manually instead of using List cells so that we can
|
|
|
|
* guarantee to save a CachedPlanSource without error.
|
|
|
|
*/
|
|
|
|
static CachedPlanSource *first_saved_plan = NULL;
|
|
|
|
|
|
|
|
static void ReleaseGenericPlan(CachedPlanSource *plansource);
|
|
|
|
static List *RevalidateCachedQuery(CachedPlanSource *plansource);
|
|
|
|
static bool CheckCachedPlan(CachedPlanSource *plansource);
|
|
|
|
static CachedPlan *BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
|
2012-06-10 21:20:04 +02:00
|
|
|
ParamListInfo boundParams);
|
2011-09-16 06:42:53 +02:00
|
|
|
static bool choose_custom_plan(CachedPlanSource *plansource,
|
2012-06-10 21:20:04 +02:00
|
|
|
ParamListInfo boundParams);
|
2013-08-24 21:14:17 +02:00
|
|
|
static double cached_plan_cost(CachedPlan *plan, bool include_planner);
|
2007-03-13 01:33:44 +01:00
|
|
|
static void AcquireExecutorLocks(List *stmt_list, bool acquire);
|
|
|
|
static void AcquirePlannerLocks(List *stmt_list, bool acquire);
|
2008-09-09 20:58:09 +02:00
|
|
|
static void ScanQueryForLocks(Query *parsetree, bool acquire);
|
|
|
|
static bool ScanQueryWalker(Node *node, bool *acquire);
|
2007-09-20 19:56:33 +02:00
|
|
|
static bool plan_list_is_transient(List *stmt_list);
|
2011-09-16 06:42:53 +02:00
|
|
|
static TupleDesc PlanCacheComputeResultDesc(List *stmt_list);
|
2008-09-09 20:58:09 +02:00
|
|
|
static void PlanCacheRelCallback(Datum arg, Oid relid);
|
2011-08-17 01:27:46 +02:00
|
|
|
static void PlanCacheFuncCallback(Datum arg, int cacheid, uint32 hashvalue);
|
|
|
|
static void PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue);
|
2016-01-28 20:05:36 +01:00
|
|
|
static void PlanCacheUserMappingCallback(Datum arg, int cacheid,
|
2016-06-10 00:02:36 +02:00
|
|
|
uint32 hashvalue);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* InitPlanCache: initialize module during InitPostgres.
|
|
|
|
*
|
2008-09-09 20:58:09 +02:00
|
|
|
* All we need to do is hook into inval.c's callback lists.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
void
|
|
|
|
InitPlanCache(void)
|
|
|
|
{
|
2008-09-09 20:58:09 +02:00
|
|
|
CacheRegisterRelcacheCallback(PlanCacheRelCallback, (Datum) 0);
|
|
|
|
CacheRegisterSyscacheCallback(PROCOID, PlanCacheFuncCallback, (Datum) 0);
|
|
|
|
CacheRegisterSyscacheCallback(NAMESPACEOID, PlanCacheSysCallback, (Datum) 0);
|
|
|
|
CacheRegisterSyscacheCallback(OPEROID, PlanCacheSysCallback, (Datum) 0);
|
|
|
|
CacheRegisterSyscacheCallback(AMOPOPID, PlanCacheSysCallback, (Datum) 0);
|
2016-01-28 20:05:36 +01:00
|
|
|
/* User mapping change may invalidate plans with pushed down foreign join */
|
|
|
|
CacheRegisterSyscacheCallback(USERMAPPINGOID, PlanCacheUserMappingCallback, (Datum) 0);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* CreateCachedPlan: initially create a plan cache entry.
|
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Creation of a cached plan is divided into two steps, CreateCachedPlan and
|
2014-05-06 18:12:18 +02:00
|
|
|
* CompleteCachedPlan. CreateCachedPlan should be called after running the
|
2011-09-16 06:42:53 +02:00
|
|
|
* query through raw_parser, but before doing parse analysis and rewrite;
|
|
|
|
* CompleteCachedPlan is called after that. The reason for this arrangement
|
|
|
|
* is that it can save one round of copying of the raw parse tree, since
|
|
|
|
* the parser will normally scribble on the raw parse tree. Callers would
|
|
|
|
* otherwise need to make an extra copy of the parse tree to ensure they
|
|
|
|
* still had a clean copy to present at plan cache creation time.
|
|
|
|
*
|
|
|
|
* All arguments presented to CreateCachedPlan are copied into a memory
|
|
|
|
* context created as a child of the call-time CurrentMemoryContext, which
|
|
|
|
* should be a reasonably short-lived working context that will go away in
|
|
|
|
* event of an error. This ensures that the cached plan data structure will
|
|
|
|
* likewise disappear if an error occurs before we have fully constructed it.
|
|
|
|
* Once constructed, the cached plan can be made longer-lived, if needed,
|
|
|
|
* by calling SaveCachedPlan.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
2014-11-12 21:58:37 +01:00
|
|
|
* raw_parse_tree: output of raw_parser(), or NULL if empty query
|
2011-09-16 06:42:53 +02:00
|
|
|
* query_string: original query text
|
2007-03-13 01:33:44 +01:00
|
|
|
* commandTag: compile-time-constant tag for query, or NULL if empty query
|
|
|
|
*/
|
|
|
|
CachedPlanSource *
|
|
|
|
CreateCachedPlan(Node *raw_parse_tree,
|
|
|
|
const char *query_string,
|
2011-09-16 06:42:53 +02:00
|
|
|
const char *commandTag)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
|
|
|
CachedPlanSource *plansource;
|
|
|
|
MemoryContext source_context;
|
|
|
|
MemoryContext oldcxt;
|
|
|
|
|
2009-06-11 16:49:15 +02:00
|
|
|
Assert(query_string != NULL); /* required as of 8.4 */
|
Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement. That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text. Fix it to copy the source text over
from the cached plan. Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.
There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
ProcessUtility((Node *) atstmt,
"(generated ALTER TABLE ADD FOREIGN KEY command)",
NULL, false, None_Receiver, NULL);
We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.
ITAGAKI Takahiro and Tom Lane
2008-07-18 22:26:06 +02:00
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
|
|
|
* Make a dedicated memory context for the CachedPlanSource and its
|
2011-09-16 06:42:53 +02:00
|
|
|
* permanent subsidiary data. It's probably not going to be large, but
|
|
|
|
* just in case, use the default maxsize parameter. Initially it's a
|
|
|
|
* child of the caller's context (which we assume to be transient), so
|
|
|
|
* that it will be cleaned up on error.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
source_context = AllocSetContextCreate(CurrentMemoryContext,
|
2007-03-13 01:33:44 +01:00
|
|
|
"CachedPlanSource",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
2011-09-16 06:42:53 +02:00
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
2007-03-23 20:53:52 +01:00
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
|
|
|
* Create and fill the CachedPlanSource struct within the new context.
|
2011-09-16 06:42:53 +02:00
|
|
|
* Most fields are just left empty for the moment.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
oldcxt = MemoryContextSwitchTo(source_context);
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
plansource = (CachedPlanSource *) palloc0(sizeof(CachedPlanSource));
|
|
|
|
plansource->magic = CACHEDPLANSOURCE_MAGIC;
|
2007-03-13 01:33:44 +01:00
|
|
|
plansource->raw_parse_tree = copyObject(raw_parse_tree);
|
Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement. That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text. Fix it to copy the source text over
from the cached plan. Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.
There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
ProcessUtility((Node *) atstmt,
"(generated ALTER TABLE ADD FOREIGN KEY command)",
NULL, false, None_Receiver, NULL);
We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.
ITAGAKI Takahiro and Tom Lane
2008-07-18 22:26:06 +02:00
|
|
|
plansource->query_string = pstrdup(query_string);
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->commandTag = commandTag;
|
|
|
|
plansource->param_types = NULL;
|
|
|
|
plansource->num_params = 0;
|
2009-11-04 23:26:08 +01:00
|
|
|
plansource->parserSetup = NULL;
|
|
|
|
plansource->parserSetupArg = NULL;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->cursor_options = 0;
|
|
|
|
plansource->fixed_result = false;
|
|
|
|
plansource->resultDesc = NULL;
|
2007-03-13 01:33:44 +01:00
|
|
|
plansource->context = source_context;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->query_list = NIL;
|
|
|
|
plansource->relationOids = NIL;
|
|
|
|
plansource->invalItems = NIL;
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
plansource->search_path = NULL;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->query_context = NULL;
|
|
|
|
plansource->gplan = NULL;
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
plansource->is_oneshot = false;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->is_complete = false;
|
|
|
|
plansource->is_saved = false;
|
|
|
|
plansource->is_valid = false;
|
|
|
|
plansource->generation = 0;
|
|
|
|
plansource->next_saved = NULL;
|
|
|
|
plansource->generic_cost = -1;
|
|
|
|
plansource->total_custom_cost = 0;
|
|
|
|
plansource->num_custom_plans = 0;
|
Rename pg_rowsecurity -> pg_policy and other fixes
As pointed out by Robert, we should really have named pg_rowsecurity
pg_policy, as the objects stored in that catalog are policies. This
patch fixes that and updates the column names to start with 'pol' to
match the new catalog name.
The security consideration for COPY with row level security, also
pointed out by Robert, has also been addressed by remembering and
re-checking the OID of the relation initially referenced during COPY
processing, to make sure it hasn't changed under us by the time we
finish planning out the query which has been built.
Robert and Alvaro also commented on missing OCLASS and OBJECT entries
for POLICY (formerly ROWSECURITY or POLICY, depending) in various
places. This patch fixes that too, which also happens to add the
ability to COMMENT on policies.
In passing, attempt to improve the consistency of messages, comments,
and documentation as well. This removes various incarnations of
'row-security', 'row-level security', 'Row-security', etc, in favor
of 'policy', 'row level security' or 'row_security' as appropriate.
Happy Thanksgiving!
2014-11-27 07:06:36 +01:00
|
|
|
plansource->hasRowSecurity = false;
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
plansource->planUserId = InvalidOid;
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
plansource->row_security_env = false;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
MemoryContextSwitchTo(oldcxt);
|
|
|
|
|
|
|
|
return plansource;
|
|
|
|
}
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/*
|
|
|
|
* CreateOneShotCachedPlan: initially create a one-shot plan cache entry.
|
|
|
|
*
|
|
|
|
* This variant of CreateCachedPlan creates a plan cache entry that is meant
|
|
|
|
* to be used only once. No data copying occurs: all data structures remain
|
|
|
|
* in the caller's memory context (which typically should get cleared after
|
|
|
|
* completing execution). The CachedPlanSource struct itself is also created
|
|
|
|
* in that context.
|
|
|
|
*
|
|
|
|
* A one-shot plan cannot be saved or copied, since we make no effort to
|
2014-05-06 18:12:18 +02:00
|
|
|
* preserve the raw parse tree unmodified. There is also no support for
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* invalidation, so plan use must be completed in the current transaction,
|
|
|
|
* and DDL that might invalidate the querytree_list must be avoided as well.
|
|
|
|
*
|
2014-11-12 21:58:37 +01:00
|
|
|
* raw_parse_tree: output of raw_parser(), or NULL if empty query
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* query_string: original query text
|
|
|
|
* commandTag: compile-time-constant tag for query, or NULL if empty query
|
|
|
|
*/
|
|
|
|
CachedPlanSource *
|
|
|
|
CreateOneShotCachedPlan(Node *raw_parse_tree,
|
|
|
|
const char *query_string,
|
|
|
|
const char *commandTag)
|
|
|
|
{
|
|
|
|
CachedPlanSource *plansource;
|
|
|
|
|
|
|
|
Assert(query_string != NULL); /* required as of 8.4 */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Create and fill the CachedPlanSource struct within the caller's memory
|
|
|
|
* context. Most fields are just left empty for the moment.
|
|
|
|
*/
|
|
|
|
plansource = (CachedPlanSource *) palloc0(sizeof(CachedPlanSource));
|
|
|
|
plansource->magic = CACHEDPLANSOURCE_MAGIC;
|
|
|
|
plansource->raw_parse_tree = raw_parse_tree;
|
|
|
|
plansource->query_string = query_string;
|
|
|
|
plansource->commandTag = commandTag;
|
|
|
|
plansource->param_types = NULL;
|
|
|
|
plansource->num_params = 0;
|
|
|
|
plansource->parserSetup = NULL;
|
|
|
|
plansource->parserSetupArg = NULL;
|
|
|
|
plansource->cursor_options = 0;
|
|
|
|
plansource->fixed_result = false;
|
|
|
|
plansource->resultDesc = NULL;
|
|
|
|
plansource->context = CurrentMemoryContext;
|
|
|
|
plansource->query_list = NIL;
|
|
|
|
plansource->relationOids = NIL;
|
|
|
|
plansource->invalItems = NIL;
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
plansource->search_path = NULL;
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
plansource->query_context = NULL;
|
|
|
|
plansource->gplan = NULL;
|
|
|
|
plansource->is_oneshot = true;
|
|
|
|
plansource->is_complete = false;
|
|
|
|
plansource->is_saved = false;
|
|
|
|
plansource->is_valid = false;
|
|
|
|
plansource->generation = 0;
|
|
|
|
plansource->next_saved = NULL;
|
|
|
|
plansource->generic_cost = -1;
|
|
|
|
plansource->total_custom_cost = 0;
|
|
|
|
plansource->num_custom_plans = 0;
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
plansource->planUserId = InvalidOid;
|
|
|
|
plansource->row_security_env = false;
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
|
|
|
|
return plansource;
|
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* CompleteCachedPlan: second step of creating a plan cache entry.
|
|
|
|
*
|
|
|
|
* Pass in the analyzed-and-rewritten form of the query, as well as the
|
2014-05-06 18:12:18 +02:00
|
|
|
* required subsidiary data about parameters and such. All passed values will
|
2011-09-16 06:42:53 +02:00
|
|
|
* be copied into the CachedPlanSource's memory, except as specified below.
|
|
|
|
* After this is called, GetCachedPlan can be called to obtain a plan, and
|
|
|
|
* optionally the CachedPlanSource can be saved using SaveCachedPlan.
|
|
|
|
*
|
|
|
|
* If querytree_context is not NULL, the querytree_list must be stored in that
|
2014-05-06 18:12:18 +02:00
|
|
|
* context (but the other parameters need not be). The querytree_list is not
|
2011-09-16 06:42:53 +02:00
|
|
|
* copied, rather the given context is kept as the initial query_context of
|
|
|
|
* the CachedPlanSource. (It should have been created as a child of the
|
|
|
|
* caller's working memory context, but it will now be reparented to belong
|
|
|
|
* to the CachedPlanSource.) The querytree_context is normally the context in
|
|
|
|
* which the caller did raw parsing and parse analysis. This approach saves
|
|
|
|
* one tree copying step compared to passing NULL, but leaves lots of extra
|
|
|
|
* cruft in the query_context, namely whatever extraneous stuff parse analysis
|
|
|
|
* created, as well as whatever went unused from the raw parse tree. Using
|
|
|
|
* this option is a space-for-time tradeoff that is appropriate if the
|
|
|
|
* CachedPlanSource is not expected to survive long.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* plancache.c cannot know how to copy the data referenced by parserSetupArg,
|
|
|
|
* and it would often be inappropriate to do so anyway. When using that
|
|
|
|
* option, it is caller's responsibility that the referenced data remains
|
|
|
|
* valid for as long as the CachedPlanSource exists.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* If the CachedPlanSource is a "oneshot" plan, then no querytree copying
|
|
|
|
* occurs at all, and querytree_context is ignored; it is caller's
|
|
|
|
* responsibility that the passed querytree_list is sufficiently long-lived.
|
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* plansource: structure returned by CreateCachedPlan
|
|
|
|
* querytree_list: analyzed-and-rewritten form of query (list of Query nodes)
|
|
|
|
* querytree_context: memory context containing querytree_list,
|
|
|
|
* or NULL to copy querytree_list into a fresh context
|
|
|
|
* param_types: array of fixed parameter type OIDs, or NULL if none
|
|
|
|
* num_params: number of fixed parameters
|
|
|
|
* parserSetup: alternate method for handling query parameters
|
|
|
|
* parserSetupArg: data to pass to parserSetup
|
|
|
|
* cursor_options: options bitmask to pass to planner
|
|
|
|
* fixed_result: TRUE to disallow future changes in query's result tupdesc
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
void
|
|
|
|
CompleteCachedPlan(CachedPlanSource *plansource,
|
|
|
|
List *querytree_list,
|
|
|
|
MemoryContext querytree_context,
|
|
|
|
Oid *param_types,
|
|
|
|
int num_params,
|
|
|
|
ParserSetupHook parserSetup,
|
|
|
|
void *parserSetupArg,
|
|
|
|
int cursor_options,
|
|
|
|
bool fixed_result)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContext source_context = plansource->context;
|
|
|
|
MemoryContext oldcxt = CurrentMemoryContext;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Assert caller is doing things in a sane order */
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(!plansource->is_complete);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If caller supplied a querytree_context, reparent it underneath the
|
|
|
|
* CachedPlanSource's context; otherwise, create a suitable context and
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* copy the querytree_list into it. But no data copying should be done
|
|
|
|
* for one-shot plans; for those, assume the passed querytree_list is
|
|
|
|
* sufficiently long-lived.
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
if (plansource->is_oneshot)
|
|
|
|
{
|
|
|
|
querytree_context = CurrentMemoryContext;
|
|
|
|
}
|
|
|
|
else if (querytree_context != NULL)
|
2011-09-16 06:42:53 +02:00
|
|
|
{
|
|
|
|
MemoryContextSetParent(querytree_context, source_context);
|
|
|
|
MemoryContextSwitchTo(querytree_context);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* Again, it's a good bet the querytree_context can be small */
|
|
|
|
querytree_context = AllocSetContextCreate(source_context,
|
|
|
|
"CachedPlanQuery",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
|
|
|
MemoryContextSwitchTo(querytree_context);
|
|
|
|
querytree_list = (List *) copyObject(querytree_list);
|
|
|
|
}
|
|
|
|
|
|
|
|
plansource->query_context = querytree_context;
|
|
|
|
plansource->query_list = querytree_list;
|
Adjust things so that the query_string of a cached plan and the sourceText of
a portal are never NULL, but reliably provide the source text of the query.
It turns out that there was only one place that was really taking a short-cut,
which was the 'EXECUTE' utility statement. That doesn't seem like a
sufficiently critical performance hotspot to justify not offering a guarantee
of validity of the portal source text. Fix it to copy the source text over
from the cached plan. Add Asserts in the places that set up cached plans and
portals to reject null source strings, and simplify a bunch of places that
formerly needed to guard against nulls.
There may be a few places that cons up statements for execution without
having any source text at all; I found one such in ConvertTriggerToFK().
It seems sufficient to inject a phony source string in such a case,
for instance
ProcessUtility((Node *) atstmt,
"(generated ALTER TABLE ADD FOREIGN KEY command)",
NULL, false, None_Receiver, NULL);
We should take a second look at the usage of debug_query_string,
particularly the recently added current_query() SQL function.
ITAGAKI Takahiro and Tom Lane
2008-07-18 22:26:06 +02:00
|
|
|
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
if (!plansource->is_oneshot && !IsTransactionStmtPlan(plansource))
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Use the planner machinery to extract dependencies. Data is saved
|
|
|
|
* in query_context. (We assume that not a lot of extra cruft is
|
|
|
|
* created by this call.) We can skip this for one-shot plans, and
|
|
|
|
* transaction control commands have no such dependencies anyway.
|
|
|
|
*/
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
extract_query_dependencies((Node *) querytree_list,
|
|
|
|
&plansource->relationOids,
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
&plansource->invalItems,
|
Rename pg_rowsecurity -> pg_policy and other fixes
As pointed out by Robert, we should really have named pg_rowsecurity
pg_policy, as the objects stored in that catalog are policies. This
patch fixes that and updates the column names to start with 'pol' to
match the new catalog name.
The security consideration for COPY with row level security, also
pointed out by Robert, has also been addressed by remembering and
re-checking the OID of the relation initially referenced during COPY
processing, to make sure it hasn't changed under us by the time we
finish planning out the query which has been built.
Robert and Alvaro also commented on missing OCLASS and OBJECT entries
for POLICY (formerly ROWSECURITY or POLICY, depending) in various
places. This patch fixes that too, which also happens to add the
ability to COMMENT on policies.
In passing, attempt to improve the consistency of messages, comments,
and documentation as well. This removes various incarnations of
'row-security', 'row-level security', 'Row-security', etc, in favor
of 'policy', 'row level security' or 'row_security' as appropriate.
Happy Thanksgiving!
2014-11-27 07:06:36 +01:00
|
|
|
&plansource->hasRowSecurity);
|
2007-03-23 20:53:52 +01:00
|
|
|
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
/*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Also save the current search_path in the query_context. (This
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
* should not generate much extra cruft either, since almost certainly
|
2013-05-29 22:58:43 +02:00
|
|
|
* the path is already valid.) Again, we don't really need this for
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
* one-shot plans; and we *must* skip this for transaction control
|
|
|
|
* commands, because this could result in catalog accesses.
|
|
|
|
*/
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
plansource->search_path = GetOverrideSearchPath(querytree_context);
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
}
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Save the final parameter types (or other parameter specification data)
|
|
|
|
* into the source_context, as well as our other parameters. Also save
|
|
|
|
* the result tuple descriptor.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContextSwitchTo(source_context);
|
|
|
|
|
|
|
|
if (num_params > 0)
|
|
|
|
{
|
|
|
|
plansource->param_types = (Oid *) palloc(num_params * sizeof(Oid));
|
|
|
|
memcpy(plansource->param_types, param_types, num_params * sizeof(Oid));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
plansource->param_types = NULL;
|
2007-03-13 01:33:44 +01:00
|
|
|
plansource->num_params = num_params;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->parserSetup = parserSetup;
|
|
|
|
plansource->parserSetupArg = parserSetupArg;
|
2007-04-16 20:21:07 +02:00
|
|
|
plansource->cursor_options = cursor_options;
|
2007-03-13 01:33:44 +01:00
|
|
|
plansource->fixed_result = fixed_result;
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->resultDesc = PlanCacheComputeResultDesc(querytree_list);
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
plansource->planUserId = GetUserId();
|
|
|
|
plansource->row_security_env = row_security;
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
MemoryContextSwitchTo(oldcxt);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->is_complete = true;
|
|
|
|
plansource->is_valid = true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* SaveCachedPlan: save a cached plan permanently
|
|
|
|
*
|
|
|
|
* This function moves the cached plan underneath CacheMemoryContext (making
|
|
|
|
* it live for the life of the backend, unless explicitly dropped), and adds
|
|
|
|
* it to the list of cached plans that are checked for invalidation when an
|
|
|
|
* sinval event occurs.
|
|
|
|
*
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* This is guaranteed not to throw error, except for the caller-error case
|
|
|
|
* of trying to save a one-shot plan. Callers typically depend on that
|
2011-09-16 06:42:53 +02:00
|
|
|
* since this is called just before or just after adding a pointer to the
|
2014-05-06 18:12:18 +02:00
|
|
|
* CachedPlanSource to some permanent data structure of their own. Up until
|
2011-09-16 06:42:53 +02:00
|
|
|
* this is done, a CachedPlanSource is just transient data that will go away
|
|
|
|
* automatically on transaction abort.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
SaveCachedPlan(CachedPlanSource *plansource)
|
|
|
|
{
|
|
|
|
/* Assert caller is doing things in a sane order */
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(plansource->is_complete);
|
|
|
|
Assert(!plansource->is_saved);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/* This seems worth a real test, though */
|
|
|
|
if (plansource->is_oneshot)
|
|
|
|
elog(ERROR, "cannot save one-shot cached plan");
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* In typical use, this function would be called before generating any
|
2012-06-10 21:20:04 +02:00
|
|
|
* plans from the CachedPlanSource. If there is a generic plan, moving it
|
|
|
|
* into CacheMemoryContext would be pretty risky since it's unclear
|
2011-09-16 06:42:53 +02:00
|
|
|
* whether the caller has taken suitable care with making references
|
2014-05-06 18:12:18 +02:00
|
|
|
* long-lived. Best thing to do seems to be to discard the plan.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
ReleaseGenericPlan(plansource);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
/*
|
2012-06-10 21:20:04 +02:00
|
|
|
* Reparent the source memory context under CacheMemoryContext so that it
|
2014-05-06 18:12:18 +02:00
|
|
|
* will live indefinitely. The query_context follows along since it's
|
2011-09-16 06:42:53 +02:00
|
|
|
* already a child of the other one.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContextSetParent(plansource->context, CacheMemoryContext);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* Add the entry to the global list of cached plans.
|
|
|
|
*/
|
|
|
|
plansource->next_saved = first_saved_plan;
|
|
|
|
first_saved_plan = plansource;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->is_saved = true;
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
2009-11-04 23:26:08 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* DropCachedPlan: destroy a cached plan.
|
2009-11-04 23:26:08 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Actually this only destroys the CachedPlanSource: any referenced CachedPlan
|
2014-05-06 18:12:18 +02:00
|
|
|
* is released, but not destroyed until its refcount goes to zero. That
|
2011-09-16 06:42:53 +02:00
|
|
|
* handles the situation where DropCachedPlan is called while the plan is
|
|
|
|
* still in use.
|
2009-11-04 23:26:08 +01:00
|
|
|
*/
|
|
|
|
void
|
2011-09-16 06:42:53 +02:00
|
|
|
DropCachedPlan(CachedPlanSource *plansource)
|
2009-11-04 23:26:08 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
|
|
|
|
/* If it's been saved, remove it from the list */
|
|
|
|
if (plansource->is_saved)
|
|
|
|
{
|
|
|
|
if (first_saved_plan == plansource)
|
|
|
|
first_saved_plan = plansource->next_saved;
|
|
|
|
else
|
|
|
|
{
|
|
|
|
CachedPlanSource *psrc;
|
|
|
|
|
|
|
|
for (psrc = first_saved_plan; psrc; psrc = psrc->next_saved)
|
|
|
|
{
|
|
|
|
if (psrc->next_saved == plansource)
|
|
|
|
{
|
|
|
|
psrc->next_saved = plansource->next_saved;
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
plansource->is_saved = false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/* Decrement generic CachePlan's refcount and drop if no longer needed */
|
|
|
|
ReleaseGenericPlan(plansource);
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/* Mark it no longer valid */
|
|
|
|
plansource->magic = 0;
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* Remove the CachedPlanSource and all subsidiary data (including the
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* query_context if any). But if it's a one-shot we can't free anything.
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
if (!plansource->is_oneshot)
|
|
|
|
MemoryContextDelete(plansource->context);
|
2009-11-04 23:26:08 +01:00
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* ReleaseGenericPlan: release a CachedPlanSource's generic plan, if any.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
static void
|
2011-09-16 06:42:53 +02:00
|
|
|
ReleaseGenericPlan(CachedPlanSource *plansource)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Be paranoid about the possibility that ReleaseCachedPlan fails */
|
|
|
|
if (plansource->gplan)
|
|
|
|
{
|
|
|
|
CachedPlan *plan = plansource->gplan;
|
|
|
|
|
|
|
|
Assert(plan->magic == CACHEDPLAN_MAGIC);
|
|
|
|
plansource->gplan = NULL;
|
|
|
|
ReleaseCachedPlan(plan, false);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* RevalidateCachedQuery: ensure validity of analyzed-and-rewritten query tree.
|
|
|
|
*
|
|
|
|
* What we do here is re-acquire locks and redo parse analysis if necessary.
|
|
|
|
* On return, the query_list is valid and we have sufficient locks to begin
|
|
|
|
* planning.
|
|
|
|
*
|
|
|
|
* If any parse analysis activity is required, the caller's memory context is
|
|
|
|
* used for that work.
|
|
|
|
*
|
|
|
|
* The result value is the transient analyzed-and-rewritten query tree if we
|
|
|
|
* had to do re-analysis, and NIL otherwise. (This is returned just to save
|
|
|
|
* a tree copying step in a subsequent BuildCachedPlan call.)
|
|
|
|
*/
|
|
|
|
static List *
|
|
|
|
RevalidateCachedQuery(CachedPlanSource *plansource)
|
|
|
|
{
|
|
|
|
bool snapshot_set;
|
|
|
|
Node *rawtree;
|
|
|
|
List *tlist; /* transient query-tree list */
|
|
|
|
List *qlist; /* permanent query-tree list */
|
|
|
|
TupleDesc resultDesc;
|
|
|
|
MemoryContext querytree_context;
|
2007-03-13 01:33:44 +01:00
|
|
|
MemoryContext oldcxt;
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/*
|
|
|
|
* For one-shot plans, we do not support revalidation checking; it's
|
|
|
|
* assumed the query is parsed, planned, and executed in one transaction,
|
2013-05-29 22:58:43 +02:00
|
|
|
* so that no lock re-acquisition is necessary. Also, there is never any
|
|
|
|
* need to revalidate plans for transaction control commands (and we
|
|
|
|
* mustn't risk any catalog accesses when handling those).
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
*/
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
if (plansource->is_oneshot || IsTransactionStmtPlan(plansource))
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
{
|
|
|
|
Assert(plansource->is_valid);
|
|
|
|
return NIL;
|
|
|
|
}
|
|
|
|
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
/*
|
|
|
|
* If the query is currently valid, we should have a saved search_path ---
|
|
|
|
* check to see if that matches the current environment. If not, we want
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
* to force replan. We should also have a valid planUserId.
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
*/
|
|
|
|
if (plansource->is_valid)
|
|
|
|
{
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
Assert(OidIsValid(plansource->planUserId));
|
|
|
|
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
Assert(plansource->search_path != NULL);
|
|
|
|
if (!OverrideSearchPathMatchesCurrent(plansource->search_path))
|
|
|
|
{
|
|
|
|
/* Invalidate the querytree and generic plan */
|
|
|
|
plansource->is_valid = false;
|
|
|
|
if (plansource->gplan)
|
|
|
|
plansource->gplan->is_valid = false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
/*
|
2015-09-21 02:47:17 +02:00
|
|
|
* If the plan has a possible RLS dependency, force a replan if either the
|
|
|
|
* role or the row_security setting has changed.
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
*/
|
|
|
|
if (plansource->is_valid
|
Rename pg_rowsecurity -> pg_policy and other fixes
As pointed out by Robert, we should really have named pg_rowsecurity
pg_policy, as the objects stored in that catalog are policies. This
patch fixes that and updates the column names to start with 'pol' to
match the new catalog name.
The security consideration for COPY with row level security, also
pointed out by Robert, has also been addressed by remembering and
re-checking the OID of the relation initially referenced during COPY
processing, to make sure it hasn't changed under us by the time we
finish planning out the query which has been built.
Robert and Alvaro also commented on missing OCLASS and OBJECT entries
for POLICY (formerly ROWSECURITY or POLICY, depending) in various
places. This patch fixes that too, which also happens to add the
ability to COMMENT on policies.
In passing, attempt to improve the consistency of messages, comments,
and documentation as well. This removes various incarnations of
'row-security', 'row-level security', 'Row-security', etc, in favor
of 'policy', 'row level security' or 'row_security' as appropriate.
Happy Thanksgiving!
2014-11-27 07:06:36 +01:00
|
|
|
&& plansource->hasRowSecurity
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
&& (plansource->planUserId != GetUserId()
|
|
|
|
|| plansource->row_security_env != row_security))
|
|
|
|
plansource->is_valid = false;
|
|
|
|
|
2016-01-28 20:05:36 +01:00
|
|
|
/*
|
2016-06-10 00:02:36 +02:00
|
|
|
* If we have a join pushed down to the foreign server and the current
|
|
|
|
* user is different from the one for which the plan was created,
|
|
|
|
* invalidate the generic plan since user mapping for the new user might
|
|
|
|
* make the join unsafe to push down, or change which user mapping is
|
|
|
|
* used.
|
2016-01-28 20:05:36 +01:00
|
|
|
*/
|
|
|
|
if (plansource->is_valid &&
|
|
|
|
plansource->gplan &&
|
|
|
|
plansource->gplan->has_foreign_join &&
|
|
|
|
plansource->planUserId != GetUserId())
|
|
|
|
plansource->gplan->is_valid = false;
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* If the query is currently valid, acquire locks on the referenced
|
|
|
|
* objects; then check again. We need to do it this way to cover the race
|
|
|
|
* condition that an invalidation message arrives before we get the locks.
|
|
|
|
*/
|
|
|
|
if (plansource->is_valid)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
AcquirePlannerLocks(plansource->query_list, true);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* By now, if any invalidation has happened, the inval callback
|
|
|
|
* functions will have marked the query invalid.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
if (plansource->is_valid)
|
|
|
|
{
|
|
|
|
/* Successfully revalidated and locked the query. */
|
|
|
|
return NIL;
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Ooops, the race case happened. Release useless locks. */
|
|
|
|
AcquirePlannerLocks(plansource->query_list, false);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/*
|
2012-06-10 21:20:04 +02:00
|
|
|
* Discard the no-longer-useful query tree. (Note: we don't want to do
|
|
|
|
* this any earlier, else we'd not have been able to release locks
|
2011-09-16 06:42:53 +02:00
|
|
|
* correctly in the race condition case.)
|
|
|
|
*/
|
|
|
|
plansource->is_valid = false;
|
|
|
|
plansource->query_list = NIL;
|
|
|
|
plansource->relationOids = NIL;
|
|
|
|
plansource->invalItems = NIL;
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
plansource->search_path = NULL;
|
2011-09-16 06:42:53 +02:00
|
|
|
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
/*
|
|
|
|
* The plan is invalid, possibly due to row security, so we need to reset
|
|
|
|
* row_security_env and planUserId as we're about to re-plan with the
|
|
|
|
* current settings.
|
|
|
|
*/
|
|
|
|
plansource->row_security_env = row_security;
|
|
|
|
plansource->planUserId = GetUserId();
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Free the query_context. We don't really expect MemoryContextDelete to
|
2011-09-16 06:42:53 +02:00
|
|
|
* fail, but just in case, make sure the CachedPlanSource is left in a
|
2012-06-10 21:20:04 +02:00
|
|
|
* reasonably sane state. (The generic plan won't get unlinked yet, but
|
|
|
|
* that's acceptable.)
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
if (plansource->query_context)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2012-06-10 21:20:04 +02:00
|
|
|
MemoryContext qcxt = plansource->query_context;
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
plansource->query_context = NULL;
|
|
|
|
MemoryContextDelete(qcxt);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Drop the generic plan reference if any */
|
|
|
|
ReleaseGenericPlan(plansource);
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Now re-do parse analysis and rewrite. This not incidentally acquires
|
|
|
|
* the locks we need to do planning safely.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(plansource->is_complete);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If a snapshot is already set (the normal case), we can just use that
|
|
|
|
* for parsing/planning. But if it isn't, install one. Note: no point in
|
|
|
|
* checking whether parse analysis requires a snapshot; utility commands
|
|
|
|
* don't have invalidatable plans, so we'd not get here for such a
|
|
|
|
* command.
|
|
|
|
*/
|
|
|
|
snapshot_set = false;
|
|
|
|
if (!ActiveSnapshotSet())
|
2007-09-20 19:56:33 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
PushActiveSnapshot(GetTransactionSnapshot());
|
|
|
|
snapshot_set = true;
|
2007-09-20 19:56:33 +02:00
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Run parse analysis and rule rewriting. The parser tends to scribble on
|
|
|
|
* its input, so we must copy the raw parse tree to prevent corruption of
|
|
|
|
* the cache.
|
|
|
|
*/
|
|
|
|
rawtree = copyObject(plansource->raw_parse_tree);
|
2014-11-12 21:58:37 +01:00
|
|
|
if (rawtree == NULL)
|
|
|
|
tlist = NIL;
|
|
|
|
else if (plansource->parserSetup != NULL)
|
2011-09-16 06:42:53 +02:00
|
|
|
tlist = pg_analyze_and_rewrite_params(rawtree,
|
|
|
|
plansource->query_string,
|
|
|
|
plansource->parserSetup,
|
|
|
|
plansource->parserSetupArg);
|
2007-09-20 19:56:33 +02:00
|
|
|
else
|
2011-09-16 06:42:53 +02:00
|
|
|
tlist = pg_analyze_and_rewrite(rawtree,
|
|
|
|
plansource->query_string,
|
|
|
|
plansource->param_types,
|
|
|
|
plansource->num_params);
|
2010-01-15 23:36:35 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Release snapshot if we got one */
|
|
|
|
if (snapshot_set)
|
|
|
|
PopActiveSnapshot();
|
2010-01-15 23:36:35 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Check or update the result tupdesc. XXX should we use a weaker
|
2011-09-16 06:42:53 +02:00
|
|
|
* condition than equalTupleDescs() here?
|
|
|
|
*
|
|
|
|
* We assume the parameter types didn't change from the first time, so no
|
|
|
|
* need to update that.
|
|
|
|
*/
|
|
|
|
resultDesc = PlanCacheComputeResultDesc(tlist);
|
|
|
|
if (resultDesc == NULL && plansource->resultDesc == NULL)
|
|
|
|
{
|
|
|
|
/* OK, doesn't return tuples */
|
2008-09-09 20:58:09 +02:00
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
else if (resultDesc == NULL || plansource->resultDesc == NULL ||
|
|
|
|
!equalTupleDescs(resultDesc, plansource->resultDesc))
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* can we give a better error message? */
|
|
|
|
if (plansource->fixed_result)
|
|
|
|
ereport(ERROR,
|
|
|
|
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
|
|
|
|
errmsg("cached plan must not change result type")));
|
|
|
|
oldcxt = MemoryContextSwitchTo(plansource->context);
|
|
|
|
if (resultDesc)
|
|
|
|
resultDesc = CreateTupleDescCopy(resultDesc);
|
|
|
|
if (plansource->resultDesc)
|
|
|
|
FreeTupleDesc(plansource->resultDesc);
|
|
|
|
plansource->resultDesc = resultDesc;
|
|
|
|
MemoryContextSwitchTo(oldcxt);
|
2008-09-09 20:58:09 +02:00
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* Allocate new query_context and copy the completed querytree into it.
|
|
|
|
* It's transient until we complete the copying and dependency extraction.
|
|
|
|
*/
|
|
|
|
querytree_context = AllocSetContextCreate(CurrentMemoryContext,
|
|
|
|
"CachedPlanQuery",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
|
|
|
oldcxt = MemoryContextSwitchTo(querytree_context);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
qlist = (List *) copyObject(tlist);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* Use the planner machinery to extract dependencies. Data is saved in
|
2012-06-10 21:20:04 +02:00
|
|
|
* query_context. (We assume that not a lot of extra cruft is created by
|
|
|
|
* this call.)
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
extract_query_dependencies((Node *) qlist,
|
|
|
|
&plansource->relationOids,
|
Row-Level Security Policies (RLS)
Building on the updatable security-barrier views work, add the
ability to define policies on tables to limit the set of rows
which are returned from a query and which are allowed to be added
to a table. Expressions defined by the policy for filtering are
added to the security barrier quals of the query, while expressions
defined to check records being added to a table are added to the
with-check options of the query.
New top-level commands are CREATE/ALTER/DROP POLICY and are
controlled by the table owner. Row Security is able to be enabled
and disabled by the owner on a per-table basis using
ALTER TABLE .. ENABLE/DISABLE ROW SECURITY.
Per discussion, ROW SECURITY is disabled on tables by default and
must be enabled for policies on the table to be used. If no
policies exist on a table with ROW SECURITY enabled, a default-deny
policy is used and no records will be visible.
By default, row security is applied at all times except for the
table owner and the superuser. A new GUC, row_security, is added
which can be set to ON, OFF, or FORCE. When set to FORCE, row
security will be applied even for the table owner and superusers.
When set to OFF, row security will be disabled when allowed and an
error will be thrown if the user does not have rights to bypass row
security.
Per discussion, pg_dump sets row_security = OFF by default to ensure
that exports and backups will have all data in the table or will
error if there are insufficient privileges to bypass row security.
A new option has been added to pg_dump, --enable-row-security, to
ask pg_dump to export with row security enabled.
A new role capability, BYPASSRLS, which can only be set by the
superuser, is added to allow other users to be able to bypass row
security using row_security = OFF.
Many thanks to the various individuals who have helped with the
design, particularly Robert Haas for his feedback.
Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean
Rasheed, with additional changes and rework by me.
Reviewers have included all of the above, Greg Smith,
Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
|
|
|
&plansource->invalItems,
|
Rename pg_rowsecurity -> pg_policy and other fixes
As pointed out by Robert, we should really have named pg_rowsecurity
pg_policy, as the objects stored in that catalog are policies. This
patch fixes that and updates the column names to start with 'pol' to
match the new catalog name.
The security consideration for COPY with row level security, also
pointed out by Robert, has also been addressed by remembering and
re-checking the OID of the relation initially referenced during COPY
processing, to make sure it hasn't changed under us by the time we
finish planning out the query which has been built.
Robert and Alvaro also commented on missing OCLASS and OBJECT entries
for POLICY (formerly ROWSECURITY or POLICY, depending) in various
places. This patch fixes that too, which also happens to add the
ability to COMMENT on policies.
In passing, attempt to improve the consistency of messages, comments,
and documentation as well. This removes various incarnations of
'row-security', 'row-level security', 'Row-security', etc, in favor
of 'policy', 'row level security' or 'row_security' as appropriate.
Happy Thanksgiving!
2014-11-27 07:06:36 +01:00
|
|
|
&plansource->hasRowSecurity);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
/*
|
2014-05-06 18:12:18 +02:00
|
|
|
* Also save the current search_path in the query_context. (This should
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
* not generate much extra cruft either, since almost certainly the path
|
|
|
|
* is already valid.)
|
|
|
|
*/
|
|
|
|
plansource->search_path = GetOverrideSearchPath(querytree_context);
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContextSwitchTo(oldcxt);
|
|
|
|
|
|
|
|
/* Now reparent the finished query_context and save the links */
|
|
|
|
MemoryContextSetParent(querytree_context, plansource->context);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->query_context = querytree_context;
|
|
|
|
plansource->query_list = qlist;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
/*
|
2012-06-10 21:20:04 +02:00
|
|
|
* Note: we do not reset generic_cost or total_custom_cost, although we
|
|
|
|
* could choose to do so. If the DDL or statistics change that prompted
|
|
|
|
* the invalidation meant a significant change in the cost estimates, it
|
|
|
|
* would be better to reset those variables and start fresh; but often it
|
|
|
|
* doesn't, and we're better retaining our hard-won knowledge about the
|
|
|
|
* relative costs.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
plansource->is_valid = true;
|
|
|
|
|
|
|
|
/* Return transient copy of querytrees for possible use in planning */
|
|
|
|
return tlist;
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* CheckCachedPlan: see if the CachedPlanSource's generic plan is valid.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Caller must have already called RevalidateCachedQuery to verify that the
|
|
|
|
* querytree is up to date.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
2011-09-16 06:42:53 +02:00
|
|
|
* On a "true" return, we have acquired the locks needed to run the plan.
|
|
|
|
* (We must do this for the "true" result to be race-condition-free.)
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
static bool
|
|
|
|
CheckCachedPlan(CachedPlanSource *plansource)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
CachedPlan *plan = plansource->gplan;
|
|
|
|
|
|
|
|
/* Assert that caller checked the querytree */
|
|
|
|
Assert(plansource->is_valid);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* If there's no generic plan, just say "false" */
|
|
|
|
if (!plan)
|
|
|
|
return false;
|
|
|
|
|
|
|
|
Assert(plan->magic == CACHEDPLAN_MAGIC);
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/* Generic plans are never one-shot */
|
|
|
|
Assert(!plan->is_oneshot);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* If it appears valid, acquire locks and recheck; this is much the same
|
|
|
|
* logic as in RevalidateCachedQuery, but for a plan.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
if (plan->is_valid)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Plan must have positive refcount because it is referenced by
|
|
|
|
* plansource; so no need to fear it disappears under us here.
|
|
|
|
*/
|
|
|
|
Assert(plan->refcount > 0);
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
AcquireExecutorLocks(plan->stmt_list, true);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2007-09-20 19:56:33 +02:00
|
|
|
/*
|
|
|
|
* If plan was transient, check to see if TransactionXmin has
|
|
|
|
* advanced, and if so invalidate it.
|
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
if (plan->is_valid &&
|
2007-09-20 19:56:33 +02:00
|
|
|
TransactionIdIsValid(plan->saved_xmin) &&
|
|
|
|
!TransactionIdEquals(plan->saved_xmin, TransactionXmin))
|
2011-09-16 06:42:53 +02:00
|
|
|
plan->is_valid = false;
|
2007-09-20 19:56:33 +02:00
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* By now, if any invalidation has happened, the inval callback
|
2011-09-16 06:42:53 +02:00
|
|
|
* functions will have marked the plan invalid.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
if (plan->is_valid)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Successfully revalidated and locked the query. */
|
|
|
|
return true;
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/* Ooops, the race case happened. Release useless locks. */
|
|
|
|
AcquireExecutorLocks(plan->stmt_list, false);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Plan has been invalidated, so unlink it from the parent and release it.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
ReleaseGenericPlan(plansource);
|
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* BuildCachedPlan: construct a new CachedPlan from a CachedPlanSource.
|
|
|
|
*
|
2011-09-26 18:44:17 +02:00
|
|
|
* qlist should be the result value from a previous RevalidateCachedQuery,
|
|
|
|
* or it can be set to NIL if we need to re-copy the plansource's query_list.
|
2011-09-16 06:42:53 +02:00
|
|
|
*
|
|
|
|
* To build a generic, parameter-value-independent plan, pass NULL for
|
|
|
|
* boundParams. To build a custom plan, pass the actual parameter values via
|
|
|
|
* boundParams. For best effect, the PARAM_FLAG_CONST flag should be set on
|
|
|
|
* each parameter value; otherwise the planner will treat the value as a
|
|
|
|
* hint rather than a hard constant.
|
|
|
|
*
|
|
|
|
* Planning work is done in the caller's memory context. The finished plan
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* is in a child memory context, which typically should get reparented
|
|
|
|
* (unless this is a one-shot plan, in which case we don't copy the plan).
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
static CachedPlan *
|
|
|
|
BuildCachedPlan(CachedPlanSource *plansource, List *qlist,
|
|
|
|
ParamListInfo boundParams)
|
|
|
|
{
|
|
|
|
CachedPlan *plan;
|
|
|
|
List *plist;
|
|
|
|
bool snapshot_set;
|
|
|
|
bool spi_pushed;
|
|
|
|
MemoryContext plan_context;
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
MemoryContext oldcxt = CurrentMemoryContext;
|
2016-06-10 00:02:36 +02:00
|
|
|
ListCell *lc;
|
2011-09-16 06:42:53 +02:00
|
|
|
|
2011-09-17 07:47:33 +02:00
|
|
|
/*
|
|
|
|
* Normally the querytree should be valid already, but if it's not,
|
|
|
|
* rebuild it.
|
|
|
|
*
|
|
|
|
* NOTE: GetCachedPlan should have called RevalidateCachedQuery first, so
|
|
|
|
* we ought to be holding sufficient locks to prevent any invalidation.
|
|
|
|
* However, if we're building a custom plan after having built and
|
|
|
|
* rejected a generic plan, it's possible to reach here with is_valid
|
2014-05-06 18:12:18 +02:00
|
|
|
* false due to an invalidation while making the generic plan. In theory
|
2011-09-17 07:47:33 +02:00
|
|
|
* the invalidation must be a false positive, perhaps a consequence of an
|
|
|
|
* sinval reset event or the CLOBBER_CACHE_ALWAYS debug code. But for
|
|
|
|
* safety, let's treat it as real and redo the RevalidateCachedQuery call.
|
|
|
|
*/
|
|
|
|
if (!plansource->is_valid)
|
|
|
|
qlist = RevalidateCachedQuery(plansource);
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If we don't already have a copy of the querytree list that can be
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* scribbled on by the planner, make one. For a one-shot plan, we assume
|
|
|
|
* it's okay to scribble on the original query_list.
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
if (qlist == NIL)
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
{
|
|
|
|
if (!plansource->is_oneshot)
|
|
|
|
qlist = (List *) copyObject(plansource->query_list);
|
|
|
|
else
|
|
|
|
qlist = plansource->query_list;
|
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/*
|
2011-09-25 23:33:32 +02:00
|
|
|
* If a snapshot is already set (the normal case), we can just use that
|
|
|
|
* for planning. But if it isn't, and we need one, install one.
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
snapshot_set = false;
|
2011-09-25 23:33:32 +02:00
|
|
|
if (!ActiveSnapshotSet() &&
|
2014-11-12 21:58:37 +01:00
|
|
|
plansource->raw_parse_tree &&
|
2011-09-25 23:33:32 +02:00
|
|
|
analyze_requires_snapshot(plansource->raw_parse_tree))
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
PushActiveSnapshot(GetTransactionSnapshot());
|
|
|
|
snapshot_set = true;
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2012-06-10 21:20:04 +02:00
|
|
|
* The planner may try to call SPI-using functions, which causes a problem
|
|
|
|
* if we're already inside one. Rather than expect all SPI-using code to
|
|
|
|
* do SPI_push whenever a replan could happen, it seems best to take care
|
|
|
|
* of the case here.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
spi_pushed = SPI_push_conditional();
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Generate the plan.
|
|
|
|
*/
|
|
|
|
plist = pg_plan_queries(qlist, plansource->cursor_options, boundParams);
|
|
|
|
|
|
|
|
/* Clean up SPI state */
|
|
|
|
SPI_pop_conditional(spi_pushed);
|
|
|
|
|
|
|
|
/* Release snapshot if we got one */
|
|
|
|
if (snapshot_set)
|
|
|
|
PopActiveSnapshot();
|
|
|
|
|
|
|
|
/*
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
* Normally we make a dedicated memory context for the CachedPlan and its
|
|
|
|
* subsidiary data. (It's probably not going to be large, but just in
|
|
|
|
* case, use the default maxsize parameter. It's transient for the
|
|
|
|
* moment.) But for a one-shot plan, we just leave it in the caller's
|
|
|
|
* memory context.
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
if (!plansource->is_oneshot)
|
|
|
|
{
|
|
|
|
plan_context = AllocSetContextCreate(CurrentMemoryContext,
|
|
|
|
"CachedPlan",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
2011-09-16 06:42:53 +02:00
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/*
|
|
|
|
* Copy plan into the new context.
|
|
|
|
*/
|
|
|
|
MemoryContextSwitchTo(plan_context);
|
2011-09-16 06:42:53 +02:00
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
plist = (List *) copyObject(plist);
|
|
|
|
}
|
|
|
|
else
|
|
|
|
plan_context = CurrentMemoryContext;
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Create and fill the CachedPlan struct within the new context.
|
|
|
|
*/
|
|
|
|
plan = (CachedPlan *) palloc(sizeof(CachedPlan));
|
|
|
|
plan->magic = CACHEDPLAN_MAGIC;
|
|
|
|
plan->stmt_list = plist;
|
|
|
|
if (plan_list_is_transient(plist))
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(TransactionIdIsNormal(TransactionXmin));
|
|
|
|
plan->saved_xmin = TransactionXmin;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
plan->saved_xmin = InvalidTransactionId;
|
|
|
|
plan->refcount = 0;
|
|
|
|
plan->context = plan_context;
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
plan->is_oneshot = plansource->is_oneshot;
|
2011-09-16 06:42:53 +02:00
|
|
|
plan->is_saved = false;
|
|
|
|
plan->is_valid = true;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2016-01-28 20:05:36 +01:00
|
|
|
/*
|
|
|
|
* Walk through the plist and set hasForeignJoin if any of the plans have
|
|
|
|
* it set.
|
|
|
|
*/
|
|
|
|
plan->has_foreign_join = false;
|
|
|
|
foreach(lc, plist)
|
|
|
|
{
|
2016-06-10 00:02:36 +02:00
|
|
|
PlannedStmt *plan_stmt = (PlannedStmt *) lfirst(lc);
|
2016-01-28 20:05:36 +01:00
|
|
|
|
|
|
|
if (IsA(plan_stmt, PlannedStmt))
|
|
|
|
plan->has_foreign_join =
|
|
|
|
plan->has_foreign_join || plan_stmt->hasForeignJoin;
|
|
|
|
}
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* assign generation number to new plan */
|
|
|
|
plan->generation = ++(plansource->generation);
|
2007-03-23 20:53:52 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContextSwitchTo(oldcxt);
|
2008-12-13 03:00:20 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
return plan;
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* choose_custom_plan: choose whether to use custom or generic plan
|
|
|
|
*
|
|
|
|
* This defines the policy followed by GetCachedPlan.
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
choose_custom_plan(CachedPlanSource *plansource, ParamListInfo boundParams)
|
|
|
|
{
|
|
|
|
double avg_custom_cost;
|
2009-07-14 17:37:50 +02:00
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/* One-shot plans will always be considered custom */
|
|
|
|
if (plansource->is_oneshot)
|
|
|
|
return true;
|
|
|
|
|
|
|
|
/* Otherwise, never any point in a custom plan if there's no parameters */
|
2011-09-16 06:42:53 +02:00
|
|
|
if (boundParams == NULL)
|
|
|
|
return false;
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
/* ... nor for transaction control statements */
|
|
|
|
if (IsTransactionStmtPlan(plansource))
|
|
|
|
return false;
|
2009-07-14 17:37:50 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* See if caller wants to force the decision */
|
|
|
|
if (plansource->cursor_options & CURSOR_OPT_GENERIC_PLAN)
|
|
|
|
return false;
|
|
|
|
if (plansource->cursor_options & CURSOR_OPT_CUSTOM_PLAN)
|
|
|
|
return true;
|
2009-07-14 17:37:50 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Generate custom plans until we have done at least 5 (arbitrary) */
|
|
|
|
if (plansource->num_custom_plans < 5)
|
|
|
|
return true;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
avg_custom_cost = plansource->total_custom_cost / plansource->num_custom_plans;
|
|
|
|
|
|
|
|
/*
|
2013-08-24 21:14:17 +02:00
|
|
|
* Prefer generic plan if it's less expensive than the average custom
|
|
|
|
* plan. (Because we include a charge for cost of planning in the
|
|
|
|
* custom-plan costs, this means the generic plan only has to be less
|
|
|
|
* expensive than the execution cost plus replan cost of the custom
|
|
|
|
* plans.)
|
2011-09-16 06:42:53 +02:00
|
|
|
*
|
|
|
|
* Note that if generic_cost is -1 (indicating we've not yet determined
|
|
|
|
* the generic plan cost), we'll always prefer generic at this point.
|
|
|
|
*/
|
2013-08-24 21:14:17 +02:00
|
|
|
if (plansource->generic_cost < avg_custom_cost)
|
2011-09-16 06:42:53 +02:00
|
|
|
return false;
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* cached_plan_cost: calculate estimated cost of a plan
|
2013-08-24 21:14:17 +02:00
|
|
|
*
|
|
|
|
* If include_planner is true, also include the estimated cost of constructing
|
|
|
|
* the plan. (We must factor that into the cost of using a custom plan, but
|
|
|
|
* we don't count it for a generic plan.)
|
2011-09-16 06:42:53 +02:00
|
|
|
*/
|
|
|
|
static double
|
2013-08-24 21:14:17 +02:00
|
|
|
cached_plan_cost(CachedPlan *plan, bool include_planner)
|
2011-09-16 06:42:53 +02:00
|
|
|
{
|
|
|
|
double result = 0;
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
foreach(lc, plan->stmt_list)
|
|
|
|
{
|
|
|
|
PlannedStmt *plannedstmt = (PlannedStmt *) lfirst(lc);
|
|
|
|
|
|
|
|
if (!IsA(plannedstmt, PlannedStmt))
|
|
|
|
continue; /* Ignore utility statements */
|
|
|
|
|
|
|
|
result += plannedstmt->planTree->total_cost;
|
2013-08-24 21:14:17 +02:00
|
|
|
|
|
|
|
if (include_planner)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Currently we use a very crude estimate of planning effort based
|
|
|
|
* on the number of relations in the finished plan's rangetable.
|
|
|
|
* Join planning effort actually scales much worse than linearly
|
|
|
|
* in the number of relations --- but only until the join collapse
|
2014-05-06 18:12:18 +02:00
|
|
|
* limits kick in. Also, while inheritance child relations surely
|
2013-08-24 21:14:17 +02:00
|
|
|
* add to planning effort, they don't make the join situation
|
|
|
|
* worse. So the actual shape of the planning cost curve versus
|
|
|
|
* number of relations isn't all that obvious. It will take
|
|
|
|
* considerable work to arrive at a less crude estimate, and for
|
|
|
|
* now it's not clear that's worth doing.
|
|
|
|
*
|
|
|
|
* The other big difficulty here is that we don't have any very
|
|
|
|
* good model of how planning cost compares to execution costs.
|
|
|
|
* The current multiplier of 1000 * cpu_operator_cost is probably
|
|
|
|
* on the low side, but we'll try this for awhile before making a
|
|
|
|
* more aggressive correction.
|
|
|
|
*
|
|
|
|
* If we ever do write a more complicated estimator, it should
|
|
|
|
* probably live in src/backend/optimizer/ not here.
|
|
|
|
*/
|
|
|
|
int nrelations = list_length(plannedstmt->rtable);
|
|
|
|
|
|
|
|
result += 1000.0 * cpu_operator_cost * (nrelations + 1);
|
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
return result;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* GetCachedPlan: get a cached plan from a CachedPlanSource.
|
|
|
|
*
|
|
|
|
* This function hides the logic that decides whether to use a generic
|
|
|
|
* plan or a custom plan for the given parameters: the caller does not know
|
|
|
|
* which it will get.
|
|
|
|
*
|
|
|
|
* On return, the plan is valid and we have sufficient locks to begin
|
|
|
|
* execution.
|
|
|
|
*
|
|
|
|
* On return, the refcount of the plan has been incremented; a later
|
|
|
|
* ReleaseCachedPlan() call is expected. The refcount has been reported
|
|
|
|
* to the CurrentResourceOwner if useResOwner is true (note that that must
|
|
|
|
* only be true if it's a "saved" CachedPlanSource).
|
|
|
|
*
|
|
|
|
* Note: if any replanning activity is required, the caller's memory context
|
|
|
|
* is used for that work.
|
|
|
|
*/
|
|
|
|
CachedPlan *
|
|
|
|
GetCachedPlan(CachedPlanSource *plansource, ParamListInfo boundParams,
|
|
|
|
bool useResOwner)
|
|
|
|
{
|
|
|
|
CachedPlan *plan;
|
|
|
|
List *qlist;
|
|
|
|
bool customplan;
|
|
|
|
|
|
|
|
/* Assert caller is doing things in a sane order */
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(plansource->is_complete);
|
|
|
|
/* This seems worth a real test, though */
|
|
|
|
if (useResOwner && !plansource->is_saved)
|
|
|
|
elog(ERROR, "cannot apply ResourceOwner to non-saved cached plan");
|
|
|
|
|
|
|
|
/* Make sure the querytree list is valid and we have parse-time locks */
|
|
|
|
qlist = RevalidateCachedQuery(plansource);
|
|
|
|
|
|
|
|
/* Decide whether to use a custom plan */
|
|
|
|
customplan = choose_custom_plan(plansource, boundParams);
|
|
|
|
|
|
|
|
if (!customplan)
|
|
|
|
{
|
|
|
|
if (CheckCachedPlan(plansource))
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* We want a generic plan, and we already have a valid one */
|
|
|
|
plan = plansource->gplan;
|
|
|
|
Assert(plan->magic == CACHEDPLAN_MAGIC);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
else
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Build a new generic plan */
|
|
|
|
plan = BuildCachedPlan(plansource, qlist, NULL);
|
|
|
|
/* Just make real sure plansource->gplan is clear */
|
|
|
|
ReleaseGenericPlan(plansource);
|
|
|
|
/* Link the new generic plan into the plansource */
|
|
|
|
plansource->gplan = plan;
|
|
|
|
plan->refcount++;
|
|
|
|
/* Immediately reparent into appropriate context */
|
|
|
|
if (plansource->is_saved)
|
|
|
|
{
|
|
|
|
/* saved plans all live under CacheMemoryContext */
|
|
|
|
MemoryContextSetParent(plan->context, CacheMemoryContext);
|
|
|
|
plan->is_saved = true;
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
|
|
|
/* otherwise, it should be a sibling of the plansource */
|
|
|
|
MemoryContextSetParent(plan->context,
|
2012-06-10 21:20:04 +02:00
|
|
|
MemoryContextGetParent(plansource->context));
|
2011-09-16 06:42:53 +02:00
|
|
|
}
|
|
|
|
/* Update generic_cost whenever we make a new generic plan */
|
2013-08-24 21:14:17 +02:00
|
|
|
plansource->generic_cost = cached_plan_cost(plan, false);
|
2007-03-23 20:53:52 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* If, based on the now-known value of generic_cost, we'd not have
|
|
|
|
* chosen to use a generic plan, then forget it and make a custom
|
|
|
|
* plan. This is a bit of a wart but is necessary to avoid a
|
|
|
|
* glitch in behavior when the custom plans are consistently big
|
|
|
|
* winners; at some point we'll experiment with a generic plan and
|
|
|
|
* find it's a loser, but we don't want to actually execute that
|
|
|
|
* plan.
|
|
|
|
*/
|
|
|
|
customplan = choose_custom_plan(plansource, boundParams);
|
2011-09-26 18:44:17 +02:00
|
|
|
|
|
|
|
/*
|
|
|
|
* If we choose to plan again, we need to re-copy the query_list,
|
2014-05-06 18:12:18 +02:00
|
|
|
* since the planner probably scribbled on it. We can force
|
2011-09-26 18:44:17 +02:00
|
|
|
* BuildCachedPlan to do that by passing NIL.
|
|
|
|
*/
|
|
|
|
qlist = NIL;
|
2011-09-16 06:42:53 +02:00
|
|
|
}
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
if (customplan)
|
|
|
|
{
|
|
|
|
/* Build a custom plan */
|
|
|
|
plan = BuildCachedPlan(plansource, qlist, boundParams);
|
|
|
|
/* Accumulate total costs of custom plans, but 'ware overflow */
|
|
|
|
if (plansource->num_custom_plans < INT_MAX)
|
|
|
|
{
|
2013-08-24 21:14:17 +02:00
|
|
|
plansource->total_custom_cost += cached_plan_cost(plan, true);
|
2011-09-16 06:42:53 +02:00
|
|
|
plansource->num_custom_plans++;
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Flag the plan as in use by caller */
|
2007-03-13 01:33:44 +01:00
|
|
|
if (useResOwner)
|
|
|
|
ResourceOwnerEnlargePlanCacheRefs(CurrentResourceOwner);
|
|
|
|
plan->refcount++;
|
|
|
|
if (useResOwner)
|
|
|
|
ResourceOwnerRememberPlanCacheRef(CurrentResourceOwner, plan);
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* Saved plans should be under CacheMemoryContext so they will not go away
|
|
|
|
* until their reference count goes to zero. In the generic-plan cases we
|
|
|
|
* already took care of that, but for a custom plan, do it as soon as we
|
|
|
|
* have created a reference-counted link.
|
|
|
|
*/
|
|
|
|
if (customplan && plansource->is_saved)
|
|
|
|
{
|
|
|
|
MemoryContextSetParent(plan->context, CacheMemoryContext);
|
|
|
|
plan->is_saved = true;
|
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
return plan;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* ReleaseCachedPlan: release active use of a cached plan.
|
|
|
|
*
|
|
|
|
* This decrements the reference count, and frees the plan if the count
|
|
|
|
* has thereby gone to zero. If useResOwner is true, it is assumed that
|
|
|
|
* the reference count is managed by the CurrentResourceOwner.
|
|
|
|
*
|
|
|
|
* Note: useResOwner = false is used for releasing references that are in
|
|
|
|
* persistent data structures, such as the parent CachedPlanSource or a
|
2014-05-06 18:12:18 +02:00
|
|
|
* Portal. Transient references should be protected by a resource owner.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
void
|
2007-11-15 23:25:18 +01:00
|
|
|
ReleaseCachedPlan(CachedPlan *plan, bool useResOwner)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(plan->magic == CACHEDPLAN_MAGIC);
|
2007-03-13 01:33:44 +01:00
|
|
|
if (useResOwner)
|
2011-09-16 06:42:53 +02:00
|
|
|
{
|
|
|
|
Assert(plan->is_saved);
|
2007-03-13 01:33:44 +01:00
|
|
|
ResourceOwnerForgetPlanCacheRef(CurrentResourceOwner, plan);
|
2011-09-16 06:42:53 +02:00
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
Assert(plan->refcount > 0);
|
|
|
|
plan->refcount--;
|
|
|
|
if (plan->refcount == 0)
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
{
|
|
|
|
/* Mark it no longer valid */
|
|
|
|
plan->magic = 0;
|
|
|
|
|
|
|
|
/* One-shot plans do not own their context, so we can't free them */
|
|
|
|
if (!plan->is_oneshot)
|
|
|
|
MemoryContextDelete(plan->context);
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
2008-09-16 01:37:40 +02:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* CachedPlanSetParentContext: move a CachedPlanSource to a new memory context
|
|
|
|
*
|
|
|
|
* This can only be applied to unsaved plans; once saved, a plan always
|
|
|
|
* lives underneath CacheMemoryContext.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
CachedPlanSetParentContext(CachedPlanSource *plansource,
|
|
|
|
MemoryContext newcontext)
|
|
|
|
{
|
|
|
|
/* Assert caller is doing things in a sane order */
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(plansource->is_complete);
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/* These seem worth real tests, though */
|
2011-09-16 06:42:53 +02:00
|
|
|
if (plansource->is_saved)
|
|
|
|
elog(ERROR, "cannot move a saved cached plan to another context");
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
if (plansource->is_oneshot)
|
|
|
|
elog(ERROR, "cannot move a one-shot cached plan to another context");
|
2011-09-16 06:42:53 +02:00
|
|
|
|
|
|
|
/* OK, let the caller keep the plan where he wishes */
|
|
|
|
MemoryContextSetParent(plansource->context, newcontext);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* The query_context needs no special handling, since it's a child of
|
|
|
|
* plansource->context. But if there's a generic plan, it should be
|
|
|
|
* maintained as a sibling of plansource->context.
|
|
|
|
*/
|
|
|
|
if (plansource->gplan)
|
|
|
|
{
|
|
|
|
Assert(plansource->gplan->magic == CACHEDPLAN_MAGIC);
|
|
|
|
MemoryContextSetParent(plansource->gplan->context, newcontext);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* CopyCachedPlan: make a copy of a CachedPlanSource
|
|
|
|
*
|
|
|
|
* This is a convenience routine that does the equivalent of
|
|
|
|
* CreateCachedPlan + CompleteCachedPlan, using the data stored in the
|
2014-05-06 18:12:18 +02:00
|
|
|
* input CachedPlanSource. The result is therefore "unsaved" (regardless
|
2011-09-16 06:42:53 +02:00
|
|
|
* of the state of the source), and we don't copy any generic plan either.
|
|
|
|
* The result will be currently valid, or not, the same as the source.
|
|
|
|
*/
|
|
|
|
CachedPlanSource *
|
|
|
|
CopyCachedPlan(CachedPlanSource *plansource)
|
|
|
|
{
|
|
|
|
CachedPlanSource *newsource;
|
|
|
|
MemoryContext source_context;
|
|
|
|
MemoryContext querytree_context;
|
|
|
|
MemoryContext oldcxt;
|
|
|
|
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(plansource->is_complete);
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
/*
|
|
|
|
* One-shot plans can't be copied, because we haven't taken care that
|
|
|
|
* parsing/planning didn't scribble on the raw parse tree or querytrees.
|
|
|
|
*/
|
|
|
|
if (plansource->is_oneshot)
|
|
|
|
elog(ERROR, "cannot copy a one-shot cached plan");
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
source_context = AllocSetContextCreate(CurrentMemoryContext,
|
|
|
|
"CachedPlanSource",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
|
|
|
|
|
|
|
oldcxt = MemoryContextSwitchTo(source_context);
|
|
|
|
|
|
|
|
newsource = (CachedPlanSource *) palloc0(sizeof(CachedPlanSource));
|
|
|
|
newsource->magic = CACHEDPLANSOURCE_MAGIC;
|
|
|
|
newsource->raw_parse_tree = copyObject(plansource->raw_parse_tree);
|
|
|
|
newsource->query_string = pstrdup(plansource->query_string);
|
|
|
|
newsource->commandTag = plansource->commandTag;
|
|
|
|
if (plansource->num_params > 0)
|
|
|
|
{
|
|
|
|
newsource->param_types = (Oid *)
|
|
|
|
palloc(plansource->num_params * sizeof(Oid));
|
|
|
|
memcpy(newsource->param_types, plansource->param_types,
|
|
|
|
plansource->num_params * sizeof(Oid));
|
|
|
|
}
|
|
|
|
else
|
|
|
|
newsource->param_types = NULL;
|
|
|
|
newsource->num_params = plansource->num_params;
|
|
|
|
newsource->parserSetup = plansource->parserSetup;
|
|
|
|
newsource->parserSetupArg = plansource->parserSetupArg;
|
|
|
|
newsource->cursor_options = plansource->cursor_options;
|
|
|
|
newsource->fixed_result = plansource->fixed_result;
|
|
|
|
if (plansource->resultDesc)
|
|
|
|
newsource->resultDesc = CreateTupleDescCopy(plansource->resultDesc);
|
|
|
|
else
|
|
|
|
newsource->resultDesc = NULL;
|
|
|
|
newsource->context = source_context;
|
|
|
|
|
|
|
|
querytree_context = AllocSetContextCreate(source_context,
|
|
|
|
"CachedPlanQuery",
|
|
|
|
ALLOCSET_SMALL_MINSIZE,
|
|
|
|
ALLOCSET_SMALL_INITSIZE,
|
|
|
|
ALLOCSET_DEFAULT_MAXSIZE);
|
|
|
|
MemoryContextSwitchTo(querytree_context);
|
|
|
|
newsource->query_list = (List *) copyObject(plansource->query_list);
|
|
|
|
newsource->relationOids = (List *) copyObject(plansource->relationOids);
|
|
|
|
newsource->invalItems = (List *) copyObject(plansource->invalItems);
|
Change plan caching to honor, not resist, changes in search_path.
In the initial implementation of plan caching, we saved the active
search_path when a plan was first cached, then reinstalled that path
anytime we needed to reparse or replan. The idea of that was to try to
reselect the same referenced objects, in somewhat the same way that views
continue to refer to the same objects in the face of schema or name
changes. Of course, that analogy doesn't bear close inspection, since
holding the search_path fixed doesn't cope with object drops or renames.
Moreover sticking with the old path seems to create more surprises than
it avoids. So instead of doing that, consider that the cached plan depends
on search_path, and force reparse/replan if the active search_path is
different than it was when we last saved the plan.
This gets us fairly close to having "transparency" of plan caching, in the
sense that the cached statement acts the same as if you'd just resubmitted
the original query text for another execution. There are still some corner
cases where this fails though: a new object added in the search path
schema(s) might capture a reference in the query text, but we'd not realize
that and force a reparse. We might try to fix that in the future, but for
the moment it looks too expensive and complicated.
2013-01-25 20:14:41 +01:00
|
|
|
if (plansource->search_path)
|
|
|
|
newsource->search_path = CopyOverrideSearchPath(plansource->search_path);
|
2011-09-16 06:42:53 +02:00
|
|
|
newsource->query_context = querytree_context;
|
|
|
|
|
|
|
|
newsource->gplan = NULL;
|
|
|
|
|
Invent a "one-shot" variant of CachedPlans for better performance.
SPI_execute() and related functions create a CachedPlan, execute it once,
and immediately discard it, so that the functionality offered by
plancache.c is of no value in this code path. And performance measurements
show that the extra data copying and invalidation checking done by
plancache.c slows down simple queries by 10% or more compared to 9.1.
However, enough of the SPI code is shared with functions that do need plan
caching that it seems impractical to bypass plancache.c altogether.
Instead, let's invent a variant version of cached plans that preserves
99% of the API but doesn't offer any of the actual functionality, nor the
overhead. This puts SPI_execute() performance back on par, or maybe even
slightly better, than it was before. This change should resolve recent
complaints of performance degradation from Dong Ye, Pavel Stehule, and
others.
By avoiding data copying, this change also reduces the amount of memory
needed to execute many-statement SPI_execute() strings, as for instance in
a recent complaint from Tomas Vondra.
An additional benefit of this change is that multi-statement SPI_execute()
query strings are now processed fully serially, that is we complete
execution of earlier statements before running parse analysis and planning
on following ones. This eliminates a long-standing POLA violation, in that
DDL that affects the behavior of a later statement will now behave as
expected.
Back-patch to 9.2, since this was a performance regression compared to 9.1.
(In 9.2, place the added struct fields so as to avoid changing the offsets
of existing fields.)
Heikki Linnakangas and Tom Lane
2013-01-04 23:42:19 +01:00
|
|
|
newsource->is_oneshot = false;
|
2011-09-16 06:42:53 +02:00
|
|
|
newsource->is_complete = true;
|
|
|
|
newsource->is_saved = false;
|
|
|
|
newsource->is_valid = plansource->is_valid;
|
|
|
|
newsource->generation = plansource->generation;
|
|
|
|
newsource->next_saved = NULL;
|
|
|
|
|
|
|
|
/* We may as well copy any acquired cost knowledge */
|
|
|
|
newsource->generic_cost = plansource->generic_cost;
|
|
|
|
newsource->total_custom_cost = plansource->total_custom_cost;
|
|
|
|
newsource->num_custom_plans = plansource->num_custom_plans;
|
|
|
|
|
Reset plan->row_security_env and planUserId
In the plancache, we check if the environment we planned the query under
has changed in a way which requires us to re-plan, such as when the user
for whom the plan was prepared changes and RLS is being used (and,
therefore, there may be different policies to apply).
Unfortunately, while those values were set and checked, they were not
being reset when the query was re-planned and therefore, in cases where
we change role, re-plan, and then change role again, we weren't
re-planning again. This leads to potentially incorrect policies being
applied in cases where role-specific policies are used and a given query
is planned under one role and then executed under other roles, which
could happen under security definer functions or when a common user and
query is planned initially and then re-used across multiple SET ROLEs.
Further, extensions which made use of CopyCachedPlan() may suffer from
similar issues as the RLS-related fields were not properly copied as
part of the plan and therefore RevalidateCachedQuery() would copy in the
current settings without invalidating the query.
Fix by using the same approach used for 'search_path', where we set the
correct values in CompleteCachedPlan(), check them early on in
RevalidateCachedQuery() and then properly reset them if re-planning.
Also, copy through the values during CopyCachedPlan().
Pointed out by Ashutosh Bapat. Reviewed by Michael Paquier.
Back-patch to 9.5 where RLS was introduced.
Security: CVE-2016-2193
2016-03-28 15:03:20 +02:00
|
|
|
/*
|
|
|
|
* Copy over the user the query was planned as, and under what RLS
|
|
|
|
* environment. We will check during RevalidateCachedQuery() if the user
|
|
|
|
* or environment has changed and, if so, will force a re-plan.
|
|
|
|
*/
|
|
|
|
newsource->planUserId = plansource->planUserId;
|
|
|
|
newsource->row_security_env = plansource->row_security_env;
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
MemoryContextSwitchTo(oldcxt);
|
|
|
|
|
|
|
|
return newsource;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* CachedPlanIsValid: test whether the rewritten querytree within a
|
|
|
|
* CachedPlanSource is currently valid (that is, not marked as being in need
|
|
|
|
* of revalidation).
|
2008-09-16 01:37:40 +02:00
|
|
|
*
|
|
|
|
* This result is only trustworthy (ie, free from race conditions) if
|
|
|
|
* the caller has acquired locks on all the relations used in the plan.
|
|
|
|
*/
|
|
|
|
bool
|
|
|
|
CachedPlanIsValid(CachedPlanSource *plansource)
|
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
return plansource->is_valid;
|
|
|
|
}
|
2008-09-16 01:37:40 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* CachedPlanGetTargetList: return tlist, if any, describing plan's output
|
|
|
|
*
|
|
|
|
* The result is guaranteed up-to-date. However, it is local storage
|
|
|
|
* within the cached plan, and may disappear next time the plan is updated.
|
|
|
|
*/
|
|
|
|
List *
|
|
|
|
CachedPlanGetTargetList(CachedPlanSource *plansource)
|
|
|
|
{
|
|
|
|
Node *pstmt;
|
2008-09-16 01:37:40 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Assert caller is doing things in a sane order */
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
Assert(plansource->is_complete);
|
2008-09-16 01:37:40 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* No work needed if statement doesn't return tuples (we assume this
|
|
|
|
* feature cannot be changed by an invalidation)
|
|
|
|
*/
|
|
|
|
if (plansource->resultDesc == NULL)
|
|
|
|
return NIL;
|
2008-09-16 01:37:40 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Make sure the querytree list is valid and we have parse-time locks */
|
|
|
|
RevalidateCachedQuery(plansource);
|
|
|
|
|
|
|
|
/* Get the primary statement and find out what it returns */
|
|
|
|
pstmt = PortalListGetPrimaryStmt(plansource->query_list);
|
|
|
|
|
|
|
|
return FetchStatementTargetList(pstmt);
|
2008-09-16 01:37:40 +02:00
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* AcquireExecutorLocks: acquire locks needed for execution of a cached plan;
|
|
|
|
* or release them if acquire is false.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
static void
|
|
|
|
AcquireExecutorLocks(List *stmt_list, bool acquire)
|
|
|
|
{
|
|
|
|
ListCell *lc1;
|
|
|
|
|
|
|
|
foreach(lc1, stmt_list)
|
|
|
|
{
|
|
|
|
PlannedStmt *plannedstmt = (PlannedStmt *) lfirst(lc1);
|
|
|
|
int rt_index;
|
|
|
|
ListCell *lc2;
|
|
|
|
|
|
|
|
Assert(!IsA(plannedstmt, Query));
|
|
|
|
if (!IsA(plannedstmt, PlannedStmt))
|
2010-01-15 23:36:35 +01:00
|
|
|
{
|
|
|
|
/*
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
* Ignore utility statements, except those (such as EXPLAIN) that
|
2014-05-06 18:12:18 +02:00
|
|
|
* contain a parsed-but-not-planned query. Note: it's okay to use
|
2010-01-15 23:36:35 +01:00
|
|
|
* ScanQueryForLocks, even though the query hasn't been through
|
|
|
|
* rule rewriting, because rewriting doesn't change the query
|
|
|
|
* representation.
|
|
|
|
*/
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
Query *query = UtilityContainsQuery((Node *) plannedstmt);
|
2010-01-15 23:36:35 +01:00
|
|
|
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
if (query)
|
2010-01-15 23:36:35 +01:00
|
|
|
ScanQueryForLocks(query, acquire);
|
|
|
|
continue;
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
rt_index = 0;
|
|
|
|
foreach(lc2, plannedstmt->rtable)
|
|
|
|
{
|
|
|
|
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc2);
|
|
|
|
LOCKMODE lockmode;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
PlanRowMark *rc;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
rt_index++;
|
|
|
|
|
|
|
|
if (rte->rtekind != RTE_RELATION)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
2007-11-15 22:14:46 +01:00
|
|
|
* Acquire the appropriate type of lock on each relation OID. Note
|
|
|
|
* that we don't actually try to open the rel, and hence will not
|
|
|
|
* fail if it's been dropped entirely --- we'll just transiently
|
|
|
|
* acquire a non-conflicting lock.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
if (list_member_int(plannedstmt->resultRelations, rt_index))
|
|
|
|
lockmode = RowExclusiveLock;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
else if ((rc = get_plan_rowmark(plannedstmt->rowMarks, rt_index)) != NULL &&
|
|
|
|
RowMarkRequiresRowShareLock(rc->markType))
|
2007-03-13 01:33:44 +01:00
|
|
|
lockmode = RowShareLock;
|
|
|
|
else
|
|
|
|
lockmode = AccessShareLock;
|
|
|
|
|
|
|
|
if (acquire)
|
|
|
|
LockRelationOid(rte->relid, lockmode);
|
|
|
|
else
|
|
|
|
UnlockRelationOid(rte->relid, lockmode);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* AcquirePlannerLocks: acquire locks needed for planning of a querytree list;
|
|
|
|
* or release them if acquire is false.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
|
|
|
* Note that we don't actually try to open the relations, and hence will not
|
|
|
|
* fail if one has been dropped entirely --- we'll just transiently acquire
|
|
|
|
* a non-conflicting lock.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
AcquirePlannerLocks(List *stmt_list, bool acquire)
|
|
|
|
{
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
foreach(lc, stmt_list)
|
|
|
|
{
|
|
|
|
Query *query = (Query *) lfirst(lc);
|
|
|
|
|
|
|
|
Assert(IsA(query, Query));
|
2010-01-15 23:36:35 +01:00
|
|
|
|
|
|
|
if (query->commandType == CMD_UTILITY)
|
|
|
|
{
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
/* Ignore utility statements, unless they contain a Query */
|
|
|
|
query = UtilityContainsQuery(query->utilityStmt);
|
|
|
|
if (query)
|
2010-01-15 23:36:35 +01:00
|
|
|
ScanQueryForLocks(query, acquire);
|
|
|
|
continue;
|
|
|
|
}
|
|
|
|
|
2008-09-09 20:58:09 +02:00
|
|
|
ScanQueryForLocks(query, acquire);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* ScanQueryForLocks: recursively scan one Query for AcquirePlannerLocks.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
static void
|
2008-09-09 20:58:09 +02:00
|
|
|
ScanQueryForLocks(Query *parsetree, bool acquire)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
|
|
|
ListCell *lc;
|
|
|
|
int rt_index;
|
|
|
|
|
2010-01-15 23:36:35 +01:00
|
|
|
/* Shouldn't get called on utility commands */
|
|
|
|
Assert(parsetree->commandType != CMD_UTILITY);
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
|
|
|
* First, process RTEs of the current query level.
|
|
|
|
*/
|
|
|
|
rt_index = 0;
|
|
|
|
foreach(lc, parsetree->rtable)
|
|
|
|
{
|
|
|
|
RangeTblEntry *rte = (RangeTblEntry *) lfirst(lc);
|
|
|
|
LOCKMODE lockmode;
|
|
|
|
|
|
|
|
rt_index++;
|
|
|
|
switch (rte->rtekind)
|
|
|
|
{
|
|
|
|
case RTE_RELATION:
|
2008-09-09 20:58:09 +02:00
|
|
|
/* Acquire or release the appropriate type of lock */
|
2007-03-13 01:33:44 +01:00
|
|
|
if (rt_index == parsetree->resultRelation)
|
|
|
|
lockmode = RowExclusiveLock;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
else if (get_parse_rowmark(parsetree, rt_index) != NULL)
|
2007-03-13 01:33:44 +01:00
|
|
|
lockmode = RowShareLock;
|
|
|
|
else
|
|
|
|
lockmode = AccessShareLock;
|
2008-09-09 20:58:09 +02:00
|
|
|
if (acquire)
|
|
|
|
LockRelationOid(rte->relid, lockmode);
|
|
|
|
else
|
|
|
|
UnlockRelationOid(rte->relid, lockmode);
|
2007-03-13 01:33:44 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
case RTE_SUBQUERY:
|
2008-09-09 20:58:09 +02:00
|
|
|
/* Recurse into subquery-in-FROM */
|
|
|
|
ScanQueryForLocks(rte->subquery, acquire);
|
2007-03-13 01:33:44 +01:00
|
|
|
break;
|
|
|
|
|
|
|
|
default:
|
|
|
|
/* ignore other types of RTEs */
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2008-10-04 23:56:55 +02:00
|
|
|
/* Recurse into subquery-in-WITH */
|
|
|
|
foreach(lc, parsetree->cteList)
|
|
|
|
{
|
|
|
|
CommonTableExpr *cte = (CommonTableExpr *) lfirst(lc);
|
|
|
|
|
|
|
|
ScanQueryForLocks((Query *) cte->ctequery, acquire);
|
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
|
|
|
* Recurse into sublink subqueries, too. But we already did the ones in
|
2008-10-04 23:56:55 +02:00
|
|
|
* the rtable and cteList.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
if (parsetree->hasSubLinks)
|
|
|
|
{
|
|
|
|
query_tree_walker(parsetree, ScanQueryWalker,
|
2008-09-09 20:58:09 +02:00
|
|
|
(void *) &acquire,
|
2008-10-04 23:56:55 +02:00
|
|
|
QTW_IGNORE_RC_SUBQUERIES);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* Walker to find sublink subqueries for ScanQueryForLocks
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
static bool
|
2008-09-09 20:58:09 +02:00
|
|
|
ScanQueryWalker(Node *node, bool *acquire)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
|
|
|
if (node == NULL)
|
|
|
|
return false;
|
|
|
|
if (IsA(node, SubLink))
|
|
|
|
{
|
|
|
|
SubLink *sub = (SubLink *) node;
|
|
|
|
|
|
|
|
/* Do what we came for */
|
2008-09-09 20:58:09 +02:00
|
|
|
ScanQueryForLocks((Query *) sub->subselect, *acquire);
|
2007-03-13 01:33:44 +01:00
|
|
|
/* Fall through to process lefthand args of SubLink */
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* Do NOT recurse into Query nodes, because ScanQueryForLocks already
|
2007-11-15 22:14:46 +01:00
|
|
|
* processed subselects of subselects for us.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
return expression_tree_walker(node, ScanQueryWalker,
|
2008-09-09 20:58:09 +02:00
|
|
|
(void *) acquire);
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
|
2007-09-20 19:56:33 +02:00
|
|
|
/*
|
|
|
|
* plan_list_is_transient: check if any of the plans in the list are transient.
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
plan_list_is_transient(List *stmt_list)
|
|
|
|
{
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
foreach(lc, stmt_list)
|
|
|
|
{
|
|
|
|
PlannedStmt *plannedstmt = (PlannedStmt *) lfirst(lc);
|
2007-11-15 22:14:46 +01:00
|
|
|
|
2007-09-20 19:56:33 +02:00
|
|
|
if (!IsA(plannedstmt, PlannedStmt))
|
|
|
|
continue; /* Ignore utility statements */
|
|
|
|
|
|
|
|
if (plannedstmt->transientPlan)
|
|
|
|
return true;
|
2007-11-15 22:14:46 +01:00
|
|
|
}
|
2007-09-20 19:56:33 +02:00
|
|
|
|
|
|
|
return false;
|
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* PlanCacheComputeResultDesc: given a list of analyzed-and-rewritten Queries,
|
2014-05-06 18:12:18 +02:00
|
|
|
* determine the result tupledesc it will produce. Returns NULL if the
|
2011-09-16 06:42:53 +02:00
|
|
|
* execution will not return tuples.
|
2007-03-13 01:33:44 +01:00
|
|
|
*
|
|
|
|
* Note: the result is created or copied into current memory context.
|
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
static TupleDesc
|
2007-03-16 00:12:07 +01:00
|
|
|
PlanCacheComputeResultDesc(List *stmt_list)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
|
|
|
Query *query;
|
|
|
|
|
|
|
|
switch (ChoosePortalStrategy(stmt_list))
|
|
|
|
{
|
|
|
|
case PORTAL_ONE_SELECT:
|
2011-02-26 00:56:23 +01:00
|
|
|
case PORTAL_ONE_MOD_WITH:
|
2011-09-16 06:42:53 +02:00
|
|
|
query = (Query *) linitial(stmt_list);
|
|
|
|
Assert(IsA(query, Query));
|
|
|
|
return ExecCleanTypeFromTL(query->targetList, false);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
case PORTAL_ONE_RETURNING:
|
2011-09-16 06:42:53 +02:00
|
|
|
query = (Query *) PortalListGetPrimaryStmt(stmt_list);
|
|
|
|
Assert(IsA(query, Query));
|
|
|
|
Assert(query->returningList);
|
|
|
|
return ExecCleanTypeFromTL(query->returningList, false);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
case PORTAL_UTIL_SELECT:
|
2011-09-16 06:42:53 +02:00
|
|
|
query = (Query *) linitial(stmt_list);
|
|
|
|
Assert(IsA(query, Query));
|
|
|
|
Assert(query->utilityStmt);
|
|
|
|
return UtilityTupleDescriptor(query->utilityStmt);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
case PORTAL_MULTI_QUERY:
|
|
|
|
/* will not return tuples */
|
|
|
|
break;
|
|
|
|
}
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* PlanCacheRelCallback
|
2007-03-13 01:33:44 +01:00
|
|
|
* Relcache inval callback function
|
2007-03-26 02:36:19 +02:00
|
|
|
*
|
|
|
|
* Invalidate all plans mentioning the given rel, or all plans mentioning
|
|
|
|
* any rel at all if relid == InvalidOid.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
|
|
|
static void
|
2008-09-09 20:58:09 +02:00
|
|
|
PlanCacheRelCallback(Datum arg, Oid relid)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
CachedPlanSource *plansource;
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
for (plansource = first_saved_plan; plansource; plansource = plansource->next_saved)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
/* No work if it's already invalidated */
|
2011-09-16 06:42:53 +02:00
|
|
|
if (!plansource->is_valid)
|
2007-03-13 01:33:44 +01:00
|
|
|
continue;
|
2010-01-15 23:36:35 +01:00
|
|
|
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
/* Never invalidate transaction control commands */
|
|
|
|
if (IsTransactionStmtPlan(plansource))
|
|
|
|
continue;
|
|
|
|
|
2010-01-15 23:36:35 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Check the dependency list for the rewritten querytree.
|
2010-01-15 23:36:35 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
if ((relid == InvalidOid) ? plansource->relationOids != NIL :
|
|
|
|
list_member_oid(plansource->relationOids, relid))
|
|
|
|
{
|
|
|
|
/* Invalidate the querytree and generic plan */
|
|
|
|
plansource->is_valid = false;
|
|
|
|
if (plansource->gplan)
|
|
|
|
plansource->gplan->is_valid = false;
|
|
|
|
}
|
2010-01-15 23:36:35 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* The generic plan, if any, could have more dependencies than the
|
|
|
|
* querytree does, so we have to check it too.
|
|
|
|
*/
|
|
|
|
if (plansource->gplan && plansource->gplan->is_valid)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
ListCell *lc;
|
2008-09-09 20:58:09 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
foreach(lc, plansource->gplan->stmt_list)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
PlannedStmt *plannedstmt = (PlannedStmt *) lfirst(lc);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
|
|
|
Assert(!IsA(plannedstmt, Query));
|
|
|
|
if (!IsA(plannedstmt, PlannedStmt))
|
2007-11-15 22:14:46 +01:00
|
|
|
continue; /* Ignore utility statements */
|
2007-10-11 20:05:27 +02:00
|
|
|
if ((relid == InvalidOid) ? plannedstmt->relationOids != NIL :
|
|
|
|
list_member_oid(plannedstmt->relationOids, relid))
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Invalidate the generic plan only */
|
|
|
|
plansource->gplan->is_valid = false;
|
2007-11-15 22:14:46 +01:00
|
|
|
break; /* out of stmt_list scan */
|
2007-10-11 20:05:27 +02:00
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
}
|
2008-09-09 20:58:09 +02:00
|
|
|
}
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2008-09-09 20:58:09 +02:00
|
|
|
/*
|
|
|
|
* PlanCacheFuncCallback
|
|
|
|
* Syscache inval callback function for PROCOID cache
|
|
|
|
*
|
2011-08-17 01:27:46 +02:00
|
|
|
* Invalidate all plans mentioning the object with the specified hash value,
|
|
|
|
* or all plans mentioning any member of this cache if hashvalue == 0.
|
2008-09-09 20:58:09 +02:00
|
|
|
*
|
|
|
|
* Note that the coding would support use for multiple caches, but right
|
|
|
|
* now only user-defined functions are tracked this way.
|
|
|
|
*/
|
|
|
|
static void
|
2011-08-17 01:27:46 +02:00
|
|
|
PlanCacheFuncCallback(Datum arg, int cacheid, uint32 hashvalue)
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
CachedPlanSource *plansource;
|
2008-09-09 20:58:09 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
for (plansource = first_saved_plan; plansource; plansource = plansource->next_saved)
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
2008-09-09 20:58:09 +02:00
|
|
|
|
|
|
|
/* No work if it's already invalidated */
|
2011-09-16 06:42:53 +02:00
|
|
|
if (!plansource->is_valid)
|
2008-09-09 20:58:09 +02:00
|
|
|
continue;
|
2010-01-15 23:36:35 +01:00
|
|
|
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
/* Never invalidate transaction control commands */
|
|
|
|
if (IsTransactionStmtPlan(plansource))
|
|
|
|
continue;
|
|
|
|
|
2010-01-15 23:36:35 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* Check the dependency list for the rewritten querytree.
|
2010-01-15 23:36:35 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
foreach(lc, plansource->invalItems)
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
PlanInvalItem *item = (PlanInvalItem *) lfirst(lc);
|
2007-03-13 01:33:44 +01:00
|
|
|
|
2010-01-15 23:36:35 +01:00
|
|
|
if (item->cacheId != cacheid)
|
|
|
|
continue;
|
2011-08-17 01:27:46 +02:00
|
|
|
if (hashvalue == 0 ||
|
|
|
|
item->hashValue == hashvalue)
|
2010-01-15 23:36:35 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Invalidate the querytree and generic plan */
|
|
|
|
plansource->is_valid = false;
|
|
|
|
if (plansource->gplan)
|
|
|
|
plansource->gplan->is_valid = false;
|
2010-01-15 23:36:35 +01:00
|
|
|
break;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
/*
|
|
|
|
* The generic plan, if any, could have more dependencies than the
|
|
|
|
* querytree does, so we have to check it too.
|
|
|
|
*/
|
|
|
|
if (plansource->gplan && plansource->gplan->is_valid)
|
2010-01-15 23:36:35 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
foreach(lc, plansource->gplan->stmt_list)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
PlannedStmt *plannedstmt = (PlannedStmt *) lfirst(lc);
|
2008-09-09 20:58:09 +02:00
|
|
|
ListCell *lc3;
|
|
|
|
|
|
|
|
Assert(!IsA(plannedstmt, Query));
|
|
|
|
if (!IsA(plannedstmt, PlannedStmt))
|
|
|
|
continue; /* Ignore utility statements */
|
|
|
|
foreach(lc3, plannedstmt->invalItems)
|
|
|
|
{
|
|
|
|
PlanInvalItem *item = (PlanInvalItem *) lfirst(lc3);
|
|
|
|
|
|
|
|
if (item->cacheId != cacheid)
|
|
|
|
continue;
|
2011-08-17 01:27:46 +02:00
|
|
|
if (hashvalue == 0 ||
|
|
|
|
item->hashValue == hashvalue)
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* Invalidate the generic plan only */
|
|
|
|
plansource->gplan->is_valid = false;
|
2009-06-11 16:49:15 +02:00
|
|
|
break; /* out of invalItems scan */
|
2008-09-09 20:58:09 +02:00
|
|
|
}
|
|
|
|
}
|
2011-09-16 06:42:53 +02:00
|
|
|
if (!plansource->gplan->is_valid)
|
2008-09-09 20:58:09 +02:00
|
|
|
break; /* out of stmt_list scan */
|
|
|
|
}
|
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2007-04-12 08:53:49 +02:00
|
|
|
/*
|
2008-09-09 20:58:09 +02:00
|
|
|
* PlanCacheSysCallback
|
|
|
|
* Syscache inval callback function for other caches
|
|
|
|
*
|
|
|
|
* Just invalidate everything...
|
2007-04-12 08:53:49 +02:00
|
|
|
*/
|
2008-09-09 20:58:09 +02:00
|
|
|
static void
|
2011-08-17 01:27:46 +02:00
|
|
|
PlanCacheSysCallback(Datum arg, int cacheid, uint32 hashvalue)
|
2007-04-12 08:53:49 +02:00
|
|
|
{
|
2008-09-09 20:58:09 +02:00
|
|
|
ResetPlanCache();
|
2007-04-12 08:53:49 +02:00
|
|
|
}
|
|
|
|
|
2016-01-28 20:05:36 +01:00
|
|
|
/*
|
|
|
|
* PlanCacheUserMappingCallback
|
2016-06-10 00:02:36 +02:00
|
|
|
* Syscache inval callback function for user mapping cache invalidation.
|
2016-01-28 20:05:36 +01:00
|
|
|
*
|
2016-06-10 00:02:36 +02:00
|
|
|
* Invalidates plans which have pushed down foreign joins.
|
2016-01-28 20:05:36 +01:00
|
|
|
*/
|
|
|
|
static void
|
|
|
|
PlanCacheUserMappingCallback(Datum arg, int cacheid, uint32 hashvalue)
|
|
|
|
{
|
|
|
|
CachedPlanSource *plansource;
|
|
|
|
|
|
|
|
for (plansource = first_saved_plan; plansource; plansource = plansource->next_saved)
|
|
|
|
{
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
|
|
|
|
|
|
|
/* No work if it's already invalidated */
|
|
|
|
if (!plansource->is_valid)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/* Never invalidate transaction control commands */
|
|
|
|
if (IsTransactionStmtPlan(plansource))
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If the plan has pushed down foreign joins, those join may become
|
2016-06-10 00:02:36 +02:00
|
|
|
* unsafe to push down because of user mapping changes. Invalidate
|
|
|
|
* only the generic plan, since changes to user mapping do not
|
|
|
|
* invalidate the parse tree.
|
2016-01-28 20:05:36 +01:00
|
|
|
*/
|
|
|
|
if (plansource->gplan && plansource->gplan->has_foreign_join)
|
|
|
|
plansource->gplan->is_valid = false;
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
2007-03-13 01:33:44 +01:00
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* ResetPlanCache: invalidate all cached plans.
|
2007-03-13 01:33:44 +01:00
|
|
|
*/
|
2008-09-09 20:58:09 +02:00
|
|
|
void
|
|
|
|
ResetPlanCache(void)
|
2007-03-13 01:33:44 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
CachedPlanSource *plansource;
|
2008-09-09 20:58:09 +02:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
for (plansource = first_saved_plan; plansource; plansource = plansource->next_saved)
|
2008-09-09 20:58:09 +02:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
Assert(plansource->magic == CACHEDPLANSOURCE_MAGIC);
|
2008-09-09 20:58:09 +02:00
|
|
|
|
2010-01-13 17:56:56 +01:00
|
|
|
/* No work if it's already invalidated */
|
2011-09-16 06:42:53 +02:00
|
|
|
if (!plansource->is_valid)
|
2010-01-13 17:56:56 +01:00
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
2011-09-16 06:42:53 +02:00
|
|
|
* We *must not* mark transaction control statements as invalid,
|
2010-01-13 17:56:56 +01:00
|
|
|
* particularly not ROLLBACK, because they may need to be executed in
|
|
|
|
* aborted transactions when we can't revalidate them (cf bug #5269).
|
Fix longstanding race condition in plancache.c.
When creating or manipulating a cached plan for a transaction control
command (particularly ROLLBACK), we must not perform any catalog accesses,
since we might be in an aborted transaction. However, plancache.c busily
saved or examined the search_path for every cached plan. If we were
unlucky enough to do this at a moment where the path's expansion into
schema OIDs wasn't already cached, we'd do some catalog accesses; and with
some more bad luck such as an ill-timed signal arrival, that could lead to
crashes or Assert failures, as exhibited in bug #8095 from Nachiket Vaidya.
Fortunately, there's no real need to consider the search path for such
commands, so we can just skip the relevant steps when the subject statement
is a TransactionStmt. This is somewhat related to bug #5269, though the
failure happens during initial cached-plan creation rather than
revalidation.
This bug has been there since the plan cache was invented, so back-patch
to all supported branches.
2013-04-20 22:59:21 +02:00
|
|
|
*/
|
|
|
|
if (IsTransactionStmtPlan(plansource))
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
2010-01-13 17:56:56 +01:00
|
|
|
* In general there is no point in invalidating utility statements
|
2011-09-16 06:42:53 +02:00
|
|
|
* since they have no plans anyway. So invalidate it only if it
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
* contains at least one non-utility statement, or contains a utility
|
|
|
|
* statement that contains a pre-analyzed query (which could have
|
|
|
|
* dependencies.)
|
2010-01-13 17:56:56 +01:00
|
|
|
*/
|
2011-09-16 06:42:53 +02:00
|
|
|
foreach(lc, plansource->query_list)
|
2010-01-13 17:56:56 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
Query *query = (Query *) lfirst(lc);
|
2010-01-13 17:56:56 +01:00
|
|
|
|
2011-09-16 06:42:53 +02:00
|
|
|
Assert(IsA(query, Query));
|
|
|
|
if (query->commandType != CMD_UTILITY ||
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
UtilityContainsQuery(query->utilityStmt))
|
2010-01-13 17:56:56 +01:00
|
|
|
{
|
2011-09-16 06:42:53 +02:00
|
|
|
/* non-utility statement, so invalidate */
|
|
|
|
plansource->is_valid = false;
|
|
|
|
if (plansource->gplan)
|
|
|
|
plansource->gplan->is_valid = false;
|
Restructure SELECT INTO's parsetree representation into CreateTableAsStmt.
Making this operation look like a utility statement seems generally a good
idea, and particularly so in light of the desire to provide command
triggers for utility statements. The original choice of representing it as
SELECT with an IntoClause appendage had metastasized into rather a lot of
places, unfortunately, so that this patch is a great deal more complicated
than one might at first expect.
In particular, keeping EXPLAIN working for SELECT INTO and CREATE TABLE AS
subcommands required restructuring some EXPLAIN-related APIs. Add-on code
that calls ExplainOnePlan or ExplainOneUtility, or uses
ExplainOneQuery_hook, will need adjustment.
Also, the cases PREPARE ... SELECT INTO and CREATE RULE ... SELECT INTO,
which formerly were accepted though undocumented, are no longer accepted.
The PREPARE case can be replaced with use of CREATE TABLE AS EXECUTE.
The CREATE RULE case doesn't seem to have much real-world use (since the
rule would work only once before failing with "table already exists"),
so we'll not bother with that one.
Both SELECT INTO and CREATE TABLE AS still return a command tag of
"SELECT nnnn". There was some discussion of returning "CREATE TABLE nnnn",
but for the moment backwards compatibility wins the day.
Andres Freund and Tom Lane
2012-03-20 02:37:19 +01:00
|
|
|
/* no need to look further */
|
2011-09-16 06:42:53 +02:00
|
|
|
break;
|
2010-01-13 17:56:56 +01:00
|
|
|
}
|
|
|
|
}
|
2008-09-09 20:58:09 +02:00
|
|
|
}
|
2007-03-13 01:33:44 +01:00
|
|
|
}
|