1996-08-28 03:59:28 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
1999-02-14 00:22:53 +01:00
|
|
|
* plannodes.h
|
1997-09-07 07:04:48 +02:00
|
|
|
* definitions for query plan nodes
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
|
|
|
*
|
2016-01-02 19:33:40 +01:00
|
|
|
* Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
|
2000-01-26 06:58:53 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/include/nodes/plannodes.h
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
#ifndef PLANNODES_H
|
1997-09-07 07:04:48 +02:00
|
|
|
#define PLANNODES_H
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2002-12-05 16:50:39 +01:00
|
|
|
#include "access/sdir.h"
|
2014-11-07 23:26:02 +01:00
|
|
|
#include "lib/stringinfo.h"
|
2003-02-09 01:30:41 +01:00
|
|
|
#include "nodes/bitmapset.h"
|
2015-03-15 20:19:04 +01:00
|
|
|
#include "nodes/lockoptions.h"
|
2002-12-05 16:50:39 +01:00
|
|
|
#include "nodes/primnodes.h"
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
|
|
|
|
/* ----------------------------------------------------------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* node definitions
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
|
2007-02-20 18:32:18 +01:00
|
|
|
/* ----------------
|
|
|
|
* PlannedStmt node
|
|
|
|
*
|
|
|
|
* The output of the planner is a Plan tree headed by a PlannedStmt node.
|
|
|
|
* PlannedStmt holds the "one time" information needed by the executor.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct PlannedStmt
|
|
|
|
{
|
|
|
|
NodeTag type;
|
|
|
|
|
|
|
|
CmdType commandType; /* select|insert|update|delete */
|
|
|
|
|
2012-03-27 21:14:13 +02:00
|
|
|
uint32 queryId; /* query identifier (copied from Query) */
|
|
|
|
|
2009-10-10 03:43:50 +02:00
|
|
|
bool hasReturning; /* is it insert|update|delete RETURNING? */
|
|
|
|
|
2011-02-26 00:56:23 +01:00
|
|
|
bool hasModifyingCTE; /* has insert|update|delete in WITH? */
|
|
|
|
|
2007-02-20 18:32:18 +01:00
|
|
|
bool canSetTag; /* do I set the command result tag? */
|
|
|
|
|
2007-09-20 19:56:33 +02:00
|
|
|
bool transientPlan; /* redo plan when TransactionXmin changes? */
|
|
|
|
|
Avoid invalidating all foreign-join cached plans when user mappings change.
We must not push down a foreign join when the foreign tables involved
should be accessed under different user mappings. Previously we tried
to enforce that rule literally during planning, but that meant that the
resulting plans were dependent on the current contents of the
pg_user_mapping catalog, and we had to blow away all cached plans
containing any remote join when anything at all changed in pg_user_mapping.
This could have been improved somewhat, but the fact that a syscache inval
callback has very limited info about what changed made it hard to do better
within that design. Instead, let's change the planner to not consider user
mappings per se, but to allow a foreign join if both RTEs have the same
checkAsUser value. If they do, then they necessarily will use the same
user mapping at runtime, and we don't need to know specifically which one
that is. Post-plan-time changes in pg_user_mapping no longer require any
plan invalidation.
This rule does give up some optimization ability, to wit where two foreign
table references come from views with different owners or one's from a view
and one's directly in the query, but nonetheless the same user mapping
would have applied. We'll sacrifice the first case, but to not regress
more than we have to in the second case, allow a foreign join involving
both zero and nonzero checkAsUser values if the nonzero one is the same as
the prevailing effective userID. In that case, mark the plan as only
runnable by that userID.
The plancache code already had a notion of plans being userID-specific,
in order to support RLS. It was a little confused though, in particular
lacking clarity of thought as to whether it was the rewritten query or just
the finished plan that's dependent on the userID. Rearrange that code so
that it's clearer what depends on which, and so that the same logic applies
to both RLS-injected role dependency and foreign-join-injected role
dependency.
Note that this patch doesn't remove the other issue mentioned in the
original complaint, which is that while we'll reliably stop using a foreign
join if it's disallowed in a new context, we might fail to start using a
foreign join if it's now allowed, but we previously created a generic
cached plan that didn't use one. It was agreed that the chance of winning
that way was not high enough to justify the much larger number of plan
invalidations that would have to occur if we tried to cause it to happen.
In passing, clean up randomly-varying spelling of EXPLAIN commands in
postgres_fdw.sql, and fix a COSTS ON example that had been allowed to
leak into the committed tests.
This reverts most of commits fbe5a3fb7 and 5d4171d1c, which were the
previous attempt at ensuring we wouldn't push down foreign joins that
span permissions contexts.
Etsuro Fujita and Tom Lane
Discussion: <d49c1e5b-f059-20f4-c132-e9752ee0113e@lab.ntt.co.jp>
2016-07-15 23:22:56 +02:00
|
|
|
bool dependsOnRole; /* is plan specific to current role? */
|
|
|
|
|
|
|
|
bool parallelModeNeeded; /* parallel mode required to execute? */
|
|
|
|
|
2007-02-20 18:32:18 +01:00
|
|
|
struct Plan *planTree; /* tree of Plan nodes */
|
|
|
|
|
|
|
|
List *rtable; /* list of RangeTblEntry nodes */
|
|
|
|
|
|
|
|
/* rtable indexes of target relations for INSERT/UPDATE/DELETE */
|
|
|
|
List *resultRelations; /* integer list of RT indexes, or NIL */
|
|
|
|
|
2007-04-28 00:05:49 +02:00
|
|
|
Node *utilityStmt; /* non-null if this is DECLARE CURSOR */
|
|
|
|
|
2007-02-22 23:00:26 +01:00
|
|
|
List *subplans; /* Plan trees for SubPlan expressions */
|
|
|
|
|
2007-02-27 02:11:26 +01:00
|
|
|
Bitmapset *rewindPlanIDs; /* indices of subplans that require REWIND */
|
|
|
|
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
List *rowMarks; /* a list of PlanRowMark's */
|
2007-02-20 18:32:18 +01:00
|
|
|
|
2007-10-11 20:05:27 +02:00
|
|
|
List *relationOids; /* OIDs of relations the plan depends on */
|
|
|
|
|
2008-09-09 20:58:09 +02:00
|
|
|
List *invalItems; /* other dependencies, as PlanInvalItems */
|
|
|
|
|
2007-02-20 18:32:18 +01:00
|
|
|
int nParamExec; /* number of PARAM_EXEC Params used */
|
2007-11-15 23:25:18 +01:00
|
|
|
} PlannedStmt;
|
2007-02-20 18:32:18 +01:00
|
|
|
|
2007-02-22 23:00:26 +01:00
|
|
|
/* macro for fetching the Plan associated with a SubPlan node */
|
|
|
|
#define exec_subplan_get_plan(plannedstmt, subplan) \
|
|
|
|
((Plan *) list_nth((plannedstmt)->subplans, (subplan)->plan_id - 1))
|
|
|
|
|
2007-02-20 18:32:18 +01:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* Plan node
|
2002-12-05 16:50:39 +01:00
|
|
|
*
|
|
|
|
* All plan nodes "derive" from the Plan structure by having the
|
|
|
|
* Plan structure as the first field. This ensures that everything works
|
|
|
|
* when nodes are cast to Plan's. (node pointers are frequently cast to Plan*
|
|
|
|
* when passed around generically in the executor)
|
|
|
|
*
|
|
|
|
* We never actually instantiate any Plan nodes; this is just the common
|
|
|
|
* abstract superclass for all Plan-type nodes.
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Plan
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
NodeTag type;
|
2000-01-09 01:26:47 +01:00
|
|
|
|
2002-12-05 16:50:39 +01:00
|
|
|
/*
|
|
|
|
* estimated execution costs for plan (see costsize.c for more info)
|
|
|
|
*/
|
2005-10-15 04:49:52 +02:00
|
|
|
Cost startup_cost; /* cost expended before fetching any tuples */
|
|
|
|
Cost total_cost; /* total cost (assuming all tuples fetched) */
|
2000-02-15 21:49:31 +01:00
|
|
|
|
2000-04-12 19:17:23 +02:00
|
|
|
/*
|
2002-12-05 16:50:39 +01:00
|
|
|
* planner's estimate of result size of this plan step
|
2000-02-15 21:49:31 +01:00
|
|
|
*/
|
|
|
|
double plan_rows; /* number of rows plan is expected to emit */
|
|
|
|
int plan_width; /* average row width in bytes */
|
2000-01-09 01:26:47 +01:00
|
|
|
|
2015-11-11 14:57:52 +01:00
|
|
|
/*
|
|
|
|
* information needed for parallel query
|
|
|
|
*/
|
|
|
|
bool parallel_aware; /* engage parallel-aware logic? */
|
|
|
|
|
2001-09-18 03:59:07 +02:00
|
|
|
/*
|
2002-12-05 16:50:39 +01:00
|
|
|
* Common structural data for all Plan types.
|
2001-09-18 03:59:07 +02:00
|
|
|
*/
|
2015-09-29 03:55:57 +02:00
|
|
|
int plan_node_id; /* unique across entire final plan tree */
|
2002-12-05 16:50:39 +01:00
|
|
|
List *targetlist; /* target list to be computed at this node */
|
|
|
|
List *qual; /* implicitly-ANDed qual conditions */
|
|
|
|
struct Plan *lefttree; /* input plan tree(s) */
|
|
|
|
struct Plan *righttree;
|
|
|
|
List *initPlan; /* Init Plan nodes (un-correlated expr
|
|
|
|
* subselects) */
|
2001-09-18 03:59:07 +02:00
|
|
|
|
|
|
|
/*
|
2002-12-05 16:50:39 +01:00
|
|
|
* Information for management of parameter-change-driven rescanning
|
2003-02-09 01:30:41 +01:00
|
|
|
*
|
2005-11-22 19:17:34 +01:00
|
|
|
* extParam includes the paramIDs of all external PARAM_EXEC params
|
|
|
|
* affecting this plan node or its children. setParam params from the
|
|
|
|
* node's initPlans are not included, but their extParams are.
|
2003-02-09 01:30:41 +01:00
|
|
|
*
|
2005-11-22 19:17:34 +01:00
|
|
|
* allParam includes all the extParam paramIDs, plus the IDs of local
|
|
|
|
* params that affect the node (i.e., the setParams of its initplans).
|
|
|
|
* These are _all_ the PARAM_EXEC params that affect this node.
|
2001-09-18 03:59:07 +02:00
|
|
|
*/
|
2003-02-09 01:30:41 +01:00
|
|
|
Bitmapset *extParam;
|
|
|
|
Bitmapset *allParam;
|
1997-09-08 23:56:23 +02:00
|
|
|
} Plan;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
/* ----------------
|
2010-10-26 10:15:17 +02:00
|
|
|
* these are defined to avoid confusion problems with "left"
|
1997-09-07 07:04:48 +02:00
|
|
|
* and "right" and "inner" and "outer". The convention is that
|
|
|
|
* the "left" plan is the "outer" plan and the "right" plan is
|
|
|
|
* the inner plan, but these make the code more readable.
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
#define innerPlan(node) (((Plan *)(node))->righttree)
|
|
|
|
#define outerPlan(node) (((Plan *)(node))->lefttree)
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
|
|
|
|
/* ----------------
|
2000-11-12 01:37:02 +01:00
|
|
|
* Result node -
|
|
|
|
* If no outer plan, evaluate a variable-free targetlist.
|
2002-11-06 01:00:45 +01:00
|
|
|
* If outer plan, return tuples from outer plan (after a level of
|
|
|
|
* projection as shown by targetlist).
|
|
|
|
*
|
|
|
|
* If resconstantqual isn't NULL, it represents a one-time qualification
|
|
|
|
* test (i.e., one that doesn't depend on any variables from the outer plan,
|
|
|
|
* so needs to be evaluated only once).
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Result
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Plan plan;
|
|
|
|
Node *resconstantqual;
|
1997-09-08 23:56:23 +02:00
|
|
|
} Result;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2009-10-10 03:43:50 +02:00
|
|
|
/* ----------------
|
|
|
|
* ModifyTable node -
|
|
|
|
* Apply rows produced by subplan(s) to result table(s),
|
|
|
|
* by inserting, updating, or deleting.
|
2011-01-13 02:47:02 +01:00
|
|
|
*
|
|
|
|
* Note that rowMarks and epqParam are presumed to be valid for all the
|
|
|
|
* subplan(s); they can't contain any info that varies across subplans.
|
2009-10-10 03:43:50 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct ModifyTable
|
|
|
|
{
|
|
|
|
Plan plan;
|
2010-02-26 03:01:40 +01:00
|
|
|
CmdType operation; /* INSERT, UPDATE, or DELETE */
|
2011-02-26 00:56:23 +01:00
|
|
|
bool canSetTag; /* do we set the command tag/es_processed? */
|
2015-02-18 00:04:11 +01:00
|
|
|
Index nominalRelation; /* Parent RT index for use of EXPLAIN */
|
2009-10-10 03:43:50 +02:00
|
|
|
List *resultRelations; /* integer list of RT indexes */
|
2011-04-10 17:42:00 +02:00
|
|
|
int resultRelIndex; /* index of first resultRel in plan's list */
|
2010-02-26 03:01:40 +01:00
|
|
|
List *plans; /* plan(s) producing source data */
|
2014-05-06 18:12:18 +02:00
|
|
|
List *withCheckOptionLists; /* per-target-table WCO lists */
|
2010-02-26 03:01:40 +01:00
|
|
|
List *returningLists; /* per-target-table RETURNING tlists */
|
2013-03-10 19:14:53 +01:00
|
|
|
List *fdwPrivLists; /* per-target-table FDW private data lists */
|
2016-03-18 18:48:58 +01:00
|
|
|
Bitmapset *fdwDirectModifyPlans; /* indices of FDW DM plans */
|
2010-02-26 03:01:40 +01:00
|
|
|
List *rowMarks; /* PlanRowMarks (non-locking only) */
|
|
|
|
int epqParam; /* ID of Param for EvalPlanQual re-eval */
|
2015-05-24 03:35:49 +02:00
|
|
|
OnConflictAction onConflictAction; /* ON CONFLICT action */
|
|
|
|
List *arbiterIndexes; /* List of ON CONFLICT arbiter index OIDs */
|
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
|
|
|
List *onConflictSet; /* SET for INSERT ON CONFLICT DO UPDATE */
|
2015-05-24 03:35:49 +02:00
|
|
|
Node *onConflictWhere; /* WHERE for ON CONFLICT UPDATE */
|
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE.
The newly added ON CONFLICT clause allows to specify an alternative to
raising a unique or exclusion constraint violation error when inserting.
ON CONFLICT refers to constraints that can either be specified using a
inference clause (by specifying the columns of a unique constraint) or
by naming a unique or exclusion constraint. DO NOTHING avoids the
constraint violation, without touching the pre-existing row. DO UPDATE
SET ... [WHERE ...] updates the pre-existing tuple, and has access to
both the tuple proposed for insertion and the existing tuple; the
optional WHERE clause can be used to prevent an update from being
executed. The UPDATE SET and WHERE clauses have access to the tuple
proposed for insertion using the "magic" EXCLUDED alias, and to the
pre-existing tuple using the table name or its alias.
This feature is often referred to as upsert.
This is implemented using a new infrastructure called "speculative
insertion". It is an optimistic variant of regular insertion that first
does a pre-check for existing tuples and then attempts an insert. If a
violating tuple was inserted concurrently, the speculatively inserted
tuple is deleted and a new attempt is made. If the pre-check finds a
matching tuple the alternative DO NOTHING or DO UPDATE action is taken.
If the insertion succeeds without detecting a conflict, the tuple is
deemed inserted.
To handle the possible ambiguity between the excluded alias and a table
named excluded, and for convenience with long relation names, INSERT
INTO now can alias its target table.
Bumps catversion as stored rules change.
Author: Peter Geoghegan, with significant contributions from Heikki
Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes.
Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs,
Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
|
|
|
Index exclRelRTI; /* RTI of the EXCLUDED pseudo relation */
|
|
|
|
List *exclRelTlist; /* tlist of the EXCLUDED pseudo relation */
|
2009-10-10 03:43:50 +02:00
|
|
|
} ModifyTable;
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ----------------
|
2000-11-12 01:37:02 +01:00
|
|
|
* Append node -
|
|
|
|
* Generate the concatenation of the results of sub-plans.
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Append
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Plan plan;
|
1998-07-15 16:54:39 +02:00
|
|
|
List *appendplans;
|
1997-09-08 22:59:27 +02:00
|
|
|
} Append;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2010-10-14 22:56:39 +02:00
|
|
|
/* ----------------
|
|
|
|
* MergeAppend node -
|
|
|
|
* Merge the results of pre-sorted sub-plans to preserve the ordering.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct MergeAppend
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
List *mergeplans;
|
|
|
|
/* remaining fields are just like the sort-key info in struct Sort */
|
|
|
|
int numCols; /* number of sort-key columns */
|
|
|
|
AttrNumber *sortColIdx; /* their indexes in the target list */
|
|
|
|
Oid *sortOperators; /* OIDs of operators to sort them by */
|
2011-02-08 22:04:18 +01:00
|
|
|
Oid *collations; /* OIDs of collations */
|
2010-10-14 22:56:39 +02:00
|
|
|
bool *nullsFirst; /* NULLS FIRST/LAST directions */
|
|
|
|
} MergeAppend;
|
|
|
|
|
2008-10-04 23:56:55 +02:00
|
|
|
/* ----------------
|
|
|
|
* RecursiveUnion node -
|
|
|
|
* Generate a recursive union of two subplans.
|
|
|
|
*
|
|
|
|
* The "outer" subplan is always the non-recursive term, and the "inner"
|
|
|
|
* subplan is the recursive term.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct RecursiveUnion
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
int wtParam; /* ID of Param representing work table */
|
2008-10-07 21:27:04 +02:00
|
|
|
/* Remaining fields are zero/null in UNION ALL case */
|
|
|
|
int numCols; /* number of columns to check for
|
|
|
|
* duplicate-ness */
|
|
|
|
AttrNumber *dupColIdx; /* their indexes in the target list */
|
|
|
|
Oid *dupOperators; /* equality operators to compare with */
|
|
|
|
long numGroups; /* estimated number of groups in input */
|
2008-10-04 23:56:55 +02:00
|
|
|
} RecursiveUnion;
|
|
|
|
|
2005-04-20 00:35:18 +02:00
|
|
|
/* ----------------
|
|
|
|
* BitmapAnd node -
|
|
|
|
* Generate the intersection of the results of sub-plans.
|
|
|
|
*
|
2014-05-06 18:12:18 +02:00
|
|
|
* The subplans must be of types that yield tuple bitmaps. The targetlist
|
2005-04-20 00:35:18 +02:00
|
|
|
* and qual fields of the plan are unused and are always NIL.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct BitmapAnd
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
List *bitmapplans;
|
|
|
|
} BitmapAnd;
|
|
|
|
|
|
|
|
/* ----------------
|
|
|
|
* BitmapOr node -
|
|
|
|
* Generate the union of the results of sub-plans.
|
|
|
|
*
|
2014-05-06 18:12:18 +02:00
|
|
|
* The subplans must be of types that yield tuple bitmaps. The targetlist
|
2005-04-20 00:35:18 +02:00
|
|
|
* and qual fields of the plan are unused and are always NIL.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct BitmapOr
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
List *bitmapplans;
|
|
|
|
} BitmapOr;
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/*
|
|
|
|
* ==========
|
|
|
|
* Scan nodes
|
|
|
|
* ==========
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Scan
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Plan plan;
|
|
|
|
Index scanrelid; /* relid is index into the range table */
|
1997-09-08 23:56:23 +02:00
|
|
|
} Scan;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* sequential scan node
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-08 04:41:22 +02:00
|
|
|
typedef Scan SeqScan;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2015-05-15 20:37:10 +02:00
|
|
|
/* ----------------
|
|
|
|
* table sample scan node
|
|
|
|
* ----------------
|
|
|
|
*/
|
Redesign tablesample method API, and do extensive code review.
The original implementation of TABLESAMPLE modeled the tablesample method
API on index access methods, which wasn't a good choice because, without
specialized DDL commands, there's no way to build an extension that can
implement a TSM. (Raw inserts into system catalogs are not an acceptable
thing to do, because we can't undo them during DROP EXTENSION, nor will
pg_upgrade behave sanely.) Instead adopt an API more like procedural
language handlers or foreign data wrappers, wherein the only SQL-level
support object needed is a single handler function identified by having
a special return type. This lets us get rid of the supporting catalog
altogether, so that no custom DDL support is needed for the feature.
Adjust the API so that it can support non-constant tablesample arguments
(the original coding assumed we could evaluate the argument expressions at
ExecInitSampleScan time, which is undesirable even if it weren't outright
unsafe), and discourage sampling methods from looking at invisible tuples.
Make sure that the BERNOULLI and SYSTEM methods are genuinely repeatable
within and across queries, as required by the SQL standard, and deal more
honestly with methods that can't support that requirement.
Make a full code-review pass over the tablesample additions, and fix
assorted bugs, omissions, infelicities, and cosmetic issues (such as
failure to put the added code stanzas in a consistent ordering).
Improve EXPLAIN's output of tablesample plans, too.
Back-patch to 9.5 so that we don't have to support the original API
in production.
2015-07-25 20:39:00 +02:00
|
|
|
typedef struct SampleScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
/* use struct pointer to avoid including parsenodes.h here */
|
|
|
|
struct TableSampleClause *tablesample;
|
|
|
|
} SampleScan;
|
2015-05-15 20:37:10 +02:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* index scan node
|
2003-11-09 22:30:38 +01:00
|
|
|
*
|
2005-04-25 03:30:14 +02:00
|
|
|
* indexqualorig is an implicitly-ANDed list of index qual expressions, each
|
|
|
|
* in the same form it appeared in the query WHERE condition. Each should
|
|
|
|
* be of the form (indexkey OP comparisonval) or (comparisonval OP indexkey).
|
|
|
|
* The indexkey is a Var or expression referencing column(s) of the index's
|
2014-05-06 18:12:18 +02:00
|
|
|
* base table. The comparisonval might be any expression, but it won't use
|
2010-12-03 02:50:48 +01:00
|
|
|
* any columns of the base table. The expressions are ordered by index
|
|
|
|
* column position (but items referencing the same index column can appear
|
|
|
|
* in any order). indexqualorig is used at runtime only if we have to recheck
|
|
|
|
* a lossy indexqual.
|
2005-04-25 03:30:14 +02:00
|
|
|
*
|
|
|
|
* indexqual has the same form, but the expressions have been commuted if
|
|
|
|
* necessary to put the indexkeys on the left, and the indexkeys are replaced
|
2011-10-11 20:20:06 +02:00
|
|
|
* by Var nodes identifying the index columns (their varno is INDEX_VAR and
|
|
|
|
* their varattno is the index column number).
|
2010-12-03 02:50:48 +01:00
|
|
|
*
|
|
|
|
* indexorderbyorig is similarly the original form of any ORDER BY expressions
|
|
|
|
* that are being implemented by the index, while indexorderby is modified to
|
|
|
|
* have index column Vars on the left-hand side. Here, multiple expressions
|
|
|
|
* must appear in exactly the ORDER BY order, and this is not necessarily the
|
2014-05-06 18:12:18 +02:00
|
|
|
* index column order. Only the expressions are provided, not the auxiliary
|
2010-12-03 02:50:48 +01:00
|
|
|
* sort-order information from the ORDER BY SortGroupClauses; it's assumed
|
|
|
|
* that the sort ordering is fully determinable from the top-level operators.
|
2015-05-15 13:26:51 +02:00
|
|
|
* indexorderbyorig is used at runtime to recheck the ordering, if the index
|
|
|
|
* cannot calculate an accurate ordering. It is also needed for EXPLAIN.
|
|
|
|
*
|
2015-05-18 03:22:12 +02:00
|
|
|
* indexorderbyops is a list of the OIDs of the operators used to sort the
|
|
|
|
* ORDER BY expressions. This is used together with indexorderbyorig to
|
|
|
|
* recheck ordering at run time. (Note that indexorderby, indexorderbyorig,
|
|
|
|
* and indexorderbyops are used for amcanorderbyop cases, not amcanorder.)
|
2011-10-08 02:13:02 +02:00
|
|
|
*
|
|
|
|
* indexorderdir specifies the scan ordering, for indexscans on amcanorder
|
2011-10-11 20:20:06 +02:00
|
|
|
* indexes (for other indexes it should be "don't care").
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct IndexScan
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Scan scan;
|
2005-10-15 04:49:52 +02:00
|
|
|
Oid indexid; /* OID of index to scan */
|
2010-12-03 02:50:48 +01:00
|
|
|
List *indexqual; /* list of index quals (usually OpExprs) */
|
2005-10-15 04:49:52 +02:00
|
|
|
List *indexqualorig; /* the same in original form */
|
2011-04-10 17:42:00 +02:00
|
|
|
List *indexorderby; /* list of index ORDER BY exprs */
|
|
|
|
List *indexorderbyorig; /* the same in original form */
|
2015-05-18 03:22:12 +02:00
|
|
|
List *indexorderbyops; /* OIDs of sort ops for ORDER BY exprs */
|
2005-04-25 03:30:14 +02:00
|
|
|
ScanDirection indexorderdir; /* forward or backward or don't care */
|
1997-09-08 23:56:23 +02:00
|
|
|
} IndexScan;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2011-10-11 20:20:06 +02:00
|
|
|
/* ----------------
|
|
|
|
* index-only scan node
|
|
|
|
*
|
|
|
|
* IndexOnlyScan is very similar to IndexScan, but it specifies an
|
|
|
|
* index-only scan, in which the data comes from the index not the heap.
|
|
|
|
* Because of this, *all* Vars in the plan node's targetlist, qual, and
|
|
|
|
* index expressions reference index columns and have varno = INDEX_VAR.
|
|
|
|
* Hence we do not need separate indexqualorig and indexorderbyorig lists,
|
|
|
|
* since their contents would be equivalent to indexqual and indexorderby.
|
|
|
|
*
|
|
|
|
* To help EXPLAIN interpret the index Vars for display, we provide
|
|
|
|
* indextlist, which represents the contents of the index as a targetlist
|
|
|
|
* with one TLE per index column. Vars appearing in this list reference
|
|
|
|
* the base table, and this is the only field in the plan node that may
|
|
|
|
* contain such Vars.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct IndexOnlyScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
Oid indexid; /* OID of index to scan */
|
|
|
|
List *indexqual; /* list of index quals (usually OpExprs) */
|
|
|
|
List *indexorderby; /* list of index ORDER BY exprs */
|
|
|
|
List *indextlist; /* TargetEntry list describing index's cols */
|
|
|
|
ScanDirection indexorderdir; /* forward or backward or don't care */
|
|
|
|
} IndexOnlyScan;
|
|
|
|
|
1999-11-23 21:07:06 +01:00
|
|
|
/* ----------------
|
2005-04-20 00:35:18 +02:00
|
|
|
* bitmap index scan node
|
|
|
|
*
|
|
|
|
* BitmapIndexScan delivers a bitmap of potential tuple locations;
|
2014-05-06 18:12:18 +02:00
|
|
|
* it does not access the heap itself. The bitmap is used by an
|
2005-04-20 00:35:18 +02:00
|
|
|
* ancestor BitmapHeapScan node, possibly after passing through
|
|
|
|
* intermediate BitmapAnd and/or BitmapOr nodes to combine it with
|
|
|
|
* the results of other BitmapIndexScans.
|
|
|
|
*
|
2005-04-25 03:30:14 +02:00
|
|
|
* The fields have the same meanings as for IndexScan, except we don't
|
|
|
|
* store a direction flag because direction is uninteresting.
|
|
|
|
*
|
2005-04-20 00:35:18 +02:00
|
|
|
* In a BitmapIndexScan plan node, the targetlist and qual fields are
|
2005-04-25 03:30:14 +02:00
|
|
|
* not used and are always NIL. The indexqualorig field is unused at
|
2005-04-20 00:35:18 +02:00
|
|
|
* run time too, but is saved for the benefit of EXPLAIN.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct BitmapIndexScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
2005-10-15 04:49:52 +02:00
|
|
|
Oid indexid; /* OID of index to scan */
|
|
|
|
List *indexqual; /* list of index quals (OpExprs) */
|
|
|
|
List *indexqualorig; /* the same in original form */
|
2005-04-20 00:35:18 +02:00
|
|
|
} BitmapIndexScan;
|
|
|
|
|
|
|
|
/* ----------------
|
|
|
|
* bitmap sequential scan node
|
|
|
|
*
|
|
|
|
* This needs a copy of the qual conditions being used by the input index
|
|
|
|
* scans because there are various cases where we need to recheck the quals;
|
|
|
|
* for example, when the bitmap is lossy about the specific rows on a page
|
|
|
|
* that meet the index condition.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct BitmapHeapScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
2005-10-15 04:49:52 +02:00
|
|
|
List *bitmapqualorig; /* index quals, in standard expr form */
|
2005-04-20 00:35:18 +02:00
|
|
|
} BitmapHeapScan;
|
|
|
|
|
|
|
|
/* ----------------
|
|
|
|
* tid scan node
|
2005-11-26 23:14:57 +01:00
|
|
|
*
|
|
|
|
* tidquals is an implicitly OR'ed list of qual expressions of the form
|
|
|
|
* "CTID = pseudoconstant" or "CTID = ANY(pseudoconstant_array)".
|
1999-11-23 21:07:06 +01:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct TidScan
|
|
|
|
{
|
2000-01-09 01:26:47 +01:00
|
|
|
Scan scan;
|
2005-11-26 23:14:57 +01:00
|
|
|
List *tidquals; /* qual(s) involving CTID = something */
|
1999-11-23 21:07:06 +01:00
|
|
|
} TidScan;
|
|
|
|
|
2000-09-29 20:21:41 +02:00
|
|
|
/* ----------------
|
|
|
|
* subquery scan node
|
|
|
|
*
|
|
|
|
* SubqueryScan is for scanning the output of a sub-query in the range table.
|
2007-02-19 03:23:12 +01:00
|
|
|
* We often need an extra plan node above the sub-query's plan to perform
|
|
|
|
* expression evaluations (which we can't push into the sub-query without
|
|
|
|
* risking changing its semantics). Although we are not scanning a physical
|
|
|
|
* relation, we make this a descendant of Scan anyway for code-sharing
|
|
|
|
* purposes.
|
2000-09-29 20:21:41 +02:00
|
|
|
*
|
|
|
|
* Note: we store the sub-plan in the type-specific subplan field, not in
|
2014-05-06 18:12:18 +02:00
|
|
|
* the generic lefttree field as you might expect. This is because we do
|
2000-09-29 20:21:41 +02:00
|
|
|
* not want plan-tree-traversal routines to recurse into the subplan without
|
|
|
|
* knowing that they are changing Query contexts.
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct SubqueryScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
Plan *subplan;
|
|
|
|
} SubqueryScan;
|
|
|
|
|
2002-05-12 22:10:05 +02:00
|
|
|
/* ----------------
|
|
|
|
* FunctionScan node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct FunctionScan
|
|
|
|
{
|
2002-09-04 22:31:48 +02:00
|
|
|
Scan scan;
|
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions.
This patch adds the ability to write TABLE( function1(), function2(), ...)
as a single FROM-clause entry. The result is the concatenation of the
first row from each function, followed by the second row from each
function, etc; with NULLs inserted if any function produces fewer rows than
others. This is believed to be a much more useful behavior than what
Postgres currently does with multiple SRFs in a SELECT list.
This syntax also provides a reasonable way to combine use of column
definition lists with WITH ORDINALITY: put the column definition list
inside TABLE(), where it's clear that it doesn't control the ordinality
column as well.
Also implement SQL-compliant multiple-argument UNNEST(), by turning
UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)).
The SQL standard specifies TABLE() with only a single function, not
multiple functions, and it seems to require an implicit UNNEST() which is
not what this patch does. There may be something wrong with that reading
of the spec, though, because if it's right then the spec's TABLE() is just
a pointless alternative spelling of UNNEST(). After further review of
that, we might choose to adopt a different syntax for what this patch does,
but in any case this functionality seems clearly worthwhile.
Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and
significantly revised by me
2013-11-22 01:37:02 +01:00
|
|
|
List *functions; /* list of RangeTblFunction nodes */
|
|
|
|
bool funcordinality; /* WITH ORDINALITY */
|
2002-05-12 22:10:05 +02:00
|
|
|
} FunctionScan;
|
|
|
|
|
2006-08-02 03:59:48 +02:00
|
|
|
/* ----------------
|
|
|
|
* ValuesScan node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct ValuesScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
2007-02-19 03:23:12 +01:00
|
|
|
List *values_lists; /* list of expression lists */
|
2006-08-02 03:59:48 +02:00
|
|
|
} ValuesScan;
|
|
|
|
|
2008-10-04 23:56:55 +02:00
|
|
|
/* ----------------
|
|
|
|
* CteScan node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct CteScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
int ctePlanId; /* ID of init SubPlan for CTE */
|
|
|
|
int cteParam; /* ID of Param representing CTE output */
|
|
|
|
} CteScan;
|
|
|
|
|
|
|
|
/* ----------------
|
|
|
|
* WorkTableScan node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct WorkTableScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
int wtParam; /* ID of Param representing work table */
|
|
|
|
} WorkTableScan;
|
|
|
|
|
2011-02-20 06:17:18 +01:00
|
|
|
/* ----------------
|
|
|
|
* ForeignScan node
|
Revise FDW planning API, again.
Further reflection shows that a single callback isn't very workable if we
desire to let FDWs generate multiple Paths, because that forces the FDW to
do all work necessary to generate a valid Plan node for each Path. Instead
split the former PlanForeignScan API into three steps: GetForeignRelSize,
GetForeignPaths, GetForeignPlan. We had already bit the bullet of breaking
the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain,
and it's substantially more flexible for complex FDWs.
Add an fdw_private field to RelOptInfo so that the new functions can save
state there rather than possibly having to recalculate information two or
three times.
In addition, we'd not thought through what would be needed to allow an FDW
to set up subexpressions of its choice for runtime execution. We could
treat ForeignScan.fdw_private as an executable expression but that seems
likely to break existing FDWs unnecessarily (in particular, it would
restrict the set of node types allowable in fdw_private to those supported
by expression_tree_walker). Instead, invent a separate field fdw_exprs
which will receive the postprocessing appropriate for expression trees.
(One field is enough since it can be a list of expressions; also, we assume
the corresponding expression state tree(s) will be held within fdw_state,
so we don't need to add anything to ForeignScanState.)
Per review of Hanada Shigeru's pgsql_fdw patch. We may need to tweak this
further as we continue to work on that patch, but to me it feels a lot
closer to being right now.
2012-03-09 18:48:48 +01:00
|
|
|
*
|
|
|
|
* fdw_exprs and fdw_private are both under the control of the foreign-data
|
|
|
|
* wrapper, but fdw_exprs is presumed to contain expression trees and will
|
|
|
|
* be post-processed accordingly by the planner; fdw_private won't be.
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
* Note that everything in both lists must be copiable by copyObject().
|
Revise FDW planning API, again.
Further reflection shows that a single callback isn't very workable if we
desire to let FDWs generate multiple Paths, because that forces the FDW to
do all work necessary to generate a valid Plan node for each Path. Instead
split the former PlanForeignScan API into three steps: GetForeignRelSize,
GetForeignPaths, GetForeignPlan. We had already bit the bullet of breaking
the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain,
and it's substantially more flexible for complex FDWs.
Add an fdw_private field to RelOptInfo so that the new functions can save
state there rather than possibly having to recalculate information two or
three times.
In addition, we'd not thought through what would be needed to allow an FDW
to set up subexpressions of its choice for runtime execution. We could
treat ForeignScan.fdw_private as an executable expression but that seems
likely to break existing FDWs unnecessarily (in particular, it would
restrict the set of node types allowable in fdw_private to those supported
by expression_tree_walker). Instead, invent a separate field fdw_exprs
which will receive the postprocessing appropriate for expression trees.
(One field is enough since it can be a list of expressions; also, we assume
the corresponding expression state tree(s) will be held within fdw_state,
so we don't need to add anything to ForeignScanState.)
Per review of Hanada Shigeru's pgsql_fdw patch. We may need to tweak this
further as we continue to work on that patch, but to me it feels a lot
closer to being right now.
2012-03-09 18:48:48 +01:00
|
|
|
* One way to store an arbitrary blob of bytes is to represent it as a bytea
|
|
|
|
* Const. Usually, though, you'll be better off choosing a representation
|
|
|
|
* that can be dumped usefully by nodeToString().
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
*
|
|
|
|
* fdw_scan_tlist is a targetlist describing the contents of the scan tuple
|
|
|
|
* returned by the FDW; it can be NIL if the scan tuple matches the declared
|
|
|
|
* rowtype of the foreign table, which is the normal case for a simple foreign
|
|
|
|
* table scan. (If the plan node represents a foreign join, fdw_scan_tlist
|
|
|
|
* is required since there is no rowtype available from the system catalogs.)
|
|
|
|
* When fdw_scan_tlist is provided, Vars in the node's tlist and quals must
|
|
|
|
* have varno INDEX_VAR, and their varattnos correspond to resnos in the
|
|
|
|
* fdw_scan_tlist (which are also column numbers in the actual scan tuple).
|
|
|
|
* fdw_scan_tlist is never actually executed; it just holds expression trees
|
|
|
|
* describing what is in the scan tuple's columns.
|
|
|
|
*
|
2015-10-15 19:00:40 +02:00
|
|
|
* fdw_recheck_quals should contain any quals which the core system passed to
|
2015-10-20 17:11:35 +02:00
|
|
|
* the FDW but which were not added to scan.plan.qual; that is, it should
|
2015-10-15 19:00:40 +02:00
|
|
|
* contain the quals being checked remotely. This is needed for correct
|
|
|
|
* behavior during EvalPlanQual rechecks.
|
|
|
|
*
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
* When the plan node represents a foreign join, scan.scanrelid is zero and
|
|
|
|
* fs_relids must be consulted to identify the join relation. (fs_relids
|
|
|
|
* is valid for simple scans as well, but will always match scan.scanrelid.)
|
2011-02-20 06:17:18 +01:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct ForeignScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
2016-03-18 18:48:58 +01:00
|
|
|
CmdType operation; /* SELECT/INSERT/UPDATE/DELETE */
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
Oid fs_server; /* OID of foreign server */
|
Revise FDW planning API, again.
Further reflection shows that a single callback isn't very workable if we
desire to let FDWs generate multiple Paths, because that forces the FDW to
do all work necessary to generate a valid Plan node for each Path. Instead
split the former PlanForeignScan API into three steps: GetForeignRelSize,
GetForeignPaths, GetForeignPlan. We had already bit the bullet of breaking
the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain,
and it's substantially more flexible for complex FDWs.
Add an fdw_private field to RelOptInfo so that the new functions can save
state there rather than possibly having to recalculate information two or
three times.
In addition, we'd not thought through what would be needed to allow an FDW
to set up subexpressions of its choice for runtime execution. We could
treat ForeignScan.fdw_private as an executable expression but that seems
likely to break existing FDWs unnecessarily (in particular, it would
restrict the set of node types allowable in fdw_private to those supported
by expression_tree_walker). Instead, invent a separate field fdw_exprs
which will receive the postprocessing appropriate for expression trees.
(One field is enough since it can be a list of expressions; also, we assume
the corresponding expression state tree(s) will be held within fdw_state,
so we don't need to add anything to ForeignScanState.)
Per review of Hanada Shigeru's pgsql_fdw patch. We may need to tweak this
further as we continue to work on that patch, but to me it feels a lot
closer to being right now.
2012-03-09 18:48:48 +01:00
|
|
|
List *fdw_exprs; /* expressions that FDW may evaluate */
|
2012-03-05 22:15:59 +01:00
|
|
|
List *fdw_private; /* private data for FDW */
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
List *fdw_scan_tlist; /* optional tlist describing scan tuple */
|
2016-06-10 00:02:36 +02:00
|
|
|
List *fdw_recheck_quals; /* original quals not in
|
|
|
|
* scan.plan.qual */
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
Bitmapset *fs_relids; /* RTIs generated by this scan */
|
Revise FDW planning API, again.
Further reflection shows that a single callback isn't very workable if we
desire to let FDWs generate multiple Paths, because that forces the FDW to
do all work necessary to generate a valid Plan node for each Path. Instead
split the former PlanForeignScan API into three steps: GetForeignRelSize,
GetForeignPaths, GetForeignPlan. We had already bit the bullet of breaking
the 9.1 FDW API for 9.2, so this shouldn't cause very much additional pain,
and it's substantially more flexible for complex FDWs.
Add an fdw_private field to RelOptInfo so that the new functions can save
state there rather than possibly having to recalculate information two or
three times.
In addition, we'd not thought through what would be needed to allow an FDW
to set up subexpressions of its choice for runtime execution. We could
treat ForeignScan.fdw_private as an executable expression but that seems
likely to break existing FDWs unnecessarily (in particular, it would
restrict the set of node types allowable in fdw_private to those supported
by expression_tree_walker). Instead, invent a separate field fdw_exprs
which will receive the postprocessing appropriate for expression trees.
(One field is enough since it can be a list of expressions; also, we assume
the corresponding expression state tree(s) will be held within fdw_state,
so we don't need to add anything to ForeignScanState.)
Per review of Hanada Shigeru's pgsql_fdw patch. We may need to tweak this
further as we continue to work on that patch, but to me it feels a lot
closer to being right now.
2012-03-09 18:48:48 +01:00
|
|
|
bool fsSystemCol; /* true if any "system column" is needed */
|
2011-02-20 06:17:18 +01:00
|
|
|
} ForeignScan;
|
|
|
|
|
2014-11-07 23:26:02 +01:00
|
|
|
/* ----------------
|
2014-11-21 00:36:07 +01:00
|
|
|
* CustomScan node
|
2014-11-22 00:21:46 +01:00
|
|
|
*
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
* The comments for ForeignScan's fdw_exprs, fdw_private, fdw_scan_tlist,
|
|
|
|
* and fs_relids fields apply equally to CustomScan's custom_exprs,
|
|
|
|
* custom_private, custom_scan_tlist, and custom_relids fields. The
|
|
|
|
* convention of setting scan.scanrelid to zero for joins applies as well.
|
|
|
|
*
|
2015-05-01 14:50:35 +02:00
|
|
|
* Note that since Plan trees can be copied, custom scan providers *must*
|
|
|
|
* fit all plan data they need into those fields; embedding CustomScan in
|
|
|
|
* a larger struct will not work.
|
2014-11-07 23:26:02 +01:00
|
|
|
* ----------------
|
|
|
|
*/
|
2016-03-29 17:00:18 +02:00
|
|
|
struct CustomScanMethods;
|
2008-10-04 23:56:55 +02:00
|
|
|
|
2014-11-21 00:36:07 +01:00
|
|
|
typedef struct CustomScan
|
|
|
|
{
|
|
|
|
Scan scan;
|
|
|
|
uint32 flags; /* mask of CUSTOMPATH_* flags, see relation.h */
|
2015-06-26 15:40:47 +02:00
|
|
|
List *custom_plans; /* list of Plan nodes, if any */
|
2014-11-22 00:21:46 +01:00
|
|
|
List *custom_exprs; /* expressions that custom code may evaluate */
|
|
|
|
List *custom_private; /* private data for custom code */
|
Code review for foreign/custom join pushdown patch.
Commit e7cb7ee14555cc9c5773e2c102efd6371f6f2005 included some design
decisions that seem pretty questionable to me, and there was quite a lot
of stuff not to like about the documentation and comments. Clean up
as follows:
* Consider foreign joins only between foreign tables on the same server,
rather than between any two foreign tables with the same underlying FDW
handler function. In most if not all cases, the FDW would simply have had
to apply the same-server restriction itself (far more expensively, both for
lack of caching and because it would be repeated for each combination of
input sub-joins), or else risk nasty bugs. Anyone who's really intent on
doing something outside this restriction can always use the
set_join_pathlist_hook.
* Rename fdw_ps_tlist/custom_ps_tlist to fdw_scan_tlist/custom_scan_tlist
to better reflect what they're for, and allow these custom scan tlists
to be used even for base relations.
* Change make_foreignscan() API to include passing the fdw_scan_tlist
value, since the FDW is required to set that. Backwards compatibility
doesn't seem like an adequate reason to expect FDWs to set it in some
ad-hoc extra step, and anyway existing FDWs can just pass NIL.
* Change the API of path-generating subroutines of add_paths_to_joinrel,
and in particular that of GetForeignJoinPaths and set_join_pathlist_hook,
so that various less-used parameters are passed in a struct rather than
as separate parameter-list entries. The objective here is to reduce the
probability that future additions to those parameter lists will result in
source-level API breaks for users of these hooks. It's possible that this
is even a small win for the core code, since most CPU architectures can't
pass more than half a dozen parameters efficiently anyway. I kept root,
joinrel, outerrel, innerrel, and jointype as separate parameters to reduce
code churn in joinpath.c --- in particular, putting jointype into the
struct would have been problematic because of the subroutines' habit of
changing their local copies of that variable.
* Avoid ad-hocery in ExecAssignScanProjectionInfo. It was probably all
right for it to know about IndexOnlyScan, but if the list is to grow
we should refactor the knowledge out to the callers.
* Restore nodeForeignscan.c's previous use of the relcache to avoid
extra GetFdwRoutine lookups for base-relation scans.
* Lots of cleanup of documentation and missed comments. Re-order some
code additions into more logical places.
2015-05-10 20:36:30 +02:00
|
|
|
List *custom_scan_tlist; /* optional tlist describing scan
|
|
|
|
* tuple */
|
2015-05-01 14:50:35 +02:00
|
|
|
Bitmapset *custom_relids; /* RTIs generated by this scan */
|
2016-03-29 17:00:18 +02:00
|
|
|
const struct CustomScanMethods *methods;
|
2014-11-21 00:36:07 +01:00
|
|
|
} CustomScan;
|
|
|
|
|
2000-04-12 19:17:23 +02:00
|
|
|
/*
|
1996-08-28 03:59:28 +02:00
|
|
|
* ==========
|
|
|
|
* Join nodes
|
|
|
|
* ==========
|
|
|
|
*/
|
|
|
|
|
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* Join node
|
2000-09-12 23:07:18 +02:00
|
|
|
*
|
|
|
|
* jointype: rule for joining tuples from left and right subtrees
|
|
|
|
* joinqual: qual conditions that came from JOIN/ON or JOIN/USING
|
|
|
|
* (plan.qual contains conditions that came from WHERE)
|
|
|
|
*
|
|
|
|
* When jointype is INNER, joinqual and plan.qual are semantically
|
|
|
|
* interchangeable. For OUTER jointypes, the two are *not* interchangeable;
|
|
|
|
* only joinqual is used to determine whether a match has been found for
|
|
|
|
* the purpose of deciding whether to generate null-extended tuples.
|
|
|
|
* (But plan.qual is still applied before actually returning a tuple.)
|
|
|
|
* For an outer join, only joinquals are allowed to be used as the merge
|
|
|
|
* or hash condition of a merge or hash join.
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
2000-09-12 23:07:18 +02:00
|
|
|
typedef struct Join
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
JoinType jointype;
|
|
|
|
List *joinqual; /* JOIN quals (in addition to plan.qual) */
|
|
|
|
} Join;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* nest loop join node
|
2010-07-12 19:01:06 +02:00
|
|
|
*
|
|
|
|
* The nestParams list identifies any executor Params that must be passed
|
|
|
|
* into execution of the inner subplan carrying values from the current row
|
|
|
|
* of the outer subplan. Currently we restrict these values to be simple
|
2011-11-03 05:50:58 +01:00
|
|
|
* Vars, but perhaps someday that'd be worth relaxing. (Note: during plan
|
|
|
|
* creation, the paramval can actually be a PlaceHolderVar expression; but it
|
|
|
|
* must be a Var with varno OUTER_VAR by the time it gets to the executor.)
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct NestLoop
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Join join;
|
2010-07-12 19:01:06 +02:00
|
|
|
List *nestParams; /* list of NestLoopParam nodes */
|
1997-09-08 23:56:23 +02:00
|
|
|
} NestLoop;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2010-07-12 19:01:06 +02:00
|
|
|
typedef struct NestLoopParam
|
|
|
|
{
|
|
|
|
NodeTag type;
|
|
|
|
int paramno; /* number of the PARAM_EXEC Param to set */
|
|
|
|
Var *paramval; /* outer-relation Var to assign to Param */
|
|
|
|
} NestLoopParam;
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* merge join node
|
2007-01-10 19:06:05 +01:00
|
|
|
*
|
|
|
|
* The expected ordering of each mergeable column is described by a btree
|
2011-03-26 21:35:25 +01:00
|
|
|
* opfamily OID, a collation OID, a direction (BTLessStrategyNumber or
|
|
|
|
* BTGreaterStrategyNumber) and a nulls-first flag. Note that the two sides
|
|
|
|
* of each mergeclause may be of different datatypes, but they are ordered the
|
|
|
|
* same way according to the common opfamily and collation. The operator in
|
|
|
|
* each mergeclause must be an equality operator of the indicated opfamily.
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct MergeJoin
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Join join;
|
2007-11-15 22:14:46 +01:00
|
|
|
List *mergeclauses; /* mergeclauses as expression trees */
|
2007-01-10 19:06:05 +01:00
|
|
|
/* these are arrays, but have the same length as the mergeclauses list: */
|
2007-11-15 22:14:46 +01:00
|
|
|
Oid *mergeFamilies; /* per-clause OIDs of btree opfamilies */
|
2011-02-08 22:04:18 +01:00
|
|
|
Oid *mergeCollations; /* per-clause OIDs of collations */
|
2007-01-10 19:06:05 +01:00
|
|
|
int *mergeStrategies; /* per-clause ordering (ASC or DESC) */
|
|
|
|
bool *mergeNullsFirst; /* per-clause nulls ordering */
|
1997-09-08 23:56:23 +02:00
|
|
|
} MergeJoin;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
/* ----------------
|
2009-03-21 01:04:40 +01:00
|
|
|
* hash join node
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct HashJoin
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Join join;
|
|
|
|
List *hashclauses;
|
1997-09-08 23:56:23 +02:00
|
|
|
} HashJoin;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2002-12-05 16:50:39 +01:00
|
|
|
/* ----------------
|
|
|
|
* materialization node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct Material
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
} Material;
|
|
|
|
|
|
|
|
/* ----------------
|
|
|
|
* sort node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct Sort
|
|
|
|
{
|
|
|
|
Plan plan;
|
2003-05-06 02:20:33 +02:00
|
|
|
int numCols; /* number of sort-key columns */
|
|
|
|
AttrNumber *sortColIdx; /* their indexes in the target list */
|
|
|
|
Oid *sortOperators; /* OIDs of operators to sort them by */
|
2011-02-08 22:04:18 +01:00
|
|
|
Oid *collations; /* OIDs of collations */
|
2007-01-09 03:14:16 +01:00
|
|
|
bool *nullsFirst; /* NULLS FIRST/LAST directions */
|
2002-12-05 16:50:39 +01:00
|
|
|
} Sort;
|
|
|
|
|
|
|
|
/* ---------------
|
|
|
|
* group node -
|
|
|
|
* Used for queries with GROUP BY (but no aggregates) specified.
|
|
|
|
* The input must be presorted according to the grouping columns.
|
|
|
|
* ---------------
|
|
|
|
*/
|
|
|
|
typedef struct Group
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
int numCols; /* number of grouping columns */
|
|
|
|
AttrNumber *grpColIdx; /* their indexes in the target list */
|
2007-01-10 19:06:05 +01:00
|
|
|
Oid *grpOperators; /* equality operators to compare with */
|
2002-12-05 16:50:39 +01:00
|
|
|
} Group;
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ---------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* aggregate node
|
2002-11-06 01:00:45 +01:00
|
|
|
*
|
|
|
|
* An Agg node implements plain or grouped aggregation. For grouped
|
|
|
|
* aggregation, we can work with presorted input or unsorted input;
|
|
|
|
* the latter strategy uses an internal hashtable.
|
|
|
|
*
|
|
|
|
* Notice the lack of any direct info about the aggregate functions to be
|
|
|
|
* computed. They are found by scanning the node's tlist and quals during
|
|
|
|
* executor startup. (It is possible that there are no aggregate functions;
|
|
|
|
* this could happen if they get optimized away by constant-folding, or if
|
|
|
|
* we are using the Agg node to implement hash-based grouping.)
|
1996-08-28 03:59:28 +02:00
|
|
|
* ---------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Agg
|
|
|
|
{
|
1997-09-08 04:41:22 +02:00
|
|
|
Plan plan;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
2016-03-07 21:58:22 +01:00
|
|
|
AggStrategy aggstrategy; /* basic strategy, see nodes.h */
|
2016-06-26 20:33:38 +02:00
|
|
|
AggSplit aggsplit; /* agg-splitting mode, see nodes.h */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
2016-03-07 21:58:22 +01:00
|
|
|
int numCols; /* number of grouping columns */
|
|
|
|
AttrNumber *grpColIdx; /* their indexes in the target list */
|
2007-01-10 19:06:05 +01:00
|
|
|
Oid *grpOperators; /* equality operators to compare with */
|
2002-11-06 23:31:24 +01:00
|
|
|
long numGroups; /* estimated number of groups in input */
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
2016-03-07 21:58:22 +01:00
|
|
|
/* Note: the planner only provides numGroups in AGG_HASHED case */
|
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
|
|
|
List *groupingSets; /* grouping sets to use */
|
|
|
|
List *chain; /* chained Agg/Sort nodes */
|
1997-09-08 22:59:27 +02:00
|
|
|
} Agg;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2008-12-28 19:54:01 +01:00
|
|
|
/* ----------------
|
|
|
|
* window aggregate node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct WindowAgg
|
|
|
|
{
|
|
|
|
Plan plan;
|
2008-12-31 01:08:39 +01:00
|
|
|
Index winref; /* ID referenced by window functions */
|
2008-12-28 19:54:01 +01:00
|
|
|
int partNumCols; /* number of columns in partition clause */
|
|
|
|
AttrNumber *partColIdx; /* their indexes in the target list */
|
|
|
|
Oid *partOperators; /* equality operators for partition columns */
|
|
|
|
int ordNumCols; /* number of columns in ordering clause */
|
|
|
|
AttrNumber *ordColIdx; /* their indexes in the target list */
|
|
|
|
Oid *ordOperators; /* equality operators for ordering columns */
|
2008-12-31 01:08:39 +01:00
|
|
|
int frameOptions; /* frame_clause options, see WindowDef */
|
2010-02-12 18:33:21 +01:00
|
|
|
Node *startOffset; /* expression for starting bound, if any */
|
|
|
|
Node *endOffset; /* expression for ending bound, if any */
|
2008-12-28 19:54:01 +01:00
|
|
|
} WindowAgg;
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/* ----------------
|
1997-09-07 07:04:48 +02:00
|
|
|
* unique node
|
1996-08-28 03:59:28 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
1997-09-07 07:04:48 +02:00
|
|
|
typedef struct Unique
|
|
|
|
{
|
2000-06-19 00:44:35 +02:00
|
|
|
Plan plan;
|
2005-10-15 04:49:52 +02:00
|
|
|
int numCols; /* number of columns to check for uniqueness */
|
2007-01-10 19:06:05 +01:00
|
|
|
AttrNumber *uniqColIdx; /* their indexes in the target list */
|
|
|
|
Oid *uniqOperators; /* equality operators to compare with */
|
1997-09-08 23:56:23 +02:00
|
|
|
} Unique;
|
1996-08-28 03:59:28 +02:00
|
|
|
|
Add a Gather executor node.
A Gather executor node runs any number of copies of a plan in an equal
number of workers and merges all of the results into a single tuple
stream. It can also run the plan itself, if the workers are
unavailable or haven't started up yet. It is intended to work with
the Partial Seq Scan node which will be added in future commits.
It could also be used to implement parallel query of a different sort
by itself, without help from Partial Seq Scan, if the single_copy mode
is used. In that mode, a worker executes the plan, and the parallel
leader does not, merely collecting the worker's results. So, a Gather
node could be inserted into a plan to split the execution of that plan
across two processes. Nested Gather nodes aren't currently supported,
but we might want to add support for that in the future.
There's nothing in the planner to actually generate Gather nodes yet,
so it's not quite time to break out the champagne. But we're getting
close.
Amit Kapila. Some designs suggestions were provided by me, and I also
reviewed the patch. Single-copy mode, documentation, and other minor
changes also by me.
2015-10-01 01:23:36 +02:00
|
|
|
/* ------------
|
|
|
|
* gather node
|
|
|
|
* ------------
|
|
|
|
*/
|
|
|
|
typedef struct Gather
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
int num_workers;
|
|
|
|
bool single_copy;
|
2016-02-07 17:39:22 +01:00
|
|
|
bool invisible; /* suppress EXPLAIN display (for testing)? */
|
Add a Gather executor node.
A Gather executor node runs any number of copies of a plan in an equal
number of workers and merges all of the results into a single tuple
stream. It can also run the plan itself, if the workers are
unavailable or haven't started up yet. It is intended to work with
the Partial Seq Scan node which will be added in future commits.
It could also be used to implement parallel query of a different sort
by itself, without help from Partial Seq Scan, if the single_copy mode
is used. In that mode, a worker executes the plan, and the parallel
leader does not, merely collecting the worker's results. So, a Gather
node could be inserted into a plan to split the execution of that plan
across two processes. Nested Gather nodes aren't currently supported,
but we might want to add support for that in the future.
There's nothing in the planner to actually generate Gather nodes yet,
so it's not quite time to break out the champagne. But we're getting
close.
Amit Kapila. Some designs suggestions were provided by me, and I also
reviewed the patch. Single-copy mode, documentation, and other minor
changes also by me.
2015-10-01 01:23:36 +02:00
|
|
|
} Gather;
|
|
|
|
|
2002-12-05 16:50:39 +01:00
|
|
|
/* ----------------
|
|
|
|
* hash build node
|
2009-03-21 01:04:40 +01:00
|
|
|
*
|
|
|
|
* If the executor is supposed to try to apply skew join optimization, then
|
2009-12-29 21:11:45 +01:00
|
|
|
* skewTable/skewColumn/skewInherit identify the outer relation's join key
|
|
|
|
* column, from which the relevant MCV statistics can be fetched. Also, its
|
|
|
|
* type information is provided to save a lookup.
|
2002-12-05 16:50:39 +01:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct Hash
|
|
|
|
{
|
|
|
|
Plan plan;
|
2009-03-21 01:04:40 +01:00
|
|
|
Oid skewTable; /* outer join key's table OID, or InvalidOid */
|
|
|
|
AttrNumber skewColumn; /* outer join key's column #, or zero */
|
2009-12-29 21:11:45 +01:00
|
|
|
bool skewInherit; /* is outer join rel an inheritance tree? */
|
2009-03-21 01:04:40 +01:00
|
|
|
Oid skewColType; /* datatype of the outer key column */
|
|
|
|
int32 skewColTypmod; /* typmod of the outer key column */
|
2003-11-25 22:00:54 +01:00
|
|
|
/* all other info is in the parent HashJoin node */
|
2002-12-05 16:50:39 +01:00
|
|
|
} Hash;
|
|
|
|
|
2000-10-05 21:11:39 +02:00
|
|
|
/* ----------------
|
|
|
|
* setop node
|
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct SetOp
|
|
|
|
{
|
|
|
|
Plan plan;
|
Make the upper part of the planner work by generating and comparing Paths.
I've been saying we needed to do this for more than five years, and here it
finally is. This patch removes the ever-growing tangle of spaghetti logic
that grouping_planner() used to use to try to identify the best plan for
post-scan/join query steps. Now, there is (nearly) independent
consideration of each execution step, and entirely separate construction of
Paths to represent each of the possible ways to do that step. We choose
the best Path or set of Paths using the same add_path() logic that's been
used inside query_planner() for years.
In addition, this patch removes the old restriction that subquery_planner()
could return only a single Plan. It now returns a RelOptInfo containing a
set of Paths, just as query_planner() does, and the parent query level can
use each of those Paths as the basis of a SubqueryScanPath at its level.
This allows finding some optimizations that we missed before, wherein a
subquery was capable of returning presorted data and thereby avoiding a
sort in the parent level, making the overall cost cheaper even though
delivering sorted output was not the cheapest plan for the subquery in
isolation. (A couple of regression test outputs change in consequence of
that. However, there is very little change in visible planner behavior
overall, because the point of this patch is not to get immediate planning
benefits but to create the infrastructure for future improvements.)
There is a great deal left to do here. This patch unblocks a lot of
planner work that was basically impractical in the old code structure,
such as allowing FDWs to implement remote aggregation, or rewriting
plan_set_operations() to allow consideration of multiple implementation
orders for set operations. (The latter will likely require a full
rewrite of plan_set_operations(); what I've done here is only to fix it
to return Paths not Plans.) I have also left unfinished some localized
refactoring in createplan.c and planner.c, because it was not necessary
to get this patch to a working state.
Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
2016-03-07 21:58:22 +01:00
|
|
|
SetOpCmd cmd; /* what to do, see nodes.h */
|
|
|
|
SetOpStrategy strategy; /* how to do it, see nodes.h */
|
2000-10-05 21:11:39 +02:00
|
|
|
int numCols; /* number of columns to check for
|
|
|
|
* duplicate-ness */
|
2007-01-10 19:06:05 +01:00
|
|
|
AttrNumber *dupColIdx; /* their indexes in the target list */
|
|
|
|
Oid *dupOperators; /* equality operators to compare with */
|
|
|
|
AttrNumber flagColIdx; /* where is the flag column, if any */
|
2008-08-07 21:35:02 +02:00
|
|
|
int firstFlag; /* flag value for first input relation */
|
2008-08-07 05:04:04 +02:00
|
|
|
long numGroups; /* estimated number of groups in input */
|
2000-10-05 21:11:39 +02:00
|
|
|
} SetOp;
|
|
|
|
|
2009-10-12 20:10:51 +02:00
|
|
|
/* ----------------
|
|
|
|
* lock-rows node
|
|
|
|
*
|
|
|
|
* rowMarks identifies the rels to be locked by this node; it should be
|
|
|
|
* a subset of the rowMarks listed in the top-level PlannedStmt.
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
* epqParam is a Param that all scan nodes below this one must depend on.
|
|
|
|
* It is used to force re-evaluation of the plan during EvalPlanQual.
|
2009-10-12 20:10:51 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct LockRows
|
|
|
|
{
|
|
|
|
Plan plan;
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
List *rowMarks; /* a list of PlanRowMark's */
|
|
|
|
int epqParam; /* ID of Param for EvalPlanQual re-eval */
|
2009-10-12 20:10:51 +02:00
|
|
|
} LockRows;
|
|
|
|
|
2000-10-26 23:38:24 +02:00
|
|
|
/* ----------------
|
|
|
|
* limit node
|
2006-07-26 21:31:51 +02:00
|
|
|
*
|
|
|
|
* Note: as of Postgres 8.2, the offset and count expressions are expected
|
|
|
|
* to yield int8, rather than int4 as before.
|
2000-10-26 23:38:24 +02:00
|
|
|
* ----------------
|
|
|
|
*/
|
|
|
|
typedef struct Limit
|
|
|
|
{
|
|
|
|
Plan plan;
|
|
|
|
Node *limitOffset; /* OFFSET parameter, or NULL if none */
|
|
|
|
Node *limitCount; /* COUNT parameter, or NULL if none */
|
|
|
|
} Limit;
|
|
|
|
|
2008-09-09 20:58:09 +02:00
|
|
|
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
/*
|
|
|
|
* RowMarkType -
|
|
|
|
* enums for types of row-marking operations
|
|
|
|
*
|
2013-03-10 19:14:53 +01:00
|
|
|
* The first four of these values represent different lock strengths that
|
|
|
|
* we can take on tuples according to SELECT FOR [KEY] UPDATE/SHARE requests.
|
Add support for doing late row locking in FDWs.
Previously, FDWs could only do "early row locking", that is lock a row as
soon as it's fetched, even though local restriction/join conditions might
discard the row later. This patch adds callbacks that allow FDWs to do
late locking in the same way that it's done for regular tables.
To make use of this feature, an FDW must support the "ctid" column as a
unique row identifier. Currently, since ctid has to be of type TID,
the feature is of limited use, though in principle it could be used by
postgres_fdw. We may eventually allow FDWs to specify another data type
for ctid, which would make it possible for more FDWs to use this feature.
This commit does not modify postgres_fdw to use late locking. We've
tested some prototype code for that, but it's not in committable shape,
and besides it's quite unclear whether it actually makes sense to do late
locking against a remote server. The extra round trips required are likely
to outweigh any benefit from improved concurrency.
Etsuro Fujita, reviewed by Ashutosh Bapat, and hacked up a lot by me
2015-05-12 20:10:10 +02:00
|
|
|
* We support these on regular tables, as well as on foreign tables whose FDWs
|
|
|
|
* report support for late locking. For other foreign tables, any locking
|
|
|
|
* that might be done for such requests must happen during the initial row
|
|
|
|
* fetch; their FDWs provide no mechanism for going back to lock a row later.
|
2013-03-10 19:14:53 +01:00
|
|
|
* This means that the semantics will be a bit different than for a local
|
|
|
|
* table; in particular we are likely to lock more rows than would be locked
|
|
|
|
* locally, since remote rows will be locked even if they then fail
|
Add support for doing late row locking in FDWs.
Previously, FDWs could only do "early row locking", that is lock a row as
soon as it's fetched, even though local restriction/join conditions might
discard the row later. This patch adds callbacks that allow FDWs to do
late locking in the same way that it's done for regular tables.
To make use of this feature, an FDW must support the "ctid" column as a
unique row identifier. Currently, since ctid has to be of type TID,
the feature is of limited use, though in principle it could be used by
postgres_fdw. We may eventually allow FDWs to specify another data type
for ctid, which would make it possible for more FDWs to use this feature.
This commit does not modify postgres_fdw to use late locking. We've
tested some prototype code for that, but it's not in committable shape,
and besides it's quite unclear whether it actually makes sense to do late
locking against a remote server. The extra round trips required are likely
to outweigh any benefit from improved concurrency.
Etsuro Fujita, reviewed by Ashutosh Bapat, and hacked up a lot by me
2015-05-12 20:10:10 +02:00
|
|
|
* locally-checked restriction or join quals. However, the prospect of
|
|
|
|
* doing a separate remote query to lock each selected row is usually pretty
|
|
|
|
* unappealing, so early locking remains a credible design choice for FDWs.
|
2013-03-10 19:14:53 +01:00
|
|
|
*
|
|
|
|
* When doing UPDATE, DELETE, or SELECT FOR UPDATE/SHARE, we have to uniquely
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
* identify all the source rows, not only those from the target relations, so
|
|
|
|
* that we can perform EvalPlanQual rechecking at need. For plain tables we
|
2013-03-10 19:14:53 +01:00
|
|
|
* can just fetch the TID, much as for a target relation; this case is
|
|
|
|
* represented by ROW_MARK_REFERENCE. Otherwise (for example for VALUES or
|
|
|
|
* FUNCTION scans) we have to copy the whole row value. ROW_MARK_COPY is
|
|
|
|
* pretty inefficient, since most of the time we'll never need the data; but
|
Add support for doing late row locking in FDWs.
Previously, FDWs could only do "early row locking", that is lock a row as
soon as it's fetched, even though local restriction/join conditions might
discard the row later. This patch adds callbacks that allow FDWs to do
late locking in the same way that it's done for regular tables.
To make use of this feature, an FDW must support the "ctid" column as a
unique row identifier. Currently, since ctid has to be of type TID,
the feature is of limited use, though in principle it could be used by
postgres_fdw. We may eventually allow FDWs to specify another data type
for ctid, which would make it possible for more FDWs to use this feature.
This commit does not modify postgres_fdw to use late locking. We've
tested some prototype code for that, but it's not in committable shape,
and besides it's quite unclear whether it actually makes sense to do late
locking against a remote server. The extra round trips required are likely
to outweigh any benefit from improved concurrency.
Etsuro Fujita, reviewed by Ashutosh Bapat, and hacked up a lot by me
2015-05-12 20:10:10 +02:00
|
|
|
* fortunately the overhead is usually not performance-critical in practice.
|
|
|
|
* By default we use ROW_MARK_COPY for foreign tables, but if the FDW has
|
|
|
|
* a concept of rowid it can request to use ROW_MARK_REFERENCE instead.
|
|
|
|
* (Again, this probably doesn't make sense if a physical remote fetch is
|
|
|
|
* needed, but for FDWs that map to local storage it might be credible.)
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
*/
|
|
|
|
typedef enum RowMarkType
|
|
|
|
{
|
|
|
|
ROW_MARK_EXCLUSIVE, /* obtain exclusive tuple lock */
|
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
|
|
|
ROW_MARK_NOKEYEXCLUSIVE, /* obtain no-key exclusive tuple lock */
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
ROW_MARK_SHARE, /* obtain shared tuple lock */
|
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
|
|
|
ROW_MARK_KEYSHARE, /* obtain keyshare tuple lock */
|
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
|
|
|
ROW_MARK_REFERENCE, /* just fetch the TID, don't lock it */
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
ROW_MARK_COPY /* physically copy the row value */
|
|
|
|
} RowMarkType;
|
|
|
|
|
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
|
|
|
#define RowMarkRequiresRowShareLock(marktype) ((marktype) <= ROW_MARK_KEYSHARE)
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* PlanRowMark -
|
Improve concurrency of foreign key locking
This patch introduces two additional lock modes for tuples: "SELECT FOR
KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each
other, in contrast with already existing "SELECT FOR SHARE" and "SELECT
FOR UPDATE". UPDATE commands that do not modify the values stored in
the columns that are part of the key of the tuple now grab a SELECT FOR
NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently
with tuple locks of the FOR KEY SHARE variety.
Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this
means the concurrency improvement applies to them, which is the whole
point of this patch.
The added tuple lock semantics require some rejiggering of the multixact
module, so that the locking level that each transaction is holding can
be stored alongside its Xid. Also, multixacts now need to persist
across server restarts and crashes, because they can now represent not
only tuple locks, but also tuple updates. This means we need more
careful tracking of lifetime of pg_multixact SLRU files; since they now
persist longer, we require more infrastructure to figure out when they
can be removed. pg_upgrade also needs to be careful to copy
pg_multixact files over from the old server to the new, or at least part
of multixact.c state, depending on the versions of the old and new
servers.
Tuple time qualification rules (HeapTupleSatisfies routines) need to be
careful not to consider tuples with the "is multi" infomask bit set as
being only locked; they might need to look up MultiXact values (i.e.
possibly do pg_multixact I/O) to find out the Xid that updated a tuple,
whereas they previously were assured to only use information readily
available from the tuple header. This is considered acceptable, because
the extra I/O would involve cases that would previously cause some
commands to block waiting for concurrent transactions to finish.
Another important change is the fact that locking tuples that have
previously been updated causes the future versions to be marked as
locked, too; this is essential for correctness of foreign key checks.
This causes additional WAL-logging, also (there was previously a single
WAL record for a locked tuple; now there are as many as updated copies
of the tuple there exist.)
With all this in place, contention related to tuples being checked by
foreign key rules should be much reduced.
As a bonus, the old behavior that a subtransaction grabbing a stronger
tuple lock than the parent (sub)transaction held on a given tuple and
later aborting caused the weaker lock to be lost, has been fixed.
Many new spec files were added for isolation tester framework, to ensure
overall behavior is sane. There's probably room for several more tests.
There were several reviewers of this patch; in particular, Noah Misch
and Andres Freund spent considerable time in it. Original idea for the
patch came from Simon Riggs, after a problem report by Joel Jacobson.
Most code is from me, with contributions from Marti Raudsepp, Alexander
Shulgin, Noah Misch and Andres Freund.
This patch was discussed in several pgsql-hackers threads; the most
important start at the following message-ids:
AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com
1290721684-sup-3951@alvh.no-ip.org
1294953201-sup-2099@alvh.no-ip.org
1320343602-sup-2290@alvh.no-ip.org
1339690386-sup-8927@alvh.no-ip.org
4FE5FF020200002500048A3D@gw.wicourts.gov
4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
|
|
|
* plan-time representation of FOR [KEY] UPDATE/SHARE clauses
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
*
|
2013-03-10 19:14:53 +01:00
|
|
|
* When doing UPDATE, DELETE, or SELECT FOR UPDATE/SHARE, we create a separate
|
2014-05-06 18:12:18 +02:00
|
|
|
* PlanRowMark node for each non-target relation in the query. Relations that
|
2013-03-10 19:14:53 +01:00
|
|
|
* are not specified as FOR UPDATE/SHARE are marked ROW_MARK_REFERENCE (if
|
Add support for doing late row locking in FDWs.
Previously, FDWs could only do "early row locking", that is lock a row as
soon as it's fetched, even though local restriction/join conditions might
discard the row later. This patch adds callbacks that allow FDWs to do
late locking in the same way that it's done for regular tables.
To make use of this feature, an FDW must support the "ctid" column as a
unique row identifier. Currently, since ctid has to be of type TID,
the feature is of limited use, though in principle it could be used by
postgres_fdw. We may eventually allow FDWs to specify another data type
for ctid, which would make it possible for more FDWs to use this feature.
This commit does not modify postgres_fdw to use late locking. We've
tested some prototype code for that, but it's not in committable shape,
and besides it's quite unclear whether it actually makes sense to do late
locking against a remote server. The extra round trips required are likely
to outweigh any benefit from improved concurrency.
Etsuro Fujita, reviewed by Ashutosh Bapat, and hacked up a lot by me
2015-05-12 20:10:10 +02:00
|
|
|
* regular tables or supported foreign tables) or ROW_MARK_COPY (if not).
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
*
|
|
|
|
* Initially all PlanRowMarks have rti == prti and isParent == false.
|
|
|
|
* When the planner discovers that a relation is the root of an inheritance
|
|
|
|
* tree, it sets isParent true, and adds an additional PlanRowMark to the
|
|
|
|
* list for each child relation (including the target rel itself in its role
|
|
|
|
* as a child). The child entries have rti == child rel's RT index and
|
|
|
|
* prti == parent's RT index, and can therefore be recognized as children by
|
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
|
|
|
* the fact that prti != rti. The parent's allMarkTypes field gets the OR
|
|
|
|
* of (1<<markType) across all its children (this definition allows children
|
|
|
|
* to use different markTypes).
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
*
|
2011-01-13 02:47:02 +01:00
|
|
|
* The planner also adds resjunk output columns to the plan that carry
|
Add support for doing late row locking in FDWs.
Previously, FDWs could only do "early row locking", that is lock a row as
soon as it's fetched, even though local restriction/join conditions might
discard the row later. This patch adds callbacks that allow FDWs to do
late locking in the same way that it's done for regular tables.
To make use of this feature, an FDW must support the "ctid" column as a
unique row identifier. Currently, since ctid has to be of type TID,
the feature is of limited use, though in principle it could be used by
postgres_fdw. We may eventually allow FDWs to specify another data type
for ctid, which would make it possible for more FDWs to use this feature.
This commit does not modify postgres_fdw to use late locking. We've
tested some prototype code for that, but it's not in committable shape,
and besides it's quite unclear whether it actually makes sense to do late
locking against a remote server. The extra round trips required are likely
to outweigh any benefit from improved concurrency.
Etsuro Fujita, reviewed by Ashutosh Bapat, and hacked up a lot by me
2015-05-12 20:10:10 +02:00
|
|
|
* information sufficient to identify the locked or fetched rows. When
|
|
|
|
* markType != ROW_MARK_COPY, these columns are named
|
2011-01-13 02:47:02 +01:00
|
|
|
* tableoid%u OID of table
|
|
|
|
* ctid%u TID of row
|
|
|
|
* The tableoid column is only present for an inheritance hierarchy.
|
|
|
|
* When markType == ROW_MARK_COPY, there is instead a single column named
|
|
|
|
* wholerow%u whole-row value of relation
|
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
|
|
|
* (An inheritance hierarchy could have all three resjunk output columns,
|
|
|
|
* if some children use a different markType than others.)
|
2011-02-10 05:27:07 +01:00
|
|
|
* In all three cases, %u represents the rowmark ID number (rowmarkId).
|
|
|
|
* This number is unique within a plan tree, except that child relation
|
|
|
|
* entries copy their parent's rowmarkId. (Assigning unique numbers
|
|
|
|
* means we needn't renumber rowmarkIds when flattening subqueries, which
|
|
|
|
* would require finding and renaming the resjunk columns as well.)
|
2011-01-13 02:47:02 +01:00
|
|
|
* Note this means that all tables in an inheritance hierarchy share the
|
|
|
|
* same resjunk column names. However, in an inherited UPDATE/DELETE the
|
|
|
|
* columns could have different physical column numbers in each subplan.
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
*/
|
|
|
|
typedef struct PlanRowMark
|
|
|
|
{
|
|
|
|
NodeTag type;
|
|
|
|
Index rti; /* range table index of markable relation */
|
|
|
|
Index prti; /* range table index of parent relation */
|
2011-02-10 05:27:07 +01:00
|
|
|
Index rowmarkId; /* unique identifier for resjunk columns */
|
2010-02-26 03:01:40 +01:00
|
|
|
RowMarkType markType; /* see enum above */
|
Improve representation of PlanRowMark.
This patch fixes two inadequacies of the PlanRowMark representation.
First, that the original LockingClauseStrength isn't stored (and cannot be
inferred for foreign tables, which always get ROW_MARK_COPY). Since some
PlanRowMarks are created out of whole cloth and don't actually have an
ancestral RowMarkClause, this requires adding a dummy LCS_NONE value to
enum LockingClauseStrength, which is fairly annoying but the alternatives
seem worse. This fix allows getting rid of the use of get_parse_rowmark()
in FDWs (as per the discussion around commits 462bd95705a0c23b and
8ec8760fc87ecde0), and it simplifies some things elsewhere.
Second, that the representation assumed that all child tables in an
inheritance hierarchy would use the same RowMarkType. That's true today
but will soon not be true. We add an "allMarkTypes" field that identifies
the union of mark types used in all a parent table's children, and use
that where appropriate (currently, only in preprocess_targetlist()).
In passing fix a couple of minor infelicities left over from the SKIP
LOCKED patch, notably that _outPlanRowMark still thought waitPolicy
is a bool.
Catversion bump is required because the numeric values of enum
LockingClauseStrength can appear in on-disk rules.
Extracted from a much larger patch to support foreign table inheritance;
it seemed worth breaking this out, since it's a separable concern.
Shigeru Hanada and Etsuro Fujita, somewhat modified by me
2015-03-15 23:41:47 +01:00
|
|
|
int allMarkTypes; /* OR of (1<<markType) for all children */
|
|
|
|
LockClauseStrength strength; /* LockingClause's strength, or LCS_NONE */
|
2014-11-21 00:36:07 +01:00
|
|
|
LockWaitPolicy waitPolicy; /* NOWAIT and SKIP LOCKED options */
|
Re-implement EvalPlanQual processing to improve its performance and eliminate
a lot of strange behaviors that occurred in join cases. We now identify the
"current" row for every joined relation in UPDATE, DELETE, and SELECT FOR
UPDATE/SHARE queries. If an EvalPlanQual recheck is necessary, we jam the
appropriate row into each scan node in the rechecking plan, forcing it to emit
only that one row. The former behavior could rescan the whole of each joined
relation for each recheck, which was terrible for performance, and what's much
worse could result in duplicated output tuples.
Also, the original implementation of EvalPlanQual could not re-use the recheck
execution tree --- it had to go through a full executor init and shutdown for
every row to be tested. To avoid this overhead, I've associated a special
runtime Param with each LockRows or ModifyTable plan node, and arranged to
make every scan node below such a node depend on that Param. Thus, by
signaling a change in that Param, the EPQ machinery can just rescan the
already-built test plan.
This patch also adds a prohibition on set-returning functions in the
targetlist of SELECT FOR UPDATE/SHARE. This is needed to avoid the
duplicate-output-tuple problem. It seems fairly reasonable since the
other restrictions on SELECT FOR UPDATE are meant to ensure that there
is a unique correspondence between source tuples and result tuples,
which an output SRF destroys as much as anything else does.
2009-10-26 03:26:45 +01:00
|
|
|
bool isParent; /* true if this is a "dummy" parent entry */
|
|
|
|
} PlanRowMark;
|
|
|
|
|
|
|
|
|
2008-09-09 20:58:09 +02:00
|
|
|
/*
|
|
|
|
* Plan invalidation info
|
|
|
|
*
|
|
|
|
* We track the objects on which a PlannedStmt depends in two ways:
|
|
|
|
* relations are recorded as a simple list of OIDs, and everything else
|
2014-05-06 18:12:18 +02:00
|
|
|
* is represented as a list of PlanInvalItems. A PlanInvalItem is designed
|
2008-09-09 20:58:09 +02:00
|
|
|
* to be used with the syscache invalidation mechanism, so it identifies a
|
2011-08-17 01:27:46 +02:00
|
|
|
* system catalog entry by cache ID and hash value.
|
2008-09-09 20:58:09 +02:00
|
|
|
*/
|
|
|
|
typedef struct PlanInvalItem
|
|
|
|
{
|
|
|
|
NodeTag type;
|
|
|
|
int cacheId; /* a syscache ID, see utils/syscache.h */
|
2011-08-17 01:27:46 +02:00
|
|
|
uint32 hashValue; /* hash value of object's cache lookup key */
|
2008-09-09 20:58:09 +02:00
|
|
|
} PlanInvalItem;
|
|
|
|
|
2001-11-05 18:46:40 +01:00
|
|
|
#endif /* PLANNODES_H */
|