rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Refactor and cleanup runtime partition prune code a little 

* Move the execution pruning initialization steps that are common
between both ExecInitAppend() and ExecInitMergeAppend() into a new
function ExecInitPartitionPruning() defined in execPartition.c.
Those steps include creation of a PartitionPruneState to be used for
all instances of pruning and determining the minimal set of child
subplans that need to be initialized by performing initial pruning if
needed, and finally adjusting the subplan_map arrays in the
PartitionPruneState to reflect the new set of subplans remaining
after initial pruning if it was indeed performed.
ExecCreatePartitionPruneState() is no longer exported out of
execPartition.c and has been renamed to CreatePartitionPruneState()
as a local sub-routine of ExecInitPartitionPruning().

* Likewise, ExecFindInitialMatchingSubPlans() that was in charge of
performing initial pruning no longer needs to be exported.  In fact,
since it would now have the same body as the more generally named
ExecFindMatchingSubPlans(), except differing in the value of
initial_prune passed to the common subroutine
find_matching_subplans_recurse(), it seems better to remove it and add
an initial_prune argument to ExecFindMatchingSubPlans().

* Add an ExprContext field to PartitionPruneContext to remove the
implicit assumption in the runtime pruning code that the ExprContext to
use to compute pruning expressions that need one can always rely on the
PlanState providing it.  A future patch will allow runtime pruning (at
least the initial pruning steps) to be performed without the
corresponding PlanState yet having been created, so this will help.

Author: Amit Langote <amitlangote09@gmail.com>
Discussion: https://postgr.es/m/CA+HiwqEYCpEqh2LMDOp9mT+4-QoVe8HgFMKBjntEMCTZLpcCCA@mail.gmail.com
This commit is contained in:
Alvaro Herrera 2022-04-05 11:46:48 +02:00
parent 7a43a1fc52
commit 297daa9d43
No known key found for this signature in database
GPG Key ID: 1C20ACB9D5C564AE
6 changed files with 300 additions and 295 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

463
src/backend/executor/execPartition.c
View File

@ -184,11 +184,17 @@ static char *ExecBuildSlotPartitionKeyDescription(Relation rel,
int maxfieldlen); int maxfieldlen);
static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri); static List *adjust_partition_colnos(List *colnos, ResultRelInfo *leaf_part_rri);
static List *adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap); static List *adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap);
static void ExecInitPruningContext(PartitionPruneContext *context, static PartitionPruneState *CreatePartitionPruneState(PlanState *planstate,
List *pruning_steps, PartitionPruneInfo *pruneinfo);
PartitionDesc partdesc, static void InitPartitionPruneContext(PartitionPruneContext *context,
PartitionKey partkey, List *pruning_steps,
PlanState *planstate); PartitionDesc partdesc,
PartitionKey partkey,
PlanState *planstate,
ExprContext *econtext);
static void PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate,
Bitmapset *initially_valid_subplans,
int n_total_subplans);
static void find_matching_subplans_recurse(PartitionPruningData *prunedata, static void find_matching_subplans_recurse(PartitionPruningData *prunedata,
PartitionedRelPruningData *pprune, PartitionedRelPruningData *pprune,
bool initial_prune, bool initial_prune,
@ -1590,64 +1596,121 @@ adjust_partition_colnos_using_map(List *colnos, AttrMap *attrMap)
* *
* Functions: * Functions:
* *
* ExecCreatePartitionPruneState: * ExecInitPartitionPruning:
* Creates the PartitionPruneState required by each of the two pruning * Creates the PartitionPruneState required by ExecFindMatchingSubPlans.
* functions. Details stored include how to map the partition index * Details stored include how to map the partition index returned by the
* returned by the partition pruning code into subplan indexes. * partition pruning code into subplan indexes. Also determines the set
* * of subplans to initialize considering the result of performing initial
* ExecFindInitialMatchingSubPlans: * pruning steps if any. Maps in PartitionPruneState are updated to
* Returns indexes of matching subplans. Partition pruning is attempted * account for initial pruning possibly having eliminated some of the
* without any evaluation of expressions containing PARAM_EXEC Params. * subplans.
* This function must be called during executor startup for the parent
* plan before the subplans themselves are initialized. Subplans which
* are found not to match by this function must be removed from the
* plan's list of subplans during execution, as this function performs a
* remap of the partition index to subplan index map and the newly
* created map provides indexes only for subplans which remain after
* calling this function.
* *
* ExecFindMatchingSubPlans: * ExecFindMatchingSubPlans:
* Returns indexes of matching subplans after evaluating all available * Returns indexes of matching subplans after evaluating the expressions
* expressions. This function can only be called during execution and * that are safe to evaluate at a given point. This function is first
* must be called again each time the value of a Param listed in * called during ExecInitPartitionPruning() to find the initially
* matching subplans based on performing the initial pruning steps and
* then must be called again each time the value of a Param listed in
* PartitionPruneState's 'execparamids' changes. * PartitionPruneState's 'execparamids' changes.
*------------------------------------------------------------------------- *-------------------------------------------------------------------------
*/ */
/* /*
* ExecCreatePartitionPruneState * ExecInitPartitionPruning
* Build the data structure required for calling * Initialize data structure needed for run-time partition pruning and
* ExecFindInitialMatchingSubPlans and ExecFindMatchingSubPlans. * do initial pruning if needed
*
* On return, *initially_valid_subplans is assigned the set of indexes of
* child subplans that must be initialized along with the parent plan node.
* Initial pruning is performed here if needed and in that case only the
* surviving subplans' indexes are added.
*
* If subplans are indeed pruned, subplan_map arrays contained in the returned
* PartitionPruneState are re-sequenced to not count those, though only if the
* maps will be needed for subsequent execution pruning passes.
*/
PartitionPruneState *
ExecInitPartitionPruning(PlanState *planstate,
int n_total_subplans,
PartitionPruneInfo *pruneinfo,
Bitmapset **initially_valid_subplans)
{
PartitionPruneState *prunestate;
EState *estate = planstate->state;
/* We may need an expression context to evaluate partition exprs */
ExecAssignExprContext(estate, planstate);
/* Create the working data structure for pruning */
prunestate = CreatePartitionPruneState(planstate, pruneinfo);
/*
* Perform an initial partition prune pass, if required.
*/
if (prunestate->do_initial_prune)
*initially_valid_subplans = ExecFindMatchingSubPlans(prunestate, true);
else
{
/* No pruning, so we'll need to initialize all subplans */
Assert(n_total_subplans > 0);
*initially_valid_subplans = bms_add_range(NULL, 0,
n_total_subplans - 1);
}
/*
* Re-sequence subplan indexes contained in prunestate to account for any
* that were removed above due to initial pruning. No need to do this if
* no steps were removed.
*/
if (bms_num_members(*initially_valid_subplans) < n_total_subplans)
{
/*
* We can safely skip this when !do_exec_prune, even though that
* leaves invalid data in prunestate, because that data won't be
* consulted again (cf initial Assert in ExecFindMatchingSubPlans).
*/
if (prunestate->do_exec_prune)
PartitionPruneFixSubPlanMap(prunestate,
*initially_valid_subplans,
n_total_subplans);
}
return prunestate;
}
/*
* CreatePartitionPruneState
* Build the data structure required for calling ExecFindMatchingSubPlans
* *
* 'planstate' is the parent plan node's execution state. * 'planstate' is the parent plan node's execution state.
* *
* 'partitionpruneinfo' is a PartitionPruneInfo as generated by * 'pruneinfo' is a PartitionPruneInfo as generated by
* make_partition_pruneinfo. Here we build a PartitionPruneState containing a * make_partition_pruneinfo. Here we build a PartitionPruneState containing a
* PartitionPruningData for each partitioning hierarchy (i.e., each sublist of * PartitionPruningData for each partitioning hierarchy (i.e., each sublist of
* partitionpruneinfo->prune_infos), each of which contains a * pruneinfo->prune_infos), each of which contains a PartitionedRelPruningData
* PartitionedRelPruningData for each PartitionedRelPruneInfo appearing in * for each PartitionedRelPruneInfo appearing in that sublist. This two-level
* that sublist. This two-level system is needed to keep from confusing the * system is needed to keep from confusing the different hierarchies when a
* different hierarchies when a UNION ALL contains multiple partitioned tables * UNION ALL contains multiple partitioned tables as children. The data
* as children. The data stored in each PartitionedRelPruningData can be * stored in each PartitionedRelPruningData can be re-used each time we
* re-used each time we re-evaluate which partitions match the pruning steps * re-evaluate which partitions match the pruning steps provided in each
* provided in each PartitionedRelPruneInfo. * PartitionedRelPruneInfo.
*/ */
PartitionPruneState * static PartitionPruneState *
ExecCreatePartitionPruneState(PlanState *planstate, CreatePartitionPruneState(PlanState *planstate, PartitionPruneInfo *pruneinfo)
PartitionPruneInfo *partitionpruneinfo)
{ {
EState *estate = planstate->state; EState *estate = planstate->state;
PartitionPruneState *prunestate; PartitionPruneState *prunestate;
int n_part_hierarchies; int n_part_hierarchies;
ListCell *lc; ListCell *lc;
int i; int i;
ExprContext *econtext = planstate->ps_ExprContext;
/* For data reading, executor always omits detached partitions */ /* For data reading, executor always omits detached partitions */
if (estate->es_partition_directory == NULL) if (estate->es_partition_directory == NULL)
estate->es_partition_directory = estate->es_partition_directory =
CreatePartitionDirectory(estate->es_query_cxt, false); CreatePartitionDirectory(estate->es_query_cxt, false);
n_part_hierarchies = list_length(partitionpruneinfo->prune_infos); n_part_hierarchies = list_length(pruneinfo->prune_infos);
Assert(n_part_hierarchies > 0); Assert(n_part_hierarchies > 0);
/* /*
@ -1659,7 +1722,7 @@ ExecCreatePartitionPruneState(PlanState *planstate,
prunestate->execparamids = NULL; prunestate->execparamids = NULL;
/* other_subplans can change at runtime, so we need our own copy */ /* other_subplans can change at runtime, so we need our own copy */
prunestate->other_subplans = bms_copy(partitionpruneinfo->other_subplans); prunestate->other_subplans = bms_copy(pruneinfo->other_subplans);
prunestate->do_initial_prune = false; /* may be set below */ prunestate->do_initial_prune = false; /* may be set below */
prunestate->do_exec_prune = false; /* may be set below */ prunestate->do_exec_prune = false; /* may be set below */
prunestate->num_partprunedata = n_part_hierarchies; prunestate->num_partprunedata = n_part_hierarchies;
@ -1676,7 +1739,7 @@ ExecCreatePartitionPruneState(PlanState *planstate,
ALLOCSET_DEFAULT_SIZES); ALLOCSET_DEFAULT_SIZES);
i = 0; i = 0;
foreach(lc, partitionpruneinfo->prune_infos) foreach(lc, pruneinfo->prune_infos)
{ {
List *partrelpruneinfos = lfirst_node(List, lc); List *partrelpruneinfos = lfirst_node(List, lc);
int npartrelpruneinfos = list_length(partrelpruneinfos); int npartrelpruneinfos = list_length(partrelpruneinfos);
@ -1812,18 +1875,20 @@ ExecCreatePartitionPruneState(PlanState *planstate,
pprune->initial_pruning_steps = pinfo->initial_pruning_steps; pprune->initial_pruning_steps = pinfo->initial_pruning_steps;
if (pinfo->initial_pruning_steps) if (pinfo->initial_pruning_steps)
{ {
ExecInitPruningContext(&pprune->initial_context, InitPartitionPruneContext(&pprune->initial_context,
pinfo->initial_pruning_steps, pinfo->initial_pruning_steps,
partdesc, partkey, planstate); partdesc, partkey, planstate,
econtext);
/* Record whether initial pruning is needed at any level */ /* Record whether initial pruning is needed at any level */
prunestate->do_initial_prune = true; prunestate->do_initial_prune = true;
} }
pprune->exec_pruning_steps = pinfo->exec_pruning_steps; pprune->exec_pruning_steps = pinfo->exec_pruning_steps;
if (pinfo->exec_pruning_steps) if (pinfo->exec_pruning_steps)
{ {
ExecInitPruningContext(&pprune->exec_context, InitPartitionPruneContext(&pprune->exec_context,
pinfo->exec_pruning_steps, pinfo->exec_pruning_steps,
partdesc, partkey, planstate); partdesc, partkey, planstate,
econtext);
/* Record whether exec pruning is needed at any level */ /* Record whether exec pruning is needed at any level */
prunestate->do_exec_prune = true; prunestate->do_exec_prune = true;
} }
@ -1847,11 +1912,12 @@ ExecCreatePartitionPruneState(PlanState *planstate,
* Initialize a PartitionPruneContext for the given list of pruning steps. * Initialize a PartitionPruneContext for the given list of pruning steps.
*/ */
static void static void
ExecInitPruningContext(PartitionPruneContext *context, InitPartitionPruneContext(PartitionPruneContext *context,
List *pruning_steps, List *pruning_steps,
PartitionDesc partdesc, PartitionDesc partdesc,
PartitionKey partkey, PartitionKey partkey,
PlanState *planstate) PlanState *planstate,
ExprContext *econtext)
{ {
int n_steps; int n_steps;
int partnatts; int partnatts;
@ -1872,6 +1938,7 @@ ExecInitPruningContext(PartitionPruneContext *context,
context->ppccontext = CurrentMemoryContext; context->ppccontext = CurrentMemoryContext;
context->planstate = planstate; context->planstate = planstate;
context->exprcontext = econtext;
/* Initialize expression state for each expression we need */ /* Initialize expression state for each expression we need */
context->exprstates = (ExprState **) context->exprstates = (ExprState **)
@ -1900,8 +1967,20 @@ ExecInitPruningContext(PartitionPruneContext *context,
step->step.step_id, step->step.step_id,
keyno); keyno);
context->exprstates[stateidx] = /*
ExecInitExpr(expr, context->planstate); * When planstate is NULL, pruning_steps is known not to
* contain any expressions that depend on the parent plan.
* Information of any available EXTERN parameters must be
* passed explicitly in that case, which the caller must have
* made available via econtext.
*/
if (planstate == NULL)
context->exprstates[stateidx] =
ExecInitExprWithParams(expr,
econtext->ecxt_param_list_info);
else
context->exprstates[stateidx] =
ExecInitExpr(expr, context->planstate);
} }
keyno++; keyno++;
} }
@ -1909,179 +1988,119 @@ ExecInitPruningContext(PartitionPruneContext *context,
} }
/* /*
* ExecFindInitialMatchingSubPlans * PartitionPruneFixSubPlanMap
* Identify the set of subplans that cannot be eliminated by initial * Fix mapping of partition indexes to subplan indexes contained in
* pruning, disregarding any pruning constraints involving PARAM_EXEC * prunestate by considering the new list of subplans that survived
* Params. * initial pruning
* *
* If additional pruning passes will be required (because of PARAM_EXEC * Current values of the indexes present in PartitionPruneState count all the
* Params), we must also update the translation data that allows conversion * subplans that would be present before initial pruning was done. If initial
* of partition indexes into subplan indexes to account for the unneeded * pruning got rid of some of the subplans, any subsequent pruning passes will
* subplans having been removed. * will be looking at a different set of target subplans to choose from than
* * those in the pre-initial-pruning set, so the maps in PartitionPruneState
* Must only be called once per 'prunestate', and only if initial pruning * containing those indexes must be updated to reflect the new indexes of
* is required. * subplans in the post-initial-pruning set.
*
* 'nsubplans' must be passed as the total number of unpruned subplans.
*/ */
Bitmapset * static void
ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate, int nsubplans) PartitionPruneFixSubPlanMap(PartitionPruneState *prunestate,
Bitmapset *initially_valid_subplans,
int n_total_subplans)
{ {
Bitmapset *result = NULL; int *new_subplan_indexes;
MemoryContext oldcontext; Bitmapset *new_other_subplans;
int i; int i;
int newidx;
/* Caller error if we get here without do_initial_prune */
Assert(prunestate->do_initial_prune);
/* /*
* Switch to a temp context to avoid leaking memory in the executor's * First we must build a temporary array which maps old subplan indexes to
* query-lifespan memory context. * new ones. For convenience of initialization, we use 1-based indexes in
* this array and leave pruned items as 0.
*/ */
oldcontext = MemoryContextSwitchTo(prunestate->prune_context); new_subplan_indexes = (int *) palloc0(sizeof(int) * n_total_subplans);
newidx = 1;
i = -1;
while ((i = bms_next_member(initially_valid_subplans, i)) >= 0)
{
Assert(i < n_total_subplans);
new_subplan_indexes[i] = newidx++;
}
/* /*
* For each hierarchy, do the pruning tests, and add nondeletable * Now we can update each PartitionedRelPruneInfo's subplan_map with new
* subplans' indexes to "result". * subplan indexes. We must also recompute its present_parts bitmap.
*/ */
for (i = 0; i < prunestate->num_partprunedata; i++) for (i = 0; i < prunestate->num_partprunedata; i++)
{ {
PartitionPruningData *prunedata; PartitionPruningData *prunedata = prunestate->partprunedata[i];
PartitionedRelPruningData *pprune; int j;
prunedata = prunestate->partprunedata[i];
pprune = &prunedata->partrelprunedata[0];
/* Perform pruning without using PARAM_EXEC Params */
find_matching_subplans_recurse(prunedata, pprune, true, &result);
/* Expression eval may have used space in node's ps_ExprContext too */
if (pprune->initial_pruning_steps)
ResetExprContext(pprune->initial_context.planstate->ps_ExprContext);
}
/* Add in any subplans that partition pruning didn't account for */
result = bms_add_members(result, prunestate->other_subplans);
MemoryContextSwitchTo(oldcontext);
/* Copy result out of the temp context before we reset it */
result = bms_copy(result);
MemoryContextReset(prunestate->prune_context);
/*
* If exec-time pruning is required and we pruned subplans above, then we
* must re-sequence the subplan indexes so that ExecFindMatchingSubPlans
* properly returns the indexes from the subplans which will remain after
* execution of this function.
*
* We can safely skip this when !do_exec_prune, even though that leaves
* invalid data in prunestate, because that data won't be consulted again
* (cf initial Assert in ExecFindMatchingSubPlans).
*/
if (prunestate->do_exec_prune && bms_num_members(result) < nsubplans)
{
int *new_subplan_indexes;
Bitmapset *new_other_subplans;
int i;
int newidx;
/* /*
* First we must build a temporary array which maps old subplan * Within each hierarchy, we perform this loop in back-to-front order
* indexes to new ones. For convenience of initialization, we use * so that we determine present_parts for the lowest-level partitioned
* 1-based indexes in this array and leave pruned items as 0. * tables first. This way we can tell whether a sub-partitioned
* table's partitions were entirely pruned so we can exclude it from
* the current level's present_parts.
*/ */
new_subplan_indexes = (int *) palloc0(sizeof(int) * nsubplans); for (j = prunedata->num_partrelprunedata - 1; j >= 0; j--)
newidx = 1;
i = -1;
while ((i = bms_next_member(result, i)) >= 0)
{ {
Assert(i < nsubplans); PartitionedRelPruningData *pprune = &prunedata->partrelprunedata[j];
new_subplan_indexes[i] = newidx++; int nparts = pprune->nparts;
} int k;
/* /* We just rebuild present_parts from scratch */
* Now we can update each PartitionedRelPruneInfo's subplan_map with bms_free(pprune->present_parts);
* new subplan indexes. We must also recompute its present_parts pprune->present_parts = NULL;
* bitmap.
*/
for (i = 0; i < prunestate->num_partprunedata; i++)
{
PartitionPruningData *prunedata = prunestate->partprunedata[i];
int j;
/* for (k = 0; k < nparts; k++)
* Within each hierarchy, we perform this loop in back-to-front
* order so that we determine present_parts for the lowest-level
* partitioned tables first. This way we can tell whether a
* sub-partitioned table's partitions were entirely pruned so we
* can exclude it from the current level's present_parts.
*/
for (j = prunedata->num_partrelprunedata - 1; j >= 0; j--)
{ {
PartitionedRelPruningData *pprune = &prunedata->partrelprunedata[j]; int oldidx = pprune->subplan_map[k];
int nparts = pprune->nparts; int subidx;
int k;
/* We just rebuild present_parts from scratch */ /*
bms_free(pprune->present_parts); * If this partition existed as a subplan then change the old
pprune->present_parts = NULL; * subplan index to the new subplan index. The new index may
* become -1 if the partition was pruned above, or it may just
for (k = 0; k < nparts; k++) * come earlier in the subplan list due to some subplans being
* removed earlier in the list. If it's a subpartition, add
* it to present_parts unless it's entirely pruned.
*/
if (oldidx >= 0)
{ {
int oldidx = pprune->subplan_map[k]; Assert(oldidx < n_total_subplans);
int subidx; pprune->subplan_map[k] = new_subplan_indexes[oldidx] - 1;
/* if (new_subplan_indexes[oldidx] > 0)
* If this partition existed as a subplan then change the pprune->present_parts =
* old subplan index to the new subplan index. The new bms_add_member(pprune->present_parts, k);
* index may become -1 if the partition was pruned above, }
* or it may just come earlier in the subplan list due to else if ((subidx = pprune->subpart_map[k]) >= 0)
* some subplans being removed earlier in the list. If {
* it's a subpartition, add it to present_parts unless PartitionedRelPruningData *subprune;
* it's entirely pruned.
*/
if (oldidx >= 0)
{
Assert(oldidx < nsubplans);
pprune->subplan_map[k] = new_subplan_indexes[oldidx] - 1;
if (new_subplan_indexes[oldidx] > 0) subprune = &prunedata->partrelprunedata[subidx];
pprune->present_parts =
bms_add_member(pprune->present_parts, k);
}
else if ((subidx = pprune->subpart_map[k]) >= 0)
{
PartitionedRelPruningData *subprune;
subprune = &prunedata->partrelprunedata[subidx]; if (!bms_is_empty(subprune->present_parts))
pprune->present_parts =
if (!bms_is_empty(subprune->present_parts)) bms_add_member(pprune->present_parts, k);
pprune->present_parts =
bms_add_member(pprune->present_parts, k);
}
} }
} }
} }
/*
* We must also recompute the other_subplans set, since indexes in it
* may change.
*/
new_other_subplans = NULL;
i = -1;
while ((i = bms_next_member(prunestate->other_subplans, i)) >= 0)
new_other_subplans = bms_add_member(new_other_subplans,
new_subplan_indexes[i] - 1);
bms_free(prunestate->other_subplans);
prunestate->other_subplans = new_other_subplans;
pfree(new_subplan_indexes);
} }
return result; /*
* We must also recompute the other_subplans set, since indexes in it may
* change.
*/
new_other_subplans = NULL;
i = -1;
while ((i = bms_next_member(prunestate->other_subplans, i)) >= 0)
new_other_subplans = bms_add_member(new_other_subplans,
new_subplan_indexes[i] - 1);
bms_free(prunestate->other_subplans);
prunestate->other_subplans = new_other_subplans;
pfree(new_subplan_indexes);
} }
/* /*
@ -2089,21 +2108,24 @@ ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate, int nsubplans)
* Determine which subplans match the pruning steps detailed in * Determine which subplans match the pruning steps detailed in
* 'prunestate' for the current comparison expression values. * 'prunestate' for the current comparison expression values.
* *
* Here we assume we may evaluate PARAM_EXEC Params. * Pass initial_prune if PARAM_EXEC Params cannot yet be evaluated. This
* differentiates the initial executor-time pruning step from later
* runtime pruning.
*/ */
Bitmapset * Bitmapset *
ExecFindMatchingSubPlans(PartitionPruneState *prunestate) ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
bool initial_prune)
{ {
Bitmapset *result = NULL; Bitmapset *result = NULL;
MemoryContext oldcontext; MemoryContext oldcontext;
int i; int i;
/* /*
* If !do_exec_prune, we've got problems because * Either we're here on the initial prune done during pruning
* ExecFindInitialMatchingSubPlans will not have bothered to update * initialization, or we're at a point where PARAM_EXEC Params can be
* prunestate for whatever pruning it did. * evaluated *and* there are steps in which to do so.
*/ */
Assert(prunestate->do_exec_prune); Assert(initial_prune || prunestate->do_exec_prune);
/* /*
* Switch to a temp context to avoid leaking memory in the executor's * Switch to a temp context to avoid leaking memory in the executor's
@ -2117,17 +2139,21 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate)
*/ */
for (i = 0; i < prunestate->num_partprunedata; i++) for (i = 0; i < prunestate->num_partprunedata; i++)
{ {
PartitionPruningData *prunedata; PartitionPruningData *prunedata = prunestate->partprunedata[i];
PartitionedRelPruningData *pprune; PartitionedRelPruningData *pprune;
prunedata = prunestate->partprunedata[i]; /*
* We pass the zeroth item, belonging to the root table of the
* hierarchy, and find_matching_subplans_recurse() takes care of
* recursing to other (lower-level) parents as needed.
*/
pprune = &prunedata->partrelprunedata[0]; pprune = &prunedata->partrelprunedata[0];
find_matching_subplans_recurse(prunedata, pprune, initial_prune,
&result);
find_matching_subplans_recurse(prunedata, pprune, false, &result); /* Expression eval may have used space in ExprContext too */
/* Expression eval may have used space in node's ps_ExprContext too */
if (pprune->exec_pruning_steps) if (pprune->exec_pruning_steps)
ResetExprContext(pprune->exec_context.planstate->ps_ExprContext); ResetExprContext(pprune->exec_context.exprcontext);
} }
/* Add in any subplans that partition pruning didn't account for */ /* Add in any subplans that partition pruning didn't account for */
@ -2145,8 +2171,7 @@ ExecFindMatchingSubPlans(PartitionPruneState *prunestate)
/* /*
* find_matching_subplans_recurse * find_matching_subplans_recurse
* Recursive worker function for ExecFindMatchingSubPlans and * Recursive worker function for ExecFindMatchingSubPlans
* ExecFindInitialMatchingSubPlans
* *
* Adds valid (non-prunable) subplan IDs to *validsubplans * Adds valid (non-prunable) subplan IDs to *validsubplans
*/ */
@ -2162,25 +2187,19 @@ find_matching_subplans_recurse(PartitionPruningData *prunedata,
/* Guard against stack overflow due to overly deep partition hierarchy. */ /* Guard against stack overflow due to overly deep partition hierarchy. */
check_stack_depth(); check_stack_depth();
/* Only prune if pruning would be useful at this level. */ /*
* Prune as appropriate, if we have pruning steps matching the current
* execution context. Otherwise just include all partitions at this
* level.
*/
if (initial_prune && pprune->initial_pruning_steps) if (initial_prune && pprune->initial_pruning_steps)
{
partset = get_matching_partitions(&pprune->initial_context, partset = get_matching_partitions(&pprune->initial_context,
pprune->initial_pruning_steps); pprune->initial_pruning_steps);
}
else if (!initial_prune && pprune->exec_pruning_steps) else if (!initial_prune && pprune->exec_pruning_steps)
{
partset = get_matching_partitions(&pprune->exec_context, partset = get_matching_partitions(&pprune->exec_context,
pprune->exec_pruning_steps); pprune->exec_pruning_steps);
}
else else
{
/*
* If no pruning is to be done, just include all partitions at this
* level.
*/
partset = pprune->present_parts; partset = pprune->present_parts;
}
/* Translate partset into subplan indexes */ /* Translate partset into subplan indexes */
i = -1; i = -1;

41
src/backend/executor/nodeAppend.c
View File

@ -138,30 +138,17 @@ ExecInitAppend(Append *node, EState *estate, int eflags)
{ {
PartitionPruneState *prunestate; PartitionPruneState *prunestate;
/* We may need an expression context to evaluate partition exprs */ /*
ExecAssignExprContext(estate, &appendstate->ps); * Set up pruning data structure. This also initializes the set of
* subplans to initialize (validsubplans) by taking into account the
/* Create the working data structure for pruning. */ * result of performing initial pruning if any.
prunestate = ExecCreatePartitionPruneState(&appendstate->ps, */
node->part_prune_info); prunestate = ExecInitPartitionPruning(&appendstate->ps,
list_length(node->appendplans),
node->part_prune_info,
&validsubplans);
appendstate->as_prune_state = prunestate; appendstate->as_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
/* Perform an initial partition prune, if required. */
if (prunestate->do_initial_prune)
{
/* Determine which subplans survive initial pruning */
validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
list_length(node->appendplans));
nplans = bms_num_members(validsubplans);
}
else
{
/* We'll need to initialize all subplans */
nplans = list_length(node->appendplans);
Assert(nplans > 0);
validsubplans = bms_add_range(NULL, 0, nplans - 1);
}
/* /*
* When no run-time pruning is required and there's at least one * When no run-time pruning is required and there's at least one
@ -590,7 +577,7 @@ choose_next_subplan_locally(AppendState *node)
} }
else if (node->as_valid_subplans == NULL) else if (node->as_valid_subplans == NULL)
node->as_valid_subplans = node->as_valid_subplans =
ExecFindMatchingSubPlans(node->as_prune_state); ExecFindMatchingSubPlans(node->as_prune_state, false);
whichplan = -1; whichplan = -1;
} }
@ -655,7 +642,7 @@ choose_next_subplan_for_leader(AppendState *node)
if (node->as_valid_subplans == NULL) if (node->as_valid_subplans == NULL)
{ {
node->as_valid_subplans = node->as_valid_subplans =
ExecFindMatchingSubPlans(node->as_prune_state); ExecFindMatchingSubPlans(node->as_prune_state, false);
/* /*
* Mark each invalid plan as finished to allow the loop below to * Mark each invalid plan as finished to allow the loop below to
@ -730,7 +717,7 @@ choose_next_subplan_for_worker(AppendState *node)
else if (node->as_valid_subplans == NULL) else if (node->as_valid_subplans == NULL)
{ {
node->as_valid_subplans = node->as_valid_subplans =
ExecFindMatchingSubPlans(node->as_prune_state); ExecFindMatchingSubPlans(node->as_prune_state, false);
mark_invalid_subplans_as_finished(node); mark_invalid_subplans_as_finished(node);
} }
@ -881,7 +868,7 @@ ExecAppendAsyncBegin(AppendState *node)
if (node->as_valid_subplans == NULL) if (node->as_valid_subplans == NULL)
{ {
node->as_valid_subplans = node->as_valid_subplans =
ExecFindMatchingSubPlans(node->as_prune_state); ExecFindMatchingSubPlans(node->as_prune_state, false);
classify_matching_subplans(node); classify_matching_subplans(node);
} }

34
src/backend/executor/nodeMergeAppend.c
View File

@ -86,29 +86,17 @@ ExecInitMergeAppend(MergeAppend *node, EState *estate, int eflags)
{ {
PartitionPruneState *prunestate; PartitionPruneState *prunestate;
/* We may need an expression context to evaluate partition exprs */ /*
ExecAssignExprContext(estate, &mergestate->ps); * Set up pruning data structure. This also initializes the set of
* subplans to initialize (validsubplans) by taking into account the
prunestate = ExecCreatePartitionPruneState(&mergestate->ps, * result of performing initial pruning if any.
node->part_prune_info); */
prunestate = ExecInitPartitionPruning(&mergestate->ps,
list_length(node->mergeplans),
node->part_prune_info,
&validsubplans);
mergestate->ms_prune_state = prunestate; mergestate->ms_prune_state = prunestate;
nplans = bms_num_members(validsubplans);
/* Perform an initial partition prune, if required. */
if (prunestate->do_initial_prune)
{
/* Determine which subplans survive initial pruning */
validsubplans = ExecFindInitialMatchingSubPlans(prunestate,
list_length(node->mergeplans));
nplans = bms_num_members(validsubplans);
}
else
{
/* We'll need to initialize all subplans */
nplans = list_length(node->mergeplans);
Assert(nplans > 0);
validsubplans = bms_add_range(NULL, 0, nplans - 1);
}
/* /*
* When no run-time pruning is required and there's at least one * When no run-time pruning is required and there's at least one
@ -230,7 +218,7 @@ ExecMergeAppend(PlanState *pstate)
*/ */
if (node->ms_valid_subplans == NULL) if (node->ms_valid_subplans == NULL)
node->ms_valid_subplans = node->ms_valid_subplans =
ExecFindMatchingSubPlans(node->ms_prune_state); ExecFindMatchingSubPlans(node->ms_prune_state, false);
/* /*
* First time through: pull the first tuple from each valid subplan, * First time through: pull the first tuple from each valid subplan,

24
src/backend/partitioning/partprune.c
View File

@ -798,6 +798,7 @@ prune_append_rel_partitions(RelOptInfo *rel)
/* These are not valid when being called from the planner */ /* These are not valid when being called from the planner */
context.planstate = NULL; context.planstate = NULL;
context.exprcontext = NULL;
context.exprstates = NULL; context.exprstates = NULL;
/* Actual pruning happens here. */ /* Actual pruning happens here. */
@ -808,8 +809,8 @@ prune_append_rel_partitions(RelOptInfo *rel)
* get_matching_partitions * get_matching_partitions
* Determine partitions that survive partition pruning * Determine partitions that survive partition pruning
* *
* Note: context->planstate must be set to a valid PlanState when the * Note: context->exprcontext must be valid when the pruning_steps were
* pruning_steps were generated with a target other than PARTTARGET_PLANNER. * generated with a target other than PARTTARGET_PLANNER.
* *
* Returns a Bitmapset of the RelOptInfo->part_rels indexes of the surviving * Returns a Bitmapset of the RelOptInfo->part_rels indexes of the surviving
* partitions. * partitions.
@ -3654,9 +3655,9 @@ match_boolean_partition_clause(Oid partopfamily, Expr *clause, Expr *partkey,
* exprstate array. * exprstate array.
* *
* Note that the evaluated result may be in the per-tuple memory context of * Note that the evaluated result may be in the per-tuple memory context of
* context->planstate->ps_ExprContext, and we may have leaked other memory * context->exprcontext, and we may have leaked other memory there too.
* there too. This memory must be recovered by resetting that ExprContext * This memory must be recovered by resetting that ExprContext after
* after we're done with the pruning operation (see execPartition.c). * we're done with the pruning operation (see execPartition.c).
*/ */
static void static void
partkey_datum_from_expr(PartitionPruneContext *context, partkey_datum_from_expr(PartitionPruneContext *context,
@ -3677,13 +3678,18 @@ partkey_datum_from_expr(PartitionPruneContext *context,
ExprContext *ectx; ExprContext *ectx;
/* /*
* We should never see a non-Const in a step unless we're running in * We should never see a non-Const in a step unless the caller has
* the executor. * passed a valid ExprContext.
*
* When context->planstate is valid, context->exprcontext is same as
* context->planstate->ps_ExprContext.
*/ */
Assert(context->planstate != NULL); Assert(context->planstate != NULL || context->exprcontext != NULL);
Assert(context->planstate == NULL ||
(context->exprcontext == context->planstate->ps_ExprContext));
exprstate = context->exprstates[stateidx]; exprstate = context->exprstates[stateidx];
ectx = context->planstate->ps_ExprContext; ectx = context->exprcontext;
*value = ExecEvalExprSwitchContext(exprstate, ectx, isnull); *value = ExecEvalExprSwitchContext(exprstate, ectx, isnull);
} }
} }

31
src/include/executor/execPartition.h
View File

@ -22,6 +22,17 @@
typedef struct PartitionDispatchData *PartitionDispatch; typedef struct PartitionDispatchData *PartitionDispatch;
typedef struct PartitionTupleRouting PartitionTupleRouting; typedef struct PartitionTupleRouting PartitionTupleRouting;
extern PartitionTupleRouting *ExecSetupPartitionTupleRouting(EState *estate,
Relation rel);
extern ResultRelInfo *ExecFindPartition(ModifyTableState *mtstate,
ResultRelInfo *rootResultRelInfo,
PartitionTupleRouting *proute,
TupleTableSlot *slot,
EState *estate);
extern void ExecCleanupTupleRouting(ModifyTableState *mtstate,
PartitionTupleRouting *proute);
/* /*
* PartitionedRelPruningData - Per-partitioned-table data for run-time pruning * PartitionedRelPruningData - Per-partitioned-table data for run-time pruning
* of partitions. For a multilevel partitioned table, we have one of these * of partitions. For a multilevel partitioned table, we have one of these
@ -110,19 +121,11 @@ typedef struct PartitionPruneState
PartitionPruningData *partprunedata[FLEXIBLE_ARRAY_MEMBER]; PartitionPruningData *partprunedata[FLEXIBLE_ARRAY_MEMBER];
} PartitionPruneState; } PartitionPruneState;
extern PartitionTupleRouting *ExecSetupPartitionTupleRouting(EState *estate, extern PartitionPruneState *ExecInitPartitionPruning(PlanState *planstate,
Relation rel); int n_total_subplans,
extern ResultRelInfo *ExecFindPartition(ModifyTableState *mtstate, PartitionPruneInfo *pruneinfo,
ResultRelInfo *rootResultRelInfo, Bitmapset **initially_valid_subplans);
PartitionTupleRouting *proute, extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate,
TupleTableSlot *slot, bool initial_prune);
EState *estate);
extern void ExecCleanupTupleRouting(ModifyTableState *mtstate,
PartitionTupleRouting *proute);
extern PartitionPruneState *ExecCreatePartitionPruneState(PlanState *planstate,
PartitionPruneInfo *partitionpruneinfo);
extern Bitmapset *ExecFindMatchingSubPlans(PartitionPruneState *prunestate);
extern Bitmapset *ExecFindInitialMatchingSubPlans(PartitionPruneState *prunestate,
int nsubplans);
#endif /* EXECPARTITION_H */ #endif /* EXECPARTITION_H */

2
src/include/partitioning/partprune.h
View File

@ -41,6 +41,7 @@ struct RelOptInfo;
* subsidiary data, such as the FmgrInfos. * subsidiary data, such as the FmgrInfos.
* planstate Points to the parent plan node's PlanState when called * planstate Points to the parent plan node's PlanState when called
* during execution; NULL when called from the planner. * during execution; NULL when called from the planner.
* exprcontext ExprContext to use when evaluating pruning expressions
* exprstates Array of ExprStates, indexed as per PruneCxtStateIdx; one * exprstates Array of ExprStates, indexed as per PruneCxtStateIdx; one
* for each partition key in each pruning step. Allocated if * for each partition key in each pruning step. Allocated if
* planstate is non-NULL, otherwise NULL. * planstate is non-NULL, otherwise NULL.
@ -56,6 +57,7 @@ typedef struct PartitionPruneContext
FmgrInfo *stepcmpfuncs; FmgrInfo *stepcmpfuncs;
MemoryContext ppccontext; MemoryContext ppccontext;
PlanState *planstate; PlanState *planstate;
ExprContext *exprcontext;
ExprState **exprstates; ExprState **exprstates;
} PartitionPruneContext; } PartitionPruneContext;