postgresql/src/backend/optimizer/util/inherit.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

954 lines
31 KiB
C
Raw Normal View History

/*-------------------------------------------------------------------------
*
* inherit.c
* Routines to process child relations in inheritance trees
*
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
* Portions Copyright (c) 1994, Regents of the University of California
*
*
* IDENTIFICATION
* src/backend/optimizer/util/inherit.c
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
#include "access/sysattr.h"
#include "access/table.h"
#include "catalog/partition.h"
#include "catalog/pg_inherits.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
#include "optimizer/appendinfo.h"
#include "optimizer/inherit.h"
#include "optimizer/optimizer.h"
#include "optimizer/pathnode.h"
Fix calculation of which GENERATED columns need to be updated. We were identifying the updatable generated columns of inheritance children by transposing the calculation made for their parent. However, there's nothing that says a traditional-inheritance child can't have generated columns that aren't there in its parent, or that have different dependencies than are in the parent's expression. (At present it seems that we don't enforce that for partitioning either, which is likely wrong to some degree or other; but the case clearly needs to be handled with traditional inheritance.) Hence, drop the very-klugy-anyway "extraUpdatedCols" RTE field in favor of identifying which generated columns depend on updated columns during executor startup. In HEAD we can remove extraUpdatedCols altogether; in back branches, it's still there but always empty. Another difference between the HEAD and back-branch versions of this patch is that in HEAD we can add the new bitmap field to ResultRelInfo, but that would cause an ABI break in back branches. Like 4b3e37993, add a List field at the end of struct EState instead. Back-patch to v13. The bogus calculation is also being made in v12, but it doesn't have the same visible effect because we don't use it to decide which generated columns to recalculate; as a consequence of which the patch doesn't apply easily. I think that there might still be a demonstrable bug associated with trigger firing conditions, but that's such a weird corner-case usage that I'm content to leave it unfixed in v12. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqFshLKNvQUd1DgwJ-7tsTp=dwv7KZqXC4j2wYBV1aCDUA@mail.gmail.com Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-05 20:12:17 +01:00
#include "optimizer/plancat.h"
#include "optimizer/planmain.h"
#include "optimizer/planner.h"
#include "optimizer/prep.h"
#include "optimizer/restrictinfo.h"
#include "parser/parsetree.h"
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
#include "parser/parse_relation.h"
#include "partitioning/partdesc.h"
#include "partitioning/partprune.h"
#include "utils/rel.h"
static void expand_partitioned_rtentry(PlannerInfo *root, RelOptInfo *relinfo,
RangeTblEntry *parentrte,
Index parentRTindex, Relation parentrel,
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
Bitmapset *parent_updatedCols,
PlanRowMark *top_parentrc, LOCKMODE lockmode);
static void expand_single_inheritance_child(PlannerInfo *root,
RangeTblEntry *parentrte,
Index parentRTindex, Relation parentrel,
PlanRowMark *top_parentrc, Relation childrel,
RangeTblEntry **childrte_p,
Index *childRTindex_p);
static Bitmapset *translate_col_privs(const Bitmapset *parent_privs,
List *translated_vars);
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
static Bitmapset *translate_col_privs_multilevel(PlannerInfo *root,
RelOptInfo *rel,
Fix calculation of which GENERATED columns need to be updated. We were identifying the updatable generated columns of inheritance children by transposing the calculation made for their parent. However, there's nothing that says a traditional-inheritance child can't have generated columns that aren't there in its parent, or that have different dependencies than are in the parent's expression. (At present it seems that we don't enforce that for partitioning either, which is likely wrong to some degree or other; but the case clearly needs to be handled with traditional inheritance.) Hence, drop the very-klugy-anyway "extraUpdatedCols" RTE field in favor of identifying which generated columns depend on updated columns during executor startup. In HEAD we can remove extraUpdatedCols altogether; in back branches, it's still there but always empty. Another difference between the HEAD and back-branch versions of this patch is that in HEAD we can add the new bitmap field to ResultRelInfo, but that would cause an ABI break in back branches. Like 4b3e37993, add a List field at the end of struct EState instead. Back-patch to v13. The bogus calculation is also being made in v12, but it doesn't have the same visible effect because we don't use it to decide which generated columns to recalculate; as a consequence of which the patch doesn't apply easily. I think that there might still be a demonstrable bug associated with trigger firing conditions, but that's such a weird corner-case usage that I'm content to leave it unfixed in v12. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqFshLKNvQUd1DgwJ-7tsTp=dwv7KZqXC4j2wYBV1aCDUA@mail.gmail.com Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-05 20:12:17 +01:00
RelOptInfo *parent_rel,
Bitmapset *parent_cols);
static void expand_appendrel_subquery(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte, Index rti);
/*
* expand_inherited_rtentry
* Expand a rangetable entry that has the "inh" bit set.
*
* "inh" is only allowed in two cases: RELATION and SUBQUERY RTEs.
*
* "inh" on a plain RELATION RTE means that it is a partitioned table or the
* parent of a traditional-inheritance set. In this case we must add entries
* for all the interesting child tables to the query's rangetable, and build
* additional planner data structures for them, including RelOptInfos,
* AppendRelInfos, and possibly PlanRowMarks.
*
* Note that the original RTE is considered to represent the whole inheritance
* set. In the case of traditional inheritance, the first of the generated
* RTEs is an RTE for the same table, but with inh = false, to represent the
* parent table in its role as a simple member of the inheritance set. For
* partitioning, we don't need a second RTE because the partitioned table
* itself has no data and need not be scanned.
*
* "inh" on a SUBQUERY RTE means that it's the parent of a UNION ALL group,
* which is treated as an appendrel similarly to inheritance cases; however,
* we already made RTEs and AppendRelInfos for the subqueries. We only need
* to build RelOptInfos for them, which is done by expand_appendrel_subquery.
*/
void
expand_inherited_rtentry(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte, Index rti)
{
Oid parentOID;
Relation oldrelation;
LOCKMODE lockmode;
PlanRowMark *oldrc;
bool old_isParent = false;
int old_allMarkTypes = 0;
Assert(rte->inh); /* else caller error */
if (rte->rtekind == RTE_SUBQUERY)
{
expand_appendrel_subquery(root, rel, rte, rti);
return;
}
Assert(rte->rtekind == RTE_RELATION);
parentOID = rte->relid;
/*
* We used to check has_subclass() here, but there's no longer any need
* to, because subquery_planner already did.
*/
/*
* The rewriter should already have obtained an appropriate lock on each
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
* relation named in the query, so we can open the parent relation without
* locking it. However, for each child relation we add to the query, we
* must obtain an appropriate lock, because this will be the first use of
* those relations in the parse/rewrite/plan pipeline. Child rels should
* use the same lockmode as their parent.
*/
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
oldrelation = table_open(parentOID, NoLock);
lockmode = rte->rellockmode;
/*
* If parent relation is selected FOR UPDATE/SHARE, we need to mark its
* PlanRowMark as isParent = true, and generate a new PlanRowMark for each
* child.
*/
oldrc = get_plan_rowmark(root->rowMarks, rti);
if (oldrc)
{
old_isParent = oldrc->isParent;
oldrc->isParent = true;
/* Save initial value of allMarkTypes before children add to it */
old_allMarkTypes = oldrc->allMarkTypes;
}
/* Scan the inheritance set and expand it */
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
if (oldrelation->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
RTEPermissionInfo *perminfo;
perminfo = getRTEPermissionInfo(root->parse->rteperminfos, rte);
/*
* Partitioned table, so set up for partitioning.
*/
Assert(rte->relkind == RELKIND_PARTITIONED_TABLE);
/*
* Recursively expand and lock the partitions. While at it, also
* extract the partition key columns of all the partitioned tables.
*/
expand_partitioned_rtentry(root, rel, rte, rti,
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
oldrelation,
perminfo->updatedCols,
oldrc, lockmode);
}
else
{
/*
* Ordinary table, so process traditional-inheritance children. (Note
* that partitioned tables are not allowed to have inheritance
* children, so it's not possible for both cases to apply.)
*/
List *inhOIDs;
ListCell *l;
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
/* Scan for all members of inheritance set, acquire needed locks */
inhOIDs = find_all_inheritors(parentOID, lockmode, NULL);
/*
* We used to special-case the situation where the table no longer has
* any children, by clearing rte->inh and exiting. That no longer
* works, because this function doesn't get run until after decisions
* have been made that depend on rte->inh. We have to treat such
* situations as normal inheritance. The table itself should always
* have been found, though.
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
*/
Assert(inhOIDs != NIL);
Assert(linitial_oid(inhOIDs) == parentOID);
/* Expand simple_rel_array and friends to hold child objects. */
expand_planner_arrays(root, list_length(inhOIDs));
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
/*
* Expand inheritance children in the order the OIDs were returned by
* find_all_inheritors.
*/
foreach(l, inhOIDs)
{
Oid childOID = lfirst_oid(l);
Relation newrelation;
RangeTblEntry *childrte;
Index childRTindex;
/* Open rel if needed; we already have required locks */
if (childOID != parentOID)
newrelation = table_open(childOID, NoLock);
else
newrelation = oldrelation;
/*
* It is possible that the parent table has children that are temp
* tables of other backends. We cannot safely access such tables
* (because of buffering issues), and the best thing to do seems
* to be to silently ignore them.
*/
if (childOID != parentOID && RELATION_IS_OTHER_TEMP(newrelation))
{
table_close(newrelation, lockmode);
continue;
}
/* Create RTE and AppendRelInfo, plus PlanRowMark if needed. */
expand_single_inheritance_child(root, rte, rti, oldrelation,
oldrc, newrelation,
&childrte, &childRTindex);
/* Create the otherrel RelOptInfo too. */
(void) build_simple_rel(root, childRTindex, rel);
/* Close child relations, but keep locks */
if (childOID != parentOID)
table_close(newrelation, NoLock);
}
}
/*
* Some children might require different mark types, which would've been
* reported into oldrc. If so, add relevant entries to the top-level
* targetlist and update parent rel's reltarget. This should match what
* preprocess_targetlist() would have added if the mark types had been
* requested originally.
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity information such as CTID. ModifyTable must re-fetch the original tuple to merge in the old values of any unchanged columns. The core advantage of this is that the changed columns are uniform across all tables of an inherited or partitioned target relation, whereas the other columns might not be. A secondary advantage, when the UPDATE involves joins, is that less data needs to pass through the plan tree. The disadvantage of course is an extra fetch of each tuple to be updated. However, that seems to be very nearly free in context; even worst-case tests don't show it to add more than a couple percent to the total query cost. At some point it might be interesting to combine the re-fetch with the tuple access that ModifyTable must do anyway to mark the old tuple dead; but that would require a good deal of refactoring and it seems it wouldn't buy all that much, so this patch doesn't attempt it. 2. For inherited UPDATE/DELETE, instead of generating a separate subplan for each target relation, we now generate a single subplan that is just exactly like a SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know which target relation a given incoming row refers to, a tableoid junk column is added to the row identity information. This gets rid of the horrid hack that was inheritance_planner(), eliminating O(N^2) planning cost and memory consumption in cases where there were many unprunable target relations. Point 2 of course requires point 1, so that there is a uniform definition of the non-junk columns to be returned by the subplan. We can't insist on uniform definition of the row identity junk columns however, if we want to keep the ability to have both plain and foreign tables in a partitioning hierarchy. Since it wouldn't scale very far to have every child table have its own row identity column, this patch includes provisions to merge similar row identity columns into one column of the subplan result. In particular, we can merge the whole-row Vars typically used as row identity by FDWs into one column by pretending they are type RECORD. (It's still okay for the actual composite Datums to be labeled with the table's rowtype OID, though.) There is more that can be done to file down residual inefficiencies in this patch, but it seems to be committable now. FDW authors should note several API changes: * The argument list for AddForeignUpdateTargets() has changed, and so has the method it must use for adding junk columns to the query. Call add_row_identity_var() instead of manipulating the parse tree directly. You might want to reconsider exactly what you're adding, too. * PlanDirectModify() must now work a little harder to find the ForeignScan plan node; if the foreign table is part of a partitioning hierarchy then the ForeignScan might not be the direct child of ModifyTable. See postgres_fdw for sample code. * To check whether a relation is a target relation, it's no longer sufficient to compare its relid to root->parse->resultRelation. Instead, check it against all_result_relids or leaf_result_relids, as appropriate. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
*
* (Someday it might be useful to fold these resjunk columns into the
* row-identity-column management used for UPDATE/DELETE. Today is not
* that day, however.)
*/
if (oldrc)
{
int new_allMarkTypes = oldrc->allMarkTypes;
Var *var;
TargetEntry *tle;
char resname[32];
List *newvars = NIL;
/* Add TID junk Var if needed, unless we had it already */
if (new_allMarkTypes & ~(1 << ROW_MARK_COPY) &&
!(old_allMarkTypes & ~(1 << ROW_MARK_COPY)))
{
/* Need to fetch TID */
var = makeVar(oldrc->rti,
SelfItemPointerAttributeNumber,
TIDOID,
-1,
InvalidOid,
0);
snprintf(resname, sizeof(resname), "ctid%u", oldrc->rowmarkId);
tle = makeTargetEntry((Expr *) var,
list_length(root->processed_tlist) + 1,
pstrdup(resname),
true);
root->processed_tlist = lappend(root->processed_tlist, tle);
newvars = lappend(newvars, var);
}
/* Add whole-row junk Var if needed, unless we had it already */
if ((new_allMarkTypes & (1 << ROW_MARK_COPY)) &&
!(old_allMarkTypes & (1 << ROW_MARK_COPY)))
{
var = makeWholeRowVar(planner_rt_fetch(oldrc->rti, root),
oldrc->rti,
0,
false);
snprintf(resname, sizeof(resname), "wholerow%u", oldrc->rowmarkId);
tle = makeTargetEntry((Expr *) var,
list_length(root->processed_tlist) + 1,
pstrdup(resname),
true);
root->processed_tlist = lappend(root->processed_tlist, tle);
newvars = lappend(newvars, var);
}
/* Add tableoid junk Var, unless we had it already */
if (!old_isParent)
{
var = makeVar(oldrc->rti,
TableOidAttributeNumber,
OIDOID,
-1,
InvalidOid,
0);
snprintf(resname, sizeof(resname), "tableoid%u", oldrc->rowmarkId);
tle = makeTargetEntry((Expr *) var,
list_length(root->processed_tlist) + 1,
pstrdup(resname),
true);
root->processed_tlist = lappend(root->processed_tlist, tle);
newvars = lappend(newvars, var);
}
/*
* Add the newly added Vars to parent's reltarget. We needn't worry
* about the children's reltargets, they'll be made later.
*/
add_vars_to_targetlist(root, newvars, bms_make_singleton(0));
}
table_close(oldrelation, NoLock);
}
/*
* expand_partitioned_rtentry
* Recursively expand an RTE for a partitioned table.
*/
static void
expand_partitioned_rtentry(PlannerInfo *root, RelOptInfo *relinfo,
RangeTblEntry *parentrte,
Index parentRTindex, Relation parentrel,
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
Bitmapset *parent_updatedCols,
PlanRowMark *top_parentrc, LOCKMODE lockmode)
{
Allow ATTACH PARTITION with only ShareUpdateExclusiveLock. We still require AccessExclusiveLock on the partition itself, because otherwise an insert that violates the newly-imposed partition constraint could be in progress at the same time that we're changing that constraint; only the lock level on the parent relation is weakened. To make this safe, we have to cope with (at least) three separate problems. First, relevant DDL might commit while we're in the process of building a PartitionDesc. If so, find_inheritance_children() might see a new partition while the RELOID system cache still has the old partition bound cached, and even before invalidation messages have been queued. To fix that, if we see that the pg_class tuple seems to be missing or to have a null relpartbound, refetch the value directly from the table. We can't get the wrong value, because DETACH PARTITION still requires AccessExclusiveLock throughout; if we ever want to change that, this will need more thought. In testing, I found it quite difficult to hit even the null-relpartbound case; the race condition is extremely tight, but the theoretical risk is there. Second, successive calls to RelationGetPartitionDesc might not return the same answer. The query planner will get confused if lookup up the PartitionDesc for a particular relation does not return a consistent answer for the entire duration of query planning. Likewise, query execution will get confused if the same relation seems to have a different PartitionDesc at different times. Invent a new PartitionDirectory concept and use it to ensure consistency. This ensures that a single invocation of either the planner or the executor sees the same view of the PartitionDesc from beginning to end, but it does not guarantee that the planner and the executor see the same view. Since this allows pointers to old PartitionDesc entries to survive even after a relcache rebuild, also postpone removing the old PartitionDesc entry until we're certain no one is using it. For the most part, it seems to be OK for the planner and executor to have different views of the PartitionDesc, because the executor will just ignore any concurrently added partitions which were unknown at plan time; those partitions won't be part of the inheritance expansion, but invalidation messages will trigger replanning at some point. Normally, this happens by the time the very next command is executed, but if the next command acquires no locks and executes a prepared query, it can manage not to notice until a new transaction is started. We might want to tighten that up, but it's material for a separate patch. There would still be a small window where a query that started just after an ATTACH PARTITION command committed might fail to notice its results -- but only if the command starts before the commit has been acknowledged to the user. All in all, the warts here around serializability seem small enough to be worth accepting for the considerable advantage of being able to add partitions without a full table lock. Although in general the consequences of new partitions showing up between planning and execution are limited to the query not noticing the new partitions, run-time partition pruning will get confused in that case, so that's the third problem that this patch fixes. Run-time partition pruning assumes that indexes into the PartitionDesc are stable between planning and execution. So, add code so that if new partitions are added between plan time and execution time, the indexes stored in the subplan_map[] and subpart_map[] arrays within the plan's PartitionedRelPruneInfo get adjusted accordingly. There does not seem to be a simple way to generalize this scheme to cope with partitions that are removed, mostly because they could then get added back again with different bounds, but it works OK for added partitions. This code does not try to ensure that every backend participating in a parallel query sees the same view of the PartitionDesc. That currently doesn't matter, because we never pass PartitionDesc indexes between backends. Each backend will ignore the concurrently added partitions which it notices, and it doesn't matter if different backends are ignoring different sets of concurrently added partitions. If in the future that matters, for example because we allow writes in parallel query and want all participants to do tuple routing to the same set of partitions, the PartitionDirectory concept could be improved to share PartitionDescs across backends. There is a draft patch to serialize and restore PartitionDescs on the thread where this patch was discussed, which may be a useful place to start. Patch by me. Thanks to Alvaro Herrera, David Rowley, Simon Riggs, Amit Langote, and Michael Paquier for discussion, and to Alvaro Herrera for some review. Discussion: http://postgr.es/m/CA+Tgmobt2upbSocvvDej3yzokd7AkiT+PvgFH+a9-5VV1oJNSQ@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZE0r9-cyA-aY6f8WFEROaDLLL7Vf81kZ8MtFCkxpeQSw@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoY13KQZF-=HNTrt9UYWYx3_oYOQpu9ioNT49jGgiDpUEA@mail.gmail.com
2019-03-07 17:13:12 +01:00
PartitionDesc partdesc;
Bitmapset *live_parts;
int num_live_parts;
int i;
check_stack_depth();
Assert(parentrte->inh);
Allow ATTACH PARTITION with only ShareUpdateExclusiveLock. We still require AccessExclusiveLock on the partition itself, because otherwise an insert that violates the newly-imposed partition constraint could be in progress at the same time that we're changing that constraint; only the lock level on the parent relation is weakened. To make this safe, we have to cope with (at least) three separate problems. First, relevant DDL might commit while we're in the process of building a PartitionDesc. If so, find_inheritance_children() might see a new partition while the RELOID system cache still has the old partition bound cached, and even before invalidation messages have been queued. To fix that, if we see that the pg_class tuple seems to be missing or to have a null relpartbound, refetch the value directly from the table. We can't get the wrong value, because DETACH PARTITION still requires AccessExclusiveLock throughout; if we ever want to change that, this will need more thought. In testing, I found it quite difficult to hit even the null-relpartbound case; the race condition is extremely tight, but the theoretical risk is there. Second, successive calls to RelationGetPartitionDesc might not return the same answer. The query planner will get confused if lookup up the PartitionDesc for a particular relation does not return a consistent answer for the entire duration of query planning. Likewise, query execution will get confused if the same relation seems to have a different PartitionDesc at different times. Invent a new PartitionDirectory concept and use it to ensure consistency. This ensures that a single invocation of either the planner or the executor sees the same view of the PartitionDesc from beginning to end, but it does not guarantee that the planner and the executor see the same view. Since this allows pointers to old PartitionDesc entries to survive even after a relcache rebuild, also postpone removing the old PartitionDesc entry until we're certain no one is using it. For the most part, it seems to be OK for the planner and executor to have different views of the PartitionDesc, because the executor will just ignore any concurrently added partitions which were unknown at plan time; those partitions won't be part of the inheritance expansion, but invalidation messages will trigger replanning at some point. Normally, this happens by the time the very next command is executed, but if the next command acquires no locks and executes a prepared query, it can manage not to notice until a new transaction is started. We might want to tighten that up, but it's material for a separate patch. There would still be a small window where a query that started just after an ATTACH PARTITION command committed might fail to notice its results -- but only if the command starts before the commit has been acknowledged to the user. All in all, the warts here around serializability seem small enough to be worth accepting for the considerable advantage of being able to add partitions without a full table lock. Although in general the consequences of new partitions showing up between planning and execution are limited to the query not noticing the new partitions, run-time partition pruning will get confused in that case, so that's the third problem that this patch fixes. Run-time partition pruning assumes that indexes into the PartitionDesc are stable between planning and execution. So, add code so that if new partitions are added between plan time and execution time, the indexes stored in the subplan_map[] and subpart_map[] arrays within the plan's PartitionedRelPruneInfo get adjusted accordingly. There does not seem to be a simple way to generalize this scheme to cope with partitions that are removed, mostly because they could then get added back again with different bounds, but it works OK for added partitions. This code does not try to ensure that every backend participating in a parallel query sees the same view of the PartitionDesc. That currently doesn't matter, because we never pass PartitionDesc indexes between backends. Each backend will ignore the concurrently added partitions which it notices, and it doesn't matter if different backends are ignoring different sets of concurrently added partitions. If in the future that matters, for example because we allow writes in parallel query and want all participants to do tuple routing to the same set of partitions, the PartitionDirectory concept could be improved to share PartitionDescs across backends. There is a draft patch to serialize and restore PartitionDescs on the thread where this patch was discussed, which may be a useful place to start. Patch by me. Thanks to Alvaro Herrera, David Rowley, Simon Riggs, Amit Langote, and Michael Paquier for discussion, and to Alvaro Herrera for some review. Discussion: http://postgr.es/m/CA+Tgmobt2upbSocvvDej3yzokd7AkiT+PvgFH+a9-5VV1oJNSQ@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZE0r9-cyA-aY6f8WFEROaDLLL7Vf81kZ8MtFCkxpeQSw@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoY13KQZF-=HNTrt9UYWYx3_oYOQpu9ioNT49jGgiDpUEA@mail.gmail.com
2019-03-07 17:13:12 +01:00
partdesc = PartitionDirectoryLookup(root->glob->partition_directory,
parentrel);
/* A partitioned table should always have a partition descriptor. */
Assert(partdesc);
/*
* Note down whether any partition key cols are being updated. Though it's
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
* the root partitioned table's updatedCols we are interested in,
* parent_updatedCols provided by the caller contains the root partrel's
* updatedCols translated to match the attribute ordering of parentrel.
*/
if (!root->partColsUpdated)
root->partColsUpdated =
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
has_partition_attrs(parentrel, parent_updatedCols, NULL);
/* Nothing further to do here if there are no partitions. */
if (partdesc->nparts == 0)
return;
/*
* Perform partition pruning using restriction clauses assigned to parent
* relation. live_parts will contain PartitionDesc indexes of partitions
* that survive pruning. Below, we will initialize child objects for the
* surviving partitions.
*/
relinfo->live_parts = live_parts = prune_append_rel_partitions(relinfo);
/* Expand simple_rel_array and friends to hold child objects. */
num_live_parts = bms_num_members(live_parts);
if (num_live_parts > 0)
expand_planner_arrays(root, num_live_parts);
/*
* We also store partition RelOptInfo pointers in the parent relation.
* Since we're palloc0'ing, slots corresponding to pruned partitions will
* contain NULL.
*/
Assert(relinfo->part_rels == NULL);
relinfo->part_rels = (RelOptInfo **)
palloc0(relinfo->nparts * sizeof(RelOptInfo *));
/*
* Create a child RTE for each live partition. Note that unlike
* traditional inheritance, we don't need a child RTE for the partitioned
* table itself, because it's not going to be scanned.
*/
i = -1;
while ((i = bms_next_member(live_parts, i)) >= 0)
{
Oid childOID = partdesc->oids[i];
Relation childrel;
RangeTblEntry *childrte;
Index childRTindex;
RelOptInfo *childrelinfo;
Change lock acquisition order in expand_inherited_rtentry. Previously, this function acquired locks in the order using find_all_inheritors(), which locks the children of each table that it processes in ascending OID order, and which processes the inheritance hierarchy as a whole in a breadth-first fashion. Now, it processes the inheritance hierarchy in a depth-first fashion, and at each level it proceeds in the order in which tables appear in the PartitionDesc. If table inheritance rather than table partitioning is used, the old order is preserved. This change moves the locking of any given partition much closer to the code that actually expands that partition. This seems essential if we ever want to allow concurrent DDL to add or remove partitions, because if the set of partitions can change, we must use the same data to decide which partitions to lock as we do to decide which partitions to expand; otherwise, we might expand a partition that we haven't locked. It should hopefully also facilitate efforts to postpone inheritance expansion or locking for performance reasons, because there's really no way to postpone locking some partitions if we're blindly locking them all using find_all_inheritors(). The only downside of this change which is known to me is that it further deviates from the principle that we should always lock the inheritance hierarchy in find_all_inheritors() order to avoid deadlock risk. However, we've already crossed that bridge in commit 9eefba181f7782d27d85d7e94e6028371e7ab2d7 and there are futher patches pending that make similar changes, so this isn't really giving up anything that we haven't surrendered already -- and it seems entirely worth it, given the performance benefits some of those changes seem likely to bring. Patch by me; thanks to David Rowley for discussion of these issues. Discussion: http://postgr.es/m/CAKJS1f_eEYVEq5tM8sm1k-HOwG0AyCPwX54XG9x4w0zy_N4Q_Q@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZUwPf_uanjF==gTGBMJrn8uCq52XYvAEorNkLrUdoawg@mail.gmail.com
2019-02-26 18:22:57 +01:00
/* Open rel, acquiring required locks */
childrel = table_open(childOID, lockmode);
/*
* Temporary partitions belonging to other sessions should have been
* disallowed at definition, but for paranoia's sake, let's double
* check.
*/
if (RELATION_IS_OTHER_TEMP(childrel))
elog(ERROR, "temporary relation from another session found as partition");
/* Create RTE and AppendRelInfo, plus PlanRowMark if needed. */
expand_single_inheritance_child(root, parentrte, parentRTindex,
parentrel, top_parentrc, childrel,
&childrte, &childRTindex);
/* Create the otherrel RelOptInfo too. */
childrelinfo = build_simple_rel(root, childRTindex, relinfo);
relinfo->part_rels[i] = childrelinfo;
relinfo->all_partrels = bms_add_members(relinfo->all_partrels,
childrelinfo->relids);
/* If this child is itself partitioned, recurse */
if (childrel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
{
AppendRelInfo *appinfo = root->append_rel_array[childRTindex];
Bitmapset *child_updatedCols;
child_updatedCols = translate_col_privs(parent_updatedCols,
appinfo->translated_vars);
expand_partitioned_rtentry(root, childrelinfo,
childrte, childRTindex,
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
childrel,
child_updatedCols,
top_parentrc, lockmode);
}
/* Close child relation, but keep locks */
table_close(childrel, NoLock);
}
}
/*
* expand_single_inheritance_child
* Build a RangeTblEntry and an AppendRelInfo, plus maybe a PlanRowMark.
*
* We now expand the partition hierarchy level by level, creating a
* corresponding hierarchy of AppendRelInfos and RelOptInfos, where each
* partitioned descendant acts as a parent of its immediate partitions.
* (This is a difference from what older versions of PostgreSQL did and what
* is still done in the case of table inheritance for unpartitioned tables,
* where the hierarchy is flattened during RTE expansion.)
*
* PlanRowMarks still carry the top-parent's RTI, and the top-parent's
* allMarkTypes field still accumulates values from all descendents.
*
* "parentrte" and "parentRTindex" are immediate parent's RTE and
* RTI. "top_parentrc" is top parent's PlanRowMark.
*
* The child RangeTblEntry and its RTI are returned in "childrte_p" and
* "childRTindex_p" resp.
*/
static void
expand_single_inheritance_child(PlannerInfo *root, RangeTblEntry *parentrte,
Index parentRTindex, Relation parentrel,
PlanRowMark *top_parentrc, Relation childrel,
RangeTblEntry **childrte_p,
Index *childRTindex_p)
{
Query *parse = root->parse;
Oid parentOID = RelationGetRelid(parentrel);
Oid childOID = RelationGetRelid(childrel);
RangeTblEntry *childrte;
Index childRTindex;
AppendRelInfo *appinfo;
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
TupleDesc child_tupdesc;
List *parent_colnames;
List *child_colnames;
/*
* Build an RTE for the child, and attach to query's rangetable list. We
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
* copy most scalar fields of the parent's RTE, but replace relation OID,
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
* relkind, and inh for the child. Set the child's securityQuals to
* empty, because we only want to apply the parent's RLS conditions
* regardless of what RLS properties individual children may have. (This
* is an intentional choice to make inherited RLS work like regular
* permissions checks.) The parent securityQuals will be propagated to
* children along with other base restriction clauses, so we don't need to
* do it here. Other infrastructure of the parent RTE has to be
* translated to match the child table's column ordering, which we do
* below, so a "flat" copy is sufficient to start with.
*/
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
childrte = makeNode(RangeTblEntry);
memcpy(childrte, parentrte, sizeof(RangeTblEntry));
Assert(parentrte->rtekind == RTE_RELATION); /* else this is dubious */
childrte->relid = childOID;
childrte->relkind = childrel->rd_rel->relkind;
/* A partitioned child will need to be expanded further. */
if (childrte->relkind == RELKIND_PARTITIONED_TABLE)
{
Assert(childOID != parentOID);
childrte->inh = true;
}
else
childrte->inh = false;
childrte->securityQuals = NIL;
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
/*
* No permission checking for the child RTE unless it's the parent
* relation in its child role, which only applies to traditional
* inheritance.
*/
if (childOID != parentOID)
childrte->perminfoindex = 0;
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
/* Link not-yet-fully-filled child RTE into data structures */
parse->rtable = lappend(parse->rtable, childrte);
childRTindex = list_length(parse->rtable);
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
*childrte_p = childrte;
*childRTindex_p = childRTindex;
/*
* Build an AppendRelInfo struct for each parent/child pair.
*/
appinfo = make_append_rel_info(parentrel, childrel,
parentRTindex, childRTindex);
root->append_rel_list = lappend(root->append_rel_list, appinfo);
Fix EXPLAIN's column alias output for mismatched child tables. If an inheritance/partitioning parent table is assigned some column alias names in the query, EXPLAIN mapped those aliases onto the child tables' columns by physical position, resulting in bogus output if a child table's columns aren't one-for-one with the parent's. To fix, make expand_single_inheritance_child() generate a correctly re-mapped column alias list, rather than just copying the parent RTE's alias node. (We have to fill the alias field, not just adjust the eref field, because ruleutils.c will ignore eref in favor of looking at the real column names.) This means that child tables will now always have alias fields in plan rtables, where before they might not have. That results in a rather substantial set of regression test output changes: EXPLAIN will now always show child tables with aliases that match the parent table (usually with "_N" appended for uniqueness). But that seems like a net positive for understandability, since the parent alias corresponds to something that actually appeared in the original query, while the child table names didn't. (Note that this does not change anything for cases where an explicit table alias was written in the query for the parent table; it just makes cases without such aliases behave similarly to that.) Hence, while we could avoid these subsidiary changes if we made inherit.c more complicated, we choose not to. Discussion: https://postgr.es/m/12424.1575168015@sss.pgh.pa.us
2019-12-03 01:08:10 +01:00
/* tablesample is probably null, but copy it */
childrte->tablesample = copyObject(parentrte->tablesample);
/*
* Construct an alias clause for the child, which we can also use as eref.
* This is important so that EXPLAIN will print the right column aliases
* for child-table columns. (Since ruleutils.c doesn't have any easy way
* to reassociate parent and child columns, we must get the child column
* aliases right to start with. Note that setting childrte->alias forces
* ruleutils.c to use these column names, which it otherwise would not.)
*/
child_tupdesc = RelationGetDescr(childrel);
parent_colnames = parentrte->eref->colnames;
child_colnames = NIL;
for (int cattno = 0; cattno < child_tupdesc->natts; cattno++)
{
Form_pg_attribute att = TupleDescAttr(child_tupdesc, cattno);
const char *attname;
if (att->attisdropped)
{
/* Always insert an empty string for a dropped column */
attname = "";
}
else if (appinfo->parent_colnos[cattno] > 0 &&
appinfo->parent_colnos[cattno] <= list_length(parent_colnames))
{
/* Duplicate the query-assigned name for the parent column */
attname = strVal(list_nth(parent_colnames,
appinfo->parent_colnos[cattno] - 1));
}
else
{
/* New column, just use its real name */
attname = NameStr(att->attname);
}
child_colnames = lappend(child_colnames, makeString(pstrdup(attname)));
}
/*
* We just duplicate the parent's table alias name for each child. If the
* plan gets printed, ruleutils.c has to sort out unique table aliases to
* use, which it can handle.
*/
childrte->alias = childrte->eref = makeAlias(parentrte->eref->aliasname,
child_colnames);
/*
* Store the RTE and appinfo in the respective PlannerInfo arrays, which
* the caller must already have allocated space for.
*/
Assert(childRTindex < root->simple_rel_array_size);
Assert(root->simple_rte_array[childRTindex] == NULL);
root->simple_rte_array[childRTindex] = childrte;
Assert(root->append_rel_array[childRTindex] == NULL);
root->append_rel_array[childRTindex] = appinfo;
/*
* Build a PlanRowMark if parent is marked FOR UPDATE/SHARE.
*/
if (top_parentrc)
{
PlanRowMark *childrc = makeNode(PlanRowMark);
childrc->rti = childRTindex;
childrc->prti = top_parentrc->rti;
childrc->rowmarkId = top_parentrc->rowmarkId;
/* Reselect rowmark type, because relkind might not match parent */
childrc->markType = select_rowmark_type(childrte,
top_parentrc->strength);
childrc->allMarkTypes = (1 << childrc->markType);
childrc->strength = top_parentrc->strength;
childrc->waitPolicy = top_parentrc->waitPolicy;
/*
* We mark RowMarks for partitioned child tables as parent RowMarks so
* that the executor ignores them (except their existence means that
* the child tables will be locked using the appropriate mode).
*/
childrc->isParent = (childrte->relkind == RELKIND_PARTITIONED_TABLE);
/* Include child's rowmark type in top parent's allMarkTypes */
top_parentrc->allMarkTypes |= childrc->allMarkTypes;
root->rowMarks = lappend(root->rowMarks, childrc);
}
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity information such as CTID. ModifyTable must re-fetch the original tuple to merge in the old values of any unchanged columns. The core advantage of this is that the changed columns are uniform across all tables of an inherited or partitioned target relation, whereas the other columns might not be. A secondary advantage, when the UPDATE involves joins, is that less data needs to pass through the plan tree. The disadvantage of course is an extra fetch of each tuple to be updated. However, that seems to be very nearly free in context; even worst-case tests don't show it to add more than a couple percent to the total query cost. At some point it might be interesting to combine the re-fetch with the tuple access that ModifyTable must do anyway to mark the old tuple dead; but that would require a good deal of refactoring and it seems it wouldn't buy all that much, so this patch doesn't attempt it. 2. For inherited UPDATE/DELETE, instead of generating a separate subplan for each target relation, we now generate a single subplan that is just exactly like a SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know which target relation a given incoming row refers to, a tableoid junk column is added to the row identity information. This gets rid of the horrid hack that was inheritance_planner(), eliminating O(N^2) planning cost and memory consumption in cases where there were many unprunable target relations. Point 2 of course requires point 1, so that there is a uniform definition of the non-junk columns to be returned by the subplan. We can't insist on uniform definition of the row identity junk columns however, if we want to keep the ability to have both plain and foreign tables in a partitioning hierarchy. Since it wouldn't scale very far to have every child table have its own row identity column, this patch includes provisions to merge similar row identity columns into one column of the subplan result. In particular, we can merge the whole-row Vars typically used as row identity by FDWs into one column by pretending they are type RECORD. (It's still okay for the actual composite Datums to be labeled with the table's rowtype OID, though.) There is more that can be done to file down residual inefficiencies in this patch, but it seems to be committable now. FDW authors should note several API changes: * The argument list for AddForeignUpdateTargets() has changed, and so has the method it must use for adding junk columns to the query. Call add_row_identity_var() instead of manipulating the parse tree directly. You might want to reconsider exactly what you're adding, too. * PlanDirectModify() must now work a little harder to find the ForeignScan plan node; if the foreign table is part of a partitioning hierarchy then the ForeignScan might not be the direct child of ModifyTable. See postgres_fdw for sample code. * To check whether a relation is a target relation, it's no longer sufficient to compare its relid to root->parse->resultRelation. Instead, check it against all_result_relids or leaf_result_relids, as appropriate. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
/*
* If we are creating a child of the query target relation (only possible
* in UPDATE/DELETE/MERGE), add it to all_result_relids, as well as
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity information such as CTID. ModifyTable must re-fetch the original tuple to merge in the old values of any unchanged columns. The core advantage of this is that the changed columns are uniform across all tables of an inherited or partitioned target relation, whereas the other columns might not be. A secondary advantage, when the UPDATE involves joins, is that less data needs to pass through the plan tree. The disadvantage of course is an extra fetch of each tuple to be updated. However, that seems to be very nearly free in context; even worst-case tests don't show it to add more than a couple percent to the total query cost. At some point it might be interesting to combine the re-fetch with the tuple access that ModifyTable must do anyway to mark the old tuple dead; but that would require a good deal of refactoring and it seems it wouldn't buy all that much, so this patch doesn't attempt it. 2. For inherited UPDATE/DELETE, instead of generating a separate subplan for each target relation, we now generate a single subplan that is just exactly like a SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know which target relation a given incoming row refers to, a tableoid junk column is added to the row identity information. This gets rid of the horrid hack that was inheritance_planner(), eliminating O(N^2) planning cost and memory consumption in cases where there were many unprunable target relations. Point 2 of course requires point 1, so that there is a uniform definition of the non-junk columns to be returned by the subplan. We can't insist on uniform definition of the row identity junk columns however, if we want to keep the ability to have both plain and foreign tables in a partitioning hierarchy. Since it wouldn't scale very far to have every child table have its own row identity column, this patch includes provisions to merge similar row identity columns into one column of the subplan result. In particular, we can merge the whole-row Vars typically used as row identity by FDWs into one column by pretending they are type RECORD. (It's still okay for the actual composite Datums to be labeled with the table's rowtype OID, though.) There is more that can be done to file down residual inefficiencies in this patch, but it seems to be committable now. FDW authors should note several API changes: * The argument list for AddForeignUpdateTargets() has changed, and so has the method it must use for adding junk columns to the query. Call add_row_identity_var() instead of manipulating the parse tree directly. You might want to reconsider exactly what you're adding, too. * PlanDirectModify() must now work a little harder to find the ForeignScan plan node; if the foreign table is part of a partitioning hierarchy then the ForeignScan might not be the direct child of ModifyTable. See postgres_fdw for sample code. * To check whether a relation is a target relation, it's no longer sufficient to compare its relid to root->parse->resultRelation. Instead, check it against all_result_relids or leaf_result_relids, as appropriate. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
* leaf_result_relids if appropriate, and make sure that we generate
* required row-identity data.
*/
if (bms_is_member(parentRTindex, root->all_result_relids))
{
/* OK, record the child as a result rel too. */
root->all_result_relids = bms_add_member(root->all_result_relids,
childRTindex);
/* Non-leaf partitions don't need any row identity info. */
if (childrte->relkind != RELKIND_PARTITIONED_TABLE)
{
Var *rrvar;
root->leaf_result_relids = bms_add_member(root->leaf_result_relids,
childRTindex);
/*
* If we have any child target relations, assume they all need to
* generate a junk "tableoid" column. (If only one child survives
* pruning, we wouldn't really need this, but it's not worth
* thrashing about to avoid it.)
*/
rrvar = makeVar(childRTindex,
TableOidAttributeNumber,
OIDOID,
-1,
InvalidOid,
0);
add_row_identity_var(root, rrvar, childRTindex, "tableoid");
/* Register any row-identity columns needed by this child. */
add_row_identity_columns(root, childRTindex,
childrte, childrel);
}
}
}
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
/*
* get_rel_all_updated_cols
* Returns the set of columns of a given "simple" relation that are
* updated by this query.
*/
Bitmapset *
get_rel_all_updated_cols(PlannerInfo *root, RelOptInfo *rel)
{
Index relid;
RangeTblEntry *rte;
RTEPermissionInfo *perminfo;
Bitmapset *updatedCols,
*extraUpdatedCols;
Assert(root->parse->commandType == CMD_UPDATE);
Assert(IS_SIMPLE_REL(rel));
/*
Fix calculation of which GENERATED columns need to be updated. We were identifying the updatable generated columns of inheritance children by transposing the calculation made for their parent. However, there's nothing that says a traditional-inheritance child can't have generated columns that aren't there in its parent, or that have different dependencies than are in the parent's expression. (At present it seems that we don't enforce that for partitioning either, which is likely wrong to some degree or other; but the case clearly needs to be handled with traditional inheritance.) Hence, drop the very-klugy-anyway "extraUpdatedCols" RTE field in favor of identifying which generated columns depend on updated columns during executor startup. In HEAD we can remove extraUpdatedCols altogether; in back branches, it's still there but always empty. Another difference between the HEAD and back-branch versions of this patch is that in HEAD we can add the new bitmap field to ResultRelInfo, but that would cause an ABI break in back branches. Like 4b3e37993, add a List field at the end of struct EState instead. Back-patch to v13. The bogus calculation is also being made in v12, but it doesn't have the same visible effect because we don't use it to decide which generated columns to recalculate; as a consequence of which the patch doesn't apply easily. I think that there might still be a demonstrable bug associated with trigger firing conditions, but that's such a weird corner-case usage that I'm content to leave it unfixed in v12. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqFshLKNvQUd1DgwJ-7tsTp=dwv7KZqXC4j2wYBV1aCDUA@mail.gmail.com Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-05 20:12:17 +01:00
* We obtain updatedCols for the query's result relation. Then, if
* necessary, we map it to the column numbers of the relation for which
* they were requested.
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
*/
relid = root->parse->resultRelation;
rte = planner_rt_fetch(relid, root);
perminfo = getRTEPermissionInfo(root->parse->rteperminfos, rte);
updatedCols = perminfo->updatedCols;
if (rel->relid != relid)
{
RelOptInfo *top_parent_rel = find_base_rel(root, relid);
Assert(IS_OTHER_REL(rel));
updatedCols = translate_col_privs_multilevel(root, rel, top_parent_rel,
updatedCols);
}
Fix calculation of which GENERATED columns need to be updated. We were identifying the updatable generated columns of inheritance children by transposing the calculation made for their parent. However, there's nothing that says a traditional-inheritance child can't have generated columns that aren't there in its parent, or that have different dependencies than are in the parent's expression. (At present it seems that we don't enforce that for partitioning either, which is likely wrong to some degree or other; but the case clearly needs to be handled with traditional inheritance.) Hence, drop the very-klugy-anyway "extraUpdatedCols" RTE field in favor of identifying which generated columns depend on updated columns during executor startup. In HEAD we can remove extraUpdatedCols altogether; in back branches, it's still there but always empty. Another difference between the HEAD and back-branch versions of this patch is that in HEAD we can add the new bitmap field to ResultRelInfo, but that would cause an ABI break in back branches. Like 4b3e37993, add a List field at the end of struct EState instead. Back-patch to v13. The bogus calculation is also being made in v12, but it doesn't have the same visible effect because we don't use it to decide which generated columns to recalculate; as a consequence of which the patch doesn't apply easily. I think that there might still be a demonstrable bug associated with trigger firing conditions, but that's such a weird corner-case usage that I'm content to leave it unfixed in v12. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqFshLKNvQUd1DgwJ-7tsTp=dwv7KZqXC4j2wYBV1aCDUA@mail.gmail.com Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-05 20:12:17 +01:00
/*
* Now we must check to see if there are any generated columns that depend
* on the updatedCols, and add them to the result.
*/
extraUpdatedCols = get_dependent_generated_columns(root, rel->relid,
updatedCols);
Rework query relation permission checking Currently, information about the permissions to be checked on relations mentioned in a query is stored in their range table entries. So the executor must scan the entire range table looking for relations that need to have permissions checked. This can make the permission checking part of the executor initialization needlessly expensive when many inheritance children are present in the range range. While the permissions need not be checked on the individual child relations, the executor still must visit every range table entry to filter them out. This commit moves the permission checking information out of the range table entries into a new plan node called RTEPermissionInfo. Every top-level (inheritance "root") RTE_RELATION entry in the range table gets one and a list of those is maintained alongside the range table. This new list is initialized by the parser when initializing the range table. The rewriter can add more entries to it as rules/views are expanded. Finally, the planner combines the lists of the individual subqueries into one flat list that is passed to the executor for checking. To make it quick to find the RTEPermissionInfo entry belonging to a given relation, RangeTblEntry gets a new Index field 'perminfoindex' that stores the corresponding RTEPermissionInfo's index in the query's list of the latter. ExecutorCheckPerms_hook has gained another List * argument; the signature is now: typedef bool (*ExecutorCheckPerms_hook_type) (List *rangeTable, List *rtePermInfos, bool ereport_on_violation); The first argument is no longer used by any in-core uses of the hook, but we leave it in place because there may be other implementations that do. Implementations should likely scan the rtePermInfos list to determine which operations to allow or deny. Author: Amit Langote <amitlangote09@gmail.com> Discussion: https://postgr.es/m/CA+HiwqGjJDmUhDSfv-U2qhKJjt9ST7Xh9JXC_irsAQ1TAUsJYg@mail.gmail.com
2022-12-06 16:09:24 +01:00
return bms_union(updatedCols, extraUpdatedCols);
}
/*
* translate_col_privs
* Translate a bitmapset representing per-column privileges from the
* parent rel's attribute numbering to the child's.
*
* The only surprise here is that we don't translate a parent whole-row
* reference into a child whole-row reference. That would mean requiring
* permissions on all child columns, which is overly strict, since the
* query is really only going to reference the inherited columns. Instead
* we set the per-column bits for all inherited columns.
*/
static Bitmapset *
translate_col_privs(const Bitmapset *parent_privs,
List *translated_vars)
{
Bitmapset *child_privs = NULL;
bool whole_row;
int attno;
ListCell *lc;
/* System attributes have the same numbers in all tables */
for (attno = FirstLowInvalidHeapAttributeNumber + 1; attno < 0; attno++)
{
if (bms_is_member(attno - FirstLowInvalidHeapAttributeNumber,
parent_privs))
child_privs = bms_add_member(child_privs,
attno - FirstLowInvalidHeapAttributeNumber);
}
/* Check if parent has whole-row reference */
whole_row = bms_is_member(InvalidAttrNumber - FirstLowInvalidHeapAttributeNumber,
parent_privs);
/* And now translate the regular user attributes, using the vars list */
attno = InvalidAttrNumber;
foreach(lc, translated_vars)
{
Var *var = lfirst_node(Var, lc);
attno++;
if (var == NULL) /* ignore dropped columns */
continue;
if (whole_row ||
bms_is_member(attno - FirstLowInvalidHeapAttributeNumber,
parent_privs))
child_privs = bms_add_member(child_privs,
var->varattno - FirstLowInvalidHeapAttributeNumber);
}
return child_privs;
}
Fix calculation of which GENERATED columns need to be updated. We were identifying the updatable generated columns of inheritance children by transposing the calculation made for their parent. However, there's nothing that says a traditional-inheritance child can't have generated columns that aren't there in its parent, or that have different dependencies than are in the parent's expression. (At present it seems that we don't enforce that for partitioning either, which is likely wrong to some degree or other; but the case clearly needs to be handled with traditional inheritance.) Hence, drop the very-klugy-anyway "extraUpdatedCols" RTE field in favor of identifying which generated columns depend on updated columns during executor startup. In HEAD we can remove extraUpdatedCols altogether; in back branches, it's still there but always empty. Another difference between the HEAD and back-branch versions of this patch is that in HEAD we can add the new bitmap field to ResultRelInfo, but that would cause an ABI break in back branches. Like 4b3e37993, add a List field at the end of struct EState instead. Back-patch to v13. The bogus calculation is also being made in v12, but it doesn't have the same visible effect because we don't use it to decide which generated columns to recalculate; as a consequence of which the patch doesn't apply easily. I think that there might still be a demonstrable bug associated with trigger firing conditions, but that's such a weird corner-case usage that I'm content to leave it unfixed in v12. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqFshLKNvQUd1DgwJ-7tsTp=dwv7KZqXC4j2wYBV1aCDUA@mail.gmail.com Discussion: https://postgr.es/m/2793383.1672944799@sss.pgh.pa.us
2023-01-05 20:12:17 +01:00
/*
* translate_col_privs_multilevel
* Recursively translates the column numbers contained in 'parent_cols'
* to the column numbers of a descendant relation given by 'rel'
*
* Note that because this is based on translate_col_privs, it will expand
* a whole-row reference into all inherited columns. This is not an issue
* for current usages, but beware.
*/
static Bitmapset *
translate_col_privs_multilevel(PlannerInfo *root, RelOptInfo *rel,
RelOptInfo *parent_rel,
Bitmapset *parent_cols)
{
AppendRelInfo *appinfo;
/* Fast path for easy case. */
if (parent_cols == NULL)
return NULL;
/* Recurse if immediate parent is not the top parent. */
if (rel->parent != parent_rel)
{
if (rel->parent)
parent_cols = translate_col_privs_multilevel(root, rel->parent,
parent_rel,
parent_cols);
else
elog(ERROR, "rel with relid %u is not a child rel", rel->relid);
}
/* Now translate for this child. */
Assert(root->append_rel_array != NULL);
appinfo = root->append_rel_array[rel->relid];
Assert(appinfo != NULL);
return translate_col_privs(parent_cols, appinfo->translated_vars);
}
/*
* expand_appendrel_subquery
* Add "other rel" RelOptInfos for the children of an appendrel baserel
*
* "rel" is a subquery relation that has the rte->inh flag set, meaning it
* is a UNION ALL subquery that's been flattened into an appendrel, with
* child subqueries listed in root->append_rel_list. We need to build
* a RelOptInfo for each child relation so that we can plan scans on them.
*/
static void
expand_appendrel_subquery(PlannerInfo *root, RelOptInfo *rel,
RangeTblEntry *rte, Index rti)
{
ListCell *l;
foreach(l, root->append_rel_list)
{
AppendRelInfo *appinfo = (AppendRelInfo *) lfirst(l);
Index childRTindex = appinfo->child_relid;
RangeTblEntry *childrte;
RelOptInfo *childrel;
/* append_rel_list contains all append rels; ignore others */
if (appinfo->parent_relid != rti)
continue;
/* find the child RTE, which should already exist */
Assert(childRTindex < root->simple_rel_array_size);
childrte = root->simple_rte_array[childRTindex];
Assert(childrte != NULL);
/* Build the child RelOptInfo. */
childrel = build_simple_rel(root, childRTindex, rel);
/* Child may itself be an inherited rel, either table or subquery. */
if (childrte->inh)
expand_inherited_rtentry(root, childrel, childrte, childRTindex);
}
}
/*
* apply_child_basequals
* Populate childrel's base restriction quals from parent rel's quals,
* translating them using appinfo.
*
* If any of the resulting clauses evaluate to constant false or NULL, we
* return false and don't apply any quals. Caller should mark the relation as
* a dummy rel in this case, since it doesn't need to be scanned.
*/
bool
apply_child_basequals(PlannerInfo *root, RelOptInfo *parentrel,
RelOptInfo *childrel, RangeTblEntry *childRTE,
AppendRelInfo *appinfo)
{
List *childquals;
Index cq_min_security;
ListCell *lc;
/*
* The child rel's targetlist might contain non-Var expressions, which
* means that substitution into the quals could produce opportunities for
* const-simplification, and perhaps even pseudoconstant quals. Therefore,
* transform each RestrictInfo separately to see if it reduces to a
* constant or pseudoconstant. (We must process them separately to keep
* track of the security level of each qual.)
*/
childquals = NIL;
cq_min_security = UINT_MAX;
foreach(lc, parentrel->baserestrictinfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Node *childqual;
ListCell *lc2;
Assert(IsA(rinfo, RestrictInfo));
childqual = adjust_appendrel_attrs(root,
(Node *) rinfo->clause,
1, &appinfo);
childqual = eval_const_expressions(root, childqual);
/* check for flat-out constant */
if (childqual && IsA(childqual, Const))
{
if (((Const *) childqual)->constisnull ||
!DatumGetBool(((Const *) childqual)->constvalue))
{
/* Restriction reduces to constant FALSE or NULL */
return false;
}
/* Restriction reduces to constant TRUE, so drop it */
continue;
}
/* might have gotten an AND clause, if so flatten it */
foreach(lc2, make_ands_implicit((Expr *) childqual))
{
Node *onecq = (Node *) lfirst(lc2);
bool pseudoconstant;
/* check for pseudoconstant (no Vars or volatile functions) */
pseudoconstant =
!contain_vars_of_level(onecq, 0) &&
!contain_volatile_functions(onecq);
if (pseudoconstant)
{
/* tell createplan.c to check for gating quals */
root->hasPseudoConstantQuals = true;
}
/* reconstitute RestrictInfo with appropriate properties */
childquals = lappend(childquals,
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar. Previously, pull_varnos() took the relids of a PlaceHolderVar as being equal to the relids in its contents, but that fails to account for the possibility that we have to postpone evaluation of the PHV due to outer joins. This could result in a malformed plan. The known cases end up triggering the "failed to assign all NestLoopParams to plan nodes" sanity check in createplan.c, but other symptoms may be possible. The right value to use is the join level we actually intend to evaluate the PHV at. We can get that from the ph_eval_at field of the associated PlaceHolderInfo. However, there are some places that call pull_varnos() before the PlaceHolderInfos have been created; in that case, fall back to the conservative assumption that the PHV will be evaluated at its syntactic level. (In principle this might result in missing some legal optimization, but I'm not aware of any cases where it's an issue in practice.) Things are also a bit ticklish for calls occurring during deconstruct_jointree(), but AFAICS the ph_eval_at fields should have reached their final values by the time we need them. The main problem in making this work is that pull_varnos() has no way to get at the PlaceHolderInfos. We can fix that easily, if a bit tediously, in HEAD by passing it the planner "root" pointer. In the back branches that'd cause an unacceptable API/ABI break for extensions, so leave the existing entry points alone and add new ones with the additional parameter. (If an old entry point is called and encounters a PHV, it'll fall back to using the syntactic level, again possibly missing some valid optimization.) Back-patch to v12. The computation is surely also wrong before that, but it appears that we cannot reach a bad plan thanks to join order restrictions imposed on the subquery that the PlaceHolderVar came from. The error only became reachable when commit 4be058fe9 allowed trivial subqueries to be collapsed out completely, eliminating their join order restrictions. Per report from Stephan Springl. Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
make_restrictinfo(root,
(Expr *) onecq,
rinfo->is_pushed_down,
rinfo->has_clone,
rinfo->is_clone,
pseudoconstant,
rinfo->security_level,
NULL, NULL, NULL));
/* track minimum security level among child quals */
cq_min_security = Min(cq_min_security, rinfo->security_level);
}
}
/*
* In addition to the quals inherited from the parent, we might have
* securityQuals associated with this particular child node. (Currently
* this can only happen in appendrels originating from UNION ALL;
* inheritance child tables don't have their own securityQuals, see
* expand_single_inheritance_child().) Pull any such securityQuals up
* into the baserestrictinfo for the child. This is similar to
* process_security_barrier_quals() for the parent rel, except that we
* can't make any general deductions from such quals, since they don't
* hold for the whole appendrel.
*/
if (childRTE->securityQuals)
{
Index security_level = 0;
foreach(lc, childRTE->securityQuals)
{
List *qualset = (List *) lfirst(lc);
ListCell *lc2;
foreach(lc2, qualset)
{
Expr *qual = (Expr *) lfirst(lc2);
/* not likely that we'd see constants here, so no check */
childquals = lappend(childquals,
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar. Previously, pull_varnos() took the relids of a PlaceHolderVar as being equal to the relids in its contents, but that fails to account for the possibility that we have to postpone evaluation of the PHV due to outer joins. This could result in a malformed plan. The known cases end up triggering the "failed to assign all NestLoopParams to plan nodes" sanity check in createplan.c, but other symptoms may be possible. The right value to use is the join level we actually intend to evaluate the PHV at. We can get that from the ph_eval_at field of the associated PlaceHolderInfo. However, there are some places that call pull_varnos() before the PlaceHolderInfos have been created; in that case, fall back to the conservative assumption that the PHV will be evaluated at its syntactic level. (In principle this might result in missing some legal optimization, but I'm not aware of any cases where it's an issue in practice.) Things are also a bit ticklish for calls occurring during deconstruct_jointree(), but AFAICS the ph_eval_at fields should have reached their final values by the time we need them. The main problem in making this work is that pull_varnos() has no way to get at the PlaceHolderInfos. We can fix that easily, if a bit tediously, in HEAD by passing it the planner "root" pointer. In the back branches that'd cause an unacceptable API/ABI break for extensions, so leave the existing entry points alone and add new ones with the additional parameter. (If an old entry point is called and encounters a PHV, it'll fall back to using the syntactic level, again possibly missing some valid optimization.) Back-patch to v12. The computation is surely also wrong before that, but it appears that we cannot reach a bad plan thanks to join order restrictions imposed on the subquery that the PlaceHolderVar came from. The error only became reachable when commit 4be058fe9 allowed trivial subqueries to be collapsed out completely, eliminating their join order restrictions. Per report from Stephan Springl. Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
make_restrictinfo(root, qual,
true,
false, false,
false,
security_level,
NULL, NULL, NULL));
cq_min_security = Min(cq_min_security, security_level);
}
security_level++;
}
Assert(security_level <= root->qual_security_level);
}
/*
* OK, we've got all the baserestrictinfo quals for this child.
*/
childrel->baserestrictinfo = childquals;
childrel->baserestrict_min_security = cq_min_security;
return true;
}