Existing partition pruning is only able to work at plan time, for query quals that appear in the parsed query. This is good but limiting, as there can be parameters that appear later that can be usefully used to further prune partitions. This commit adds support for pruning subnodes of Append which cannot possibly contain any matching tuples, during execution, by evaluating Params to determine the minimum set of subnodes that can possibly match. We support more than just simple Params in WHERE clauses. Support additionally includes: 1. Parameterized Nested Loop Joins: The parameter from the outer side of the join can be used to determine the minimum set of inner side partitions to scan. 2. Initplans: Once an initplan has been executed we can then determine which partitions match the value from the initplan. Partition pruning is performed in two ways. When Params external to the plan are found to match the partition key we attempt to prune away unneeded Append subplans during the initialization of the executor. This allows us to bypass the initialization of non-matching subplans meaning they won't appear in the EXPLAIN or EXPLAIN ANALYZE output. For parameters whose value is only known during the actual execution then the pruning of these subplans must wait. Subplans which are eliminated during this stage of pruning are still visible in the EXPLAIN output. In order to determine if pruning has actually taken place, the EXPLAIN ANALYZE must be viewed. If a certain Append subplan was never executed due to the elimination of the partition then the execution timing area will state "(never executed)". Whereas, if, for example in the case of parameterized nested loops, the number of loops stated in the EXPLAIN ANALYZE output for certain subplans may appear lower than others due to the subplan having been scanned fewer times. This is due to the list of matching subnodes having to be evaluated whenever a parameter which was found to match the partition key changes. This commit required some additional infrastructure that permits the building of a data structure which is able to perform the translation of the matching partition IDs, as returned by get_matching_partitions, into the list index of a subpaths list, as exist in node types such as Append, MergeAppend and ModifyTable. This allows us to translate a list of clauses into a Bitmapset of all the subpath indexes which must be included to satisfy the clause list. Author: David Rowley, based on an earlier effort by Beena Emerson Reviewers: Amit Langote, Robert Haas, Amul Sul, Rajkumar Raghuwanshi, Jesper Pedersen Discussion: https://postgr.es/m/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A@mail.gmail.com |
||
---|---|---|
.. | ||
Makefile | ||
README | ||
bitmapset.c | ||
copyfuncs.c | ||
equalfuncs.c | ||
extensible.c | ||
list.c | ||
makefuncs.c | ||
nodeFuncs.c | ||
nodes.c | ||
outfuncs.c | ||
params.c | ||
print.c | ||
read.c | ||
readfuncs.c | ||
tidbitmap.c | ||
value.c |
README
src/backend/nodes/README Node Structures =============== Andrew Yu (11/94) Introduction ------------ The current node structures are plain old C structures. "Inheritance" is achieved by convention. No additional functions will be generated. Functions that manipulate node structures reside in this directory. FILES IN THIS DIRECTORY (src/backend/nodes/) General-purpose node manipulation functions: copyfuncs.c - copy a node tree equalfuncs.c - compare two node trees outfuncs.c - convert a node tree to text representation readfuncs.c - convert text representation back to a node tree makefuncs.c - creator functions for some common node types nodeFuncs.c - some other general-purpose manipulation functions Specialized manipulation functions: bitmapset.c - Bitmapset support list.c - generic list support params.c - Param support tidbitmap.c - TIDBitmap support value.c - support for Value nodes FILES IN src/include/nodes/ Node definitions: nodes.h - define node tags (NodeTag) primnodes.h - primitive nodes parsenodes.h - parse tree nodes plannodes.h - plan tree nodes relation.h - planner internal nodes execnodes.h - executor nodes memnodes.h - memory nodes pg_list.h - generic list Steps to Add a Node ------------------- Suppose you want to define a node Foo: 1. Add a tag (T_Foo) to the enum NodeTag in nodes.h. (If you insert the tag in a way that moves the numbers associated with existing tags, you'll need to recompile the whole tree after doing this. It doesn't force initdb though, because the numbers never go to disk.) 2. Add the structure definition to the appropriate include/nodes/???.h file. If you intend to inherit from, say a Plan node, put Plan as the first field of your struct definition. 3. If you intend to use copyObject, equal, nodeToString or stringToNode, add an appropriate function to copyfuncs.c, equalfuncs.c, outfuncs.c and readfuncs.c accordingly. (Except for frequently used nodes, don't bother writing a creator function in makefuncs.c) The header comments in those files give general rules for whether you need to add support. 4. Add cases to the functions in nodeFuncs.c as needed. There are many other places you'll probably also need to teach about your new node type. Best bet is to grep for references to one or two similar existing node types to find all the places to touch. Historical Note --------------- Prior to the current simple C structure definitions, the Node structures used a pseudo-inheritance system which automatically generated creator and accessor functions. Since every node inherited from LispValue, the whole thing was a mess. Here's a little anecdote: LispValue definition -- class used to support lisp structures in C. This is here because we did not want to totally rewrite planner and executor code which depended on lisp structures when we ported postgres V1 from lisp to C. -cim 4/23/90