postgresql/src/backend/nodes
Alvaro Herrera 499be013de Support partition pruning at execution time
Existing partition pruning is only able to work at plan time, for query
quals that appear in the parsed query.  This is good but limiting, as
there can be parameters that appear later that can be usefully used to
further prune partitions.

This commit adds support for pruning subnodes of Append which cannot
possibly contain any matching tuples, during execution, by evaluating
Params to determine the minimum set of subnodes that can possibly match.
We support more than just simple Params in WHERE clauses. Support
additionally includes:

1. Parameterized Nested Loop Joins: The parameter from the outer side of the
   join can be used to determine the minimum set of inner side partitions to
   scan.

2. Initplans: Once an initplan has been executed we can then determine which
   partitions match the value from the initplan.

Partition pruning is performed in two ways.  When Params external to the plan
are found to match the partition key we attempt to prune away unneeded Append
subplans during the initialization of the executor.  This allows us to bypass
the initialization of non-matching subplans meaning they won't appear in the
EXPLAIN or EXPLAIN ANALYZE output.

For parameters whose value is only known during the actual execution
then the pruning of these subplans must wait.  Subplans which are
eliminated during this stage of pruning are still visible in the EXPLAIN
output.  In order to determine if pruning has actually taken place, the
EXPLAIN ANALYZE must be viewed.  If a certain Append subplan was never
executed due to the elimination of the partition then the execution
timing area will state "(never executed)".  Whereas, if, for example in
the case of parameterized nested loops, the number of loops stated in
the EXPLAIN ANALYZE output for certain subplans may appear lower than
others due to the subplan having been scanned fewer times.  This is due
to the list of matching subnodes having to be evaluated whenever a
parameter which was found to match the partition key changes.

This commit required some additional infrastructure that permits the
building of a data structure which is able to perform the translation of
the matching partition IDs, as returned by get_matching_partitions, into
the list index of a subpaths list, as exist in node types such as
Append, MergeAppend and ModifyTable.  This allows us to translate a list
of clauses into a Bitmapset of all the subpath indexes which must be
included to satisfy the clause list.

Author: David Rowley, based on an earlier effort by Beena Emerson
Reviewers: Amit Langote, Robert Haas, Amul Sul, Rajkumar Raghuwanshi,
Jesper Pedersen
Discussion: https://postgr.es/m/CAOG9ApE16ac-_VVZVvv0gePSgkg_BwYEV1NBqZFqDR2bBE0X0A@mail.gmail.com
2018-04-07 17:54:39 -03:00
..
Makefile Introduce extensible node types. 2016-02-12 09:38:11 -05:00
README Fix various common mispellings. 2016-06-03 16:08:45 +01:00
bitmapset.c Add bms_prev_member function 2018-04-07 17:54:39 -03:00
copyfuncs.c Support partition pruning at execution time 2018-04-07 17:54:39 -03:00
equalfuncs.c Indexes with INCLUDE columns and their support in B-tree 2018-04-07 23:00:39 +03:00
extensible.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
list.c Rewrite list_qsort() to avoid trashing its input list. 2018-01-09 13:25:53 -05:00
makefuncs.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
nodeFuncs.c Support partition pruning at execution time 2018-04-07 17:54:39 -03:00
nodes.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
outfuncs.c Support partition pruning at execution time 2018-04-07 17:54:39 -03:00
params.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
print.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
read.c Move strtoint() to common 2018-03-13 10:21:09 -04:00
readfuncs.c Support partition pruning at execution time 2018-04-07 17:54:39 -03:00
tidbitmap.c Update copyright for 2018 2018-01-02 23:30:12 -05:00
value.c Change internal integer representation of Value node 2018-03-13 09:56:25 -04:00

README

src/backend/nodes/README

Node Structures
===============

Andrew Yu (11/94)

Introduction
------------

The current node structures are plain old C structures. "Inheritance" is
achieved by convention. No additional functions will be generated. Functions
that manipulate node structures reside in this directory.


FILES IN THIS DIRECTORY (src/backend/nodes/)

    General-purpose node manipulation functions:
	copyfuncs.c	- copy a node tree
	equalfuncs.c	- compare two node trees
	outfuncs.c	- convert a node tree to text representation
	readfuncs.c	- convert text representation back to a node tree
	makefuncs.c	- creator functions for some common node types
	nodeFuncs.c	- some other general-purpose manipulation functions

    Specialized manipulation functions:
	bitmapset.c	- Bitmapset support
	list.c		- generic list support
	params.c	- Param support
	tidbitmap.c	- TIDBitmap support
	value.c		- support for Value nodes

FILES IN src/include/nodes/

    Node definitions:
	nodes.h		- define node tags (NodeTag)
	primnodes.h	- primitive nodes
	parsenodes.h	- parse tree nodes
	plannodes.h	- plan tree nodes
	relation.h	- planner internal nodes
	execnodes.h	- executor nodes
	memnodes.h	- memory nodes
	pg_list.h	- generic list


Steps to Add a Node
-------------------

Suppose you want to define a node Foo:

1. Add a tag (T_Foo) to the enum NodeTag in nodes.h.  (If you insert the
   tag in a way that moves the numbers associated with existing tags,
   you'll need to recompile the whole tree after doing this.  It doesn't
   force initdb though, because the numbers never go to disk.)
2. Add the structure definition to the appropriate include/nodes/???.h file.
   If you intend to inherit from, say a Plan node, put Plan as the first field
   of your struct definition.
3. If you intend to use copyObject, equal, nodeToString or stringToNode,
   add an appropriate function to copyfuncs.c, equalfuncs.c, outfuncs.c
   and readfuncs.c accordingly.  (Except for frequently used nodes, don't
   bother writing a creator function in makefuncs.c)  The header comments
   in those files give general rules for whether you need to add support.
4. Add cases to the functions in nodeFuncs.c as needed.  There are many
   other places you'll probably also need to teach about your new node
   type.  Best bet is to grep for references to one or two similar existing
   node types to find all the places to touch.


Historical Note
---------------

Prior to the current simple C structure definitions, the Node structures
used a pseudo-inheritance system which automatically generated creator and
accessor functions. Since every node inherited from LispValue, the whole thing
was a mess. Here's a little anecdote:

    LispValue definition -- class used to support lisp structures
    in C.  This is here because we did not want to totally rewrite
    planner and executor code which depended on lisp structures when
    we ported postgres V1 from lisp to C. -cim 4/23/90