Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
*
|
|
|
|
* orclauses.c
|
|
|
|
* Routines to extract restriction OR clauses from join OR clauses
|
|
|
|
*
|
2023-01-02 21:00:37 +01:00
|
|
|
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
|
|
|
*
|
|
|
|
*
|
|
|
|
* IDENTIFICATION
|
|
|
|
* src/backend/optimizer/util/orclauses.c
|
|
|
|
*
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
*/
|
|
|
|
|
|
|
|
#include "postgres.h"
|
|
|
|
|
2019-01-29 21:26:44 +01:00
|
|
|
#include "nodes/makefuncs.h"
|
|
|
|
#include "nodes/nodeFuncs.h"
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
#include "optimizer/clauses.h"
|
|
|
|
#include "optimizer/cost.h"
|
2019-01-29 21:48:51 +01:00
|
|
|
#include "optimizer/optimizer.h"
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
#include "optimizer/orclauses.h"
|
|
|
|
#include "optimizer/restrictinfo.h"
|
|
|
|
|
|
|
|
|
|
|
|
static bool is_safe_restriction_clause_for(RestrictInfo *rinfo, RelOptInfo *rel);
|
|
|
|
static Expr *extract_or_clause(RestrictInfo *or_rinfo, RelOptInfo *rel);
|
|
|
|
static void consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
Expr *orclause, RestrictInfo *join_or_rinfo);
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
* extract_restriction_or_clauses
|
|
|
|
* Examine join OR-of-AND clauses to see if any useful restriction OR
|
|
|
|
* clauses can be extracted. If so, add them to the query.
|
|
|
|
*
|
|
|
|
* Although a join clause must reference multiple relations overall,
|
|
|
|
* an OR of ANDs clause might contain sub-clauses that reference just one
|
|
|
|
* relation and can be used to build a restriction clause for that rel.
|
|
|
|
* For example consider
|
|
|
|
* WHERE ((a.x = 42 AND b.y = 43) OR (a.x = 44 AND b.z = 45));
|
|
|
|
* We can transform this into
|
|
|
|
* WHERE ((a.x = 42 AND b.y = 43) OR (a.x = 44 AND b.z = 45))
|
|
|
|
* AND (a.x = 42 OR a.x = 44)
|
|
|
|
* AND (b.y = 43 OR b.z = 45);
|
|
|
|
* which allows the latter clauses to be applied during the scans of a and b,
|
|
|
|
* perhaps as index qualifications, and in any case reducing the number of
|
|
|
|
* rows arriving at the join. In essence this is a partial transformation to
|
|
|
|
* CNF (AND of ORs format). It is not complete, however, because we do not
|
|
|
|
* unravel the original OR --- doing so would usually bloat the qualification
|
|
|
|
* expression to little gain.
|
|
|
|
*
|
|
|
|
* The added quals are partially redundant with the original OR, and therefore
|
|
|
|
* would cause the size of the joinrel to be underestimated when it is finally
|
|
|
|
* formed. (This would be true of a full transformation to CNF as well; the
|
|
|
|
* fault is not really in the transformation, but in clauselist_selectivity's
|
|
|
|
* inability to recognize redundant conditions.) We can compensate for this
|
|
|
|
* redundancy by changing the cached selectivity of the original OR clause,
|
2016-07-15 04:48:26 +02:00
|
|
|
* canceling out the (valid) reduction in the estimated sizes of the base
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
* relations so that the estimated joinrel size remains the same. This is
|
|
|
|
* a MAJOR HACK: it depends on the fact that clause selectivities are cached
|
|
|
|
* and on the fact that the same RestrictInfo node will appear in every
|
|
|
|
* joininfo list that might be used when the joinrel is formed.
|
|
|
|
* And it doesn't work in cases where the size estimation is nonlinear
|
|
|
|
* (i.e., outer and IN joins). But it beats not doing anything.
|
|
|
|
*
|
|
|
|
* We examine each base relation to see if join clauses associated with it
|
|
|
|
* contain extractable restriction conditions. If so, add those conditions
|
|
|
|
* to the rel's baserestrictinfo and update the cached selectivities of the
|
|
|
|
* join clauses. Note that the same join clause will be examined afresh
|
|
|
|
* from the point of view of each baserel that participates in it, so its
|
|
|
|
* cached selectivity may get updated multiple times.
|
|
|
|
*/
|
|
|
|
void
|
|
|
|
extract_restriction_or_clauses(PlannerInfo *root)
|
|
|
|
{
|
|
|
|
Index rti;
|
|
|
|
|
|
|
|
/* Examine each baserel for potential join OR clauses */
|
|
|
|
for (rti = 1; rti < root->simple_rel_array_size; rti++)
|
|
|
|
{
|
|
|
|
RelOptInfo *rel = root->simple_rel_array[rti];
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
/* there may be empty slots corresponding to non-baserel RTEs */
|
|
|
|
if (rel == NULL)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
Assert(rel->relid == rti); /* sanity check on array */
|
|
|
|
|
|
|
|
/* ignore RTEs that are "other rels" */
|
|
|
|
if (rel->reloptkind != RELOPT_BASEREL)
|
|
|
|
continue;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Find potentially interesting OR joinclauses. We can use any
|
|
|
|
* joinclause that is considered safe to move to this rel by the
|
|
|
|
* parameterized-path machinery, even though what we are going to do
|
|
|
|
* with it is not exactly a parameterized path.
|
|
|
|
*/
|
|
|
|
foreach(lc, rel->joininfo)
|
|
|
|
{
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
|
|
|
if (restriction_is_or_clause(rinfo) &&
|
Do assorted mop-up in the planner.
Remove RestrictInfo.nullable_relids, along with a good deal of
infrastructure that calculated it. One use-case for it was in
join_clause_is_movable_to, but we can now replace that usage with
a check to see if the clause's relids include any outer join
that can null the target relation. The other use-case was in
join_clause_is_movable_into, but that test can just be dropped
entirely now that the clause's relids include outer joins.
Furthermore, join_clause_is_movable_into should now be
accurate enough that it will accept anything returned by
generate_join_implied_equalities, so we can restore the Assert
that was diked out in commit 95f4e59c3.
Remove the outerjoin_delayed mechanism. We needed this before to
prevent quals from getting evaluated below outer joins that should
null some of their vars. Now that we consider varnullingrels while
placing quals, that's taken care of automatically, so throw the
whole thing away.
Teach remove_useless_result_rtes to also remove useless FromExprs.
Having done that, the delay_upper_joins flag serves no purpose any
more and we can remove it, largely reverting 11086f2f2.
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. But we must throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end. This improves the generated plans for
such cases by not having to test a redundant join clause. We can also
get rid of the ugly hack used to mark such clauses as redundant for
selectivity estimation.
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:44:36 +01:00
|
|
|
join_clause_is_movable_to(rinfo, rel))
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
{
|
|
|
|
/* Try to extract a qual for this rel only */
|
|
|
|
Expr *orclause = extract_or_clause(rinfo, rel);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If successful, decide whether we want to use the clause,
|
|
|
|
* and insert it into the rel's restrictinfo list if so.
|
|
|
|
*/
|
|
|
|
if (orclause)
|
|
|
|
consider_new_or_clause(root, rel, orclause, rinfo);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Is the given primitive (non-OR) RestrictInfo safe to move to the rel?
|
|
|
|
*/
|
|
|
|
static bool
|
|
|
|
is_safe_restriction_clause_for(RestrictInfo *rinfo, RelOptInfo *rel)
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* We want clauses that mention the rel, and only the rel. So in
|
|
|
|
* particular pseudoconstant clauses can be rejected quickly. Then check
|
|
|
|
* the clause's Var membership.
|
|
|
|
*/
|
|
|
|
if (rinfo->pseudoconstant)
|
|
|
|
return false;
|
|
|
|
if (!bms_equal(rinfo->clause_relids, rel->relids))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
/* We don't want extra evaluations of any volatile functions */
|
|
|
|
if (contain_volatile_functions((Node *) rinfo->clause))
|
|
|
|
return false;
|
|
|
|
|
|
|
|
return true;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Try to extract a restriction clause mentioning only "rel" from the given
|
|
|
|
* join OR-clause.
|
|
|
|
*
|
|
|
|
* We must be able to extract at least one qual for this rel from each of
|
|
|
|
* the arms of the OR, else we can't use it.
|
|
|
|
*
|
|
|
|
* Returns an OR clause (not a RestrictInfo!) pertaining to rel, or NULL
|
|
|
|
* if no OR clause could be extracted.
|
|
|
|
*/
|
|
|
|
static Expr *
|
|
|
|
extract_or_clause(RestrictInfo *or_rinfo, RelOptInfo *rel)
|
|
|
|
{
|
|
|
|
List *clauselist = NIL;
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Scan each arm of the input OR clause. Notice we descend into
|
|
|
|
* or_rinfo->orclause, which has RestrictInfo nodes embedded below the
|
|
|
|
* toplevel OR/AND structure. This is useful because we can use the info
|
|
|
|
* in those nodes to make is_safe_restriction_clause_for()'s checks
|
|
|
|
* cheaper. We'll strip those nodes from the returned tree, though,
|
|
|
|
* meaning that fresh ones will be built if the clause is accepted as a
|
|
|
|
* restriction clause. This might seem wasteful --- couldn't we re-use
|
|
|
|
* the existing RestrictInfos? But that'd require assuming that
|
|
|
|
* selectivity and other cached data is computed exactly the same way for
|
|
|
|
* a restriction clause as for a join clause, which seems undesirable.
|
|
|
|
*/
|
2019-01-29 21:26:44 +01:00
|
|
|
Assert(is_orclause(or_rinfo->orclause));
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
foreach(lc, ((BoolExpr *) or_rinfo->orclause)->args)
|
|
|
|
{
|
|
|
|
Node *orarg = (Node *) lfirst(lc);
|
|
|
|
List *subclauses = NIL;
|
2014-09-10 00:35:14 +02:00
|
|
|
Node *subclause;
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
/* OR arguments should be ANDs or sub-RestrictInfos */
|
2019-01-29 21:26:44 +01:00
|
|
|
if (is_andclause(orarg))
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
{
|
|
|
|
List *andargs = ((BoolExpr *) orarg)->args;
|
|
|
|
ListCell *lc2;
|
|
|
|
|
|
|
|
foreach(lc2, andargs)
|
|
|
|
{
|
Improve castNode notation by introducing list-extraction-specific variants.
This extends the castNode() notation introduced by commit 5bcab1114 to
provide, in one step, extraction of a list cell's pointer and coercion to
a concrete node type. For example, "lfirst_node(Foo, lc)" is the same
as "castNode(Foo, lfirst(lc))". Almost half of the uses of castNode
that have appeared so far include a list extraction call, so this is
pretty widely useful, and it saves a few more keystrokes compared to the
old way.
As with the previous patch, back-patch the addition of these macros to
pg_list.h, so that the notation will be available when back-patching.
Patch by me, after an idea of Andrew Gierth's.
Discussion: https://postgr.es/m/14197.1491841216@sss.pgh.pa.us
2017-04-10 19:51:29 +02:00
|
|
|
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc2);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
if (restriction_is_or_clause(rinfo))
|
|
|
|
{
|
|
|
|
/*
|
|
|
|
* Recurse to deal with nested OR. Note we *must* recurse
|
|
|
|
* here, this isn't just overly-tense optimization: we
|
|
|
|
* have to descend far enough to find and strip all
|
|
|
|
* RestrictInfos in the expression.
|
|
|
|
*/
|
|
|
|
Expr *suborclause;
|
|
|
|
|
|
|
|
suborclause = extract_or_clause(rinfo, rel);
|
|
|
|
if (suborclause)
|
|
|
|
subclauses = lappend(subclauses, suborclause);
|
|
|
|
}
|
|
|
|
else if (is_safe_restriction_clause_for(rinfo, rel))
|
|
|
|
subclauses = lappend(subclauses, rinfo->clause);
|
|
|
|
}
|
|
|
|
}
|
|
|
|
else
|
|
|
|
{
|
2017-02-21 17:33:07 +01:00
|
|
|
RestrictInfo *rinfo = castNode(RestrictInfo, orarg);
|
|
|
|
|
|
|
|
Assert(!restriction_is_or_clause(rinfo));
|
|
|
|
if (is_safe_restriction_clause_for(rinfo, rel))
|
|
|
|
subclauses = lappend(subclauses, rinfo->clause);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If nothing could be extracted from this arm, we can't do anything
|
|
|
|
* with this OR clause.
|
|
|
|
*/
|
|
|
|
if (subclauses == NIL)
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* OK, add subclause(s) to the result OR. If we found more than one,
|
2014-09-10 00:35:14 +02:00
|
|
|
* we need an AND node. But if we found only one, and it is itself an
|
|
|
|
* OR node, add its subclauses to the result instead; this is needed
|
|
|
|
* to preserve AND/OR flatness (ie, no OR directly underneath OR).
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
*/
|
2014-09-10 00:35:14 +02:00
|
|
|
subclause = (Node *) make_ands_explicit(subclauses);
|
2019-01-29 21:26:44 +01:00
|
|
|
if (is_orclause(subclause))
|
2014-09-10 00:35:14 +02:00
|
|
|
clauselist = list_concat(clauselist,
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
((BoolExpr *) subclause)->args);
|
2014-09-10 00:35:14 +02:00
|
|
|
else
|
|
|
|
clauselist = lappend(clauselist, subclause);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* If we got a restriction clause from every arm, wrap them up in an OR
|
|
|
|
* node. (In theory the OR node might be unnecessary, if there was only
|
|
|
|
* one arm --- but then the input OR node was also redundant.)
|
|
|
|
*/
|
|
|
|
if (clauselist != NIL)
|
|
|
|
return make_orclause(clauselist);
|
|
|
|
return NULL;
|
|
|
|
}
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Consider whether a successfully-extracted restriction OR clause is
|
|
|
|
* actually worth using. If so, add it to the planner's data structures,
|
|
|
|
* and adjust the original join clause (join_or_rinfo) to compensate.
|
|
|
|
*/
|
|
|
|
static void
|
|
|
|
consider_new_or_clause(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
Expr *orclause, RestrictInfo *join_or_rinfo)
|
|
|
|
{
|
|
|
|
RestrictInfo *or_rinfo;
|
|
|
|
Selectivity or_selec,
|
|
|
|
orig_selec;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Build a RestrictInfo from the new OR clause. We can assume it's valid
|
|
|
|
* as a base restriction clause.
|
|
|
|
*/
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
or_rinfo = make_restrictinfo(root,
|
|
|
|
orclause,
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
true,
|
|
|
|
false,
|
2023-05-25 16:28:33 +02:00
|
|
|
false,
|
|
|
|
false,
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
2017-01-18 18:58:20 +01:00
|
|
|
join_or_rinfo->security_level,
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
NULL,
|
2023-05-25 16:28:33 +02:00
|
|
|
NULL,
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
NULL);
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Estimate its selectivity. (We could have done this earlier, but doing
|
|
|
|
* it on the RestrictInfo representation allows the result to get cached,
|
|
|
|
* saving work later.)
|
|
|
|
*/
|
|
|
|
or_selec = clause_selectivity(root, (Node *) or_rinfo,
|
2017-04-07 01:10:51 +02:00
|
|
|
0, JOIN_INNER, NULL);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* The clause is only worth adding to the query if it rejects a useful
|
|
|
|
* fraction of the base relation's rows; otherwise, it's just going to
|
|
|
|
* cause duplicate computation (since we will still have to check the
|
|
|
|
* original OR clause when the join is formed). Somewhat arbitrarily, we
|
|
|
|
* set the selectivity threshold at 0.9.
|
|
|
|
*/
|
|
|
|
if (or_selec > 0.9)
|
|
|
|
return; /* forget it */
|
|
|
|
|
|
|
|
/*
|
|
|
|
* OK, add it to the rel's restriction-clause list.
|
|
|
|
*/
|
|
|
|
rel->baserestrictinfo = lappend(rel->baserestrictinfo, or_rinfo);
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
2017-01-18 18:58:20 +01:00
|
|
|
rel->baserestrict_min_security = Min(rel->baserestrict_min_security,
|
|
|
|
or_rinfo->security_level);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
/*
|
|
|
|
* Adjust the original join OR clause's cached selectivity to compensate
|
|
|
|
* for the selectivity of the added (but redundant) lower-level qual. This
|
|
|
|
* should result in the join rel getting approximately the same rows
|
|
|
|
* estimate as it would have gotten without all these shenanigans.
|
|
|
|
*
|
|
|
|
* XXX major hack alert: this depends on the assumption that the
|
|
|
|
* selectivity will stay cached.
|
|
|
|
*
|
|
|
|
* XXX another major hack: we adjust only norm_selec, the cached
|
|
|
|
* selectivity for JOIN_INNER semantics, even though the join clause
|
|
|
|
* might've been an outer-join clause. This is partly because we can't
|
|
|
|
* easily identify the relevant SpecialJoinInfo here, and partly because
|
|
|
|
* the linearity assumption we're making would fail anyway. (If it is an
|
|
|
|
* outer-join clause, "rel" must be on the nullable side, else we'd not
|
|
|
|
* have gotten here. So the computation of the join size is going to be
|
|
|
|
* quite nonlinear with respect to the size of "rel", so it's not clear
|
|
|
|
* how we ought to adjust outer_selec even if we could compute its
|
|
|
|
* original value correctly.)
|
|
|
|
*/
|
|
|
|
if (or_selec > 0)
|
|
|
|
{
|
|
|
|
SpecialJoinInfo sjinfo;
|
|
|
|
|
|
|
|
/*
|
|
|
|
* Make up a SpecialJoinInfo for JOIN_INNER semantics. (Compare
|
|
|
|
* approx_tuple_count() in costsize.c.)
|
|
|
|
*/
|
|
|
|
sjinfo.type = T_SpecialJoinInfo;
|
|
|
|
sjinfo.min_lefthand = bms_difference(join_or_rinfo->clause_relids,
|
|
|
|
rel->relids);
|
|
|
|
sjinfo.min_righthand = rel->relids;
|
|
|
|
sjinfo.syn_lefthand = sjinfo.min_lefthand;
|
|
|
|
sjinfo.syn_righthand = sjinfo.min_righthand;
|
|
|
|
sjinfo.jointype = JOIN_INNER;
|
Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees. This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.
To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle. PlaceHolderVars
receive similar decoration for the same reason.
In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos. This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.
This change affects FDWs that want to plan foreign joins. They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.
Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup. (The README additions
and comments mention some stuff that will appear in the follow-up.)
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:16:20 +01:00
|
|
|
sjinfo.ojrelid = 0;
|
|
|
|
sjinfo.commute_above_l = NULL;
|
|
|
|
sjinfo.commute_above_r = NULL;
|
2023-05-17 17:13:52 +02:00
|
|
|
sjinfo.commute_below_l = NULL;
|
|
|
|
sjinfo.commute_below_r = NULL;
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
/* we don't bother trying to make the remaining fields valid */
|
|
|
|
sjinfo.lhs_strict = false;
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
sjinfo.semi_can_btree = false;
|
|
|
|
sjinfo.semi_can_hash = false;
|
|
|
|
sjinfo.semi_operators = NIL;
|
|
|
|
sjinfo.semi_rhs_exprs = NIL;
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
/* Compute inner-join size */
|
|
|
|
orig_selec = clause_selectivity(root, (Node *) join_or_rinfo,
|
2017-04-07 01:10:51 +02:00
|
|
|
0, JOIN_INNER, &sjinfo);
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
|
|
|
|
/* And hack cached selectivity so join size remains the same */
|
|
|
|
join_or_rinfo->norm_selec = orig_selec / or_selec;
|
Do assorted mop-up in the planner.
Remove RestrictInfo.nullable_relids, along with a good deal of
infrastructure that calculated it. One use-case for it was in
join_clause_is_movable_to, but we can now replace that usage with
a check to see if the clause's relids include any outer join
that can null the target relation. The other use-case was in
join_clause_is_movable_into, but that test can just be dropped
entirely now that the clause's relids include outer joins.
Furthermore, join_clause_is_movable_into should now be
accurate enough that it will accept anything returned by
generate_join_implied_equalities, so we can restore the Assert
that was diked out in commit 95f4e59c3.
Remove the outerjoin_delayed mechanism. We needed this before to
prevent quals from getting evaluated below outer joins that should
null some of their vars. Now that we consider varnullingrels while
placing quals, that's taken care of automatically, so throw the
whole thing away.
Teach remove_useless_result_rtes to also remove useless FromExprs.
Having done that, the delay_upper_joins flag serves no purpose any
more and we can remove it, largely reverting 11086f2f2.
Use constant TRUE for "dummy" clauses when throwing back outer joins.
This improves on a hack I introduced in commit 6a6522529. If we
have a left-join clause l.x = r.y, and a WHERE clause l.x = constant,
we generate r.y = constant and then don't really have a need for the
join clause. But we must throw the join clause back anyway after
marking it redundant, so that the join search heuristics won't think
this is a clauseless join and avoid it. That was a kluge introduced
under time pressure, and after looking at it I thought of a better
way: let's just introduce constant-TRUE "join clauses" instead,
and get rid of them at the end. This improves the generated plans for
such cases by not having to test a redundant join clause. We can also
get rid of the ugly hack used to mark such clauses as redundant for
selectivity estimation.
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:44:36 +01:00
|
|
|
/* ensure result stays in sane range */
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
if (join_or_rinfo->norm_selec > 1)
|
|
|
|
join_or_rinfo->norm_selec = 1;
|
|
|
|
/* as explained above, we don't touch outer_selec */
|
|
|
|
}
|
|
|
|
}
|