1996-07-09 08:22:35 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
*
|
1999-02-14 00:22:53 +01:00
|
|
|
* indxpath.c
|
2005-04-22 23:58:32 +02:00
|
|
|
* Routines to determine which indexes are usable for scanning a
|
|
|
|
|
* given relation, and create Paths accordingly.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2024-01-04 02:49:05 +01:00
|
|
|
* Portions Copyright (c) 1996-2024, PostgreSQL Global Development Group
|
2000-01-26 06:58:53 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* IDENTIFICATION
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/backend/optimizer/path/indxpath.c
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
#include "postgres.h"
|
1998-04-27 06:08:07 +02:00
|
|
|
|
2000-04-16 06:41:03 +02:00
|
|
|
#include <math.h>
|
|
|
|
|
|
2015-05-15 22:03:16 +02:00
|
|
|
#include "access/stratnum.h"
|
2011-10-08 02:13:02 +02:00
|
|
|
#include "access/sysattr.h"
|
2006-07-13 19:47:02 +02:00
|
|
|
#include "catalog/pg_am.h"
|
1999-07-27 05:51:11 +02:00
|
|
|
#include "catalog/pg_operator.h"
|
2006-12-23 01:43:13 +01:00
|
|
|
#include "catalog/pg_opfamily.h"
|
2003-05-13 06:38:58 +02:00
|
|
|
#include "catalog/pg_type.h"
|
1996-07-09 08:22:35 +02:00
|
|
|
#include "nodes/makefuncs.h"
|
2018-12-19 23:46:07 +01:00
|
|
|
#include "nodes/nodeFuncs.h"
|
2019-02-12 03:26:08 +01:00
|
|
|
#include "nodes/supportnodes.h"
|
1996-07-09 08:22:35 +02:00
|
|
|
#include "optimizer/cost.h"
|
2019-01-29 21:48:51 +01:00
|
|
|
#include "optimizer/optimizer.h"
|
1999-07-16 07:00:38 +02:00
|
|
|
#include "optimizer/pathnode.h"
|
1998-04-27 06:08:07 +02:00
|
|
|
#include "optimizer/paths.h"
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
#include "optimizer/prep.h"
|
1999-07-16 07:00:38 +02:00
|
|
|
#include "optimizer/restrictinfo.h"
|
1998-04-27 06:08:07 +02:00
|
|
|
#include "utils/lsyscache.h"
|
2001-06-25 23:11:45 +02:00
|
|
|
#include "utils/selfuncs.h"
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2000-01-09 01:26:47 +01:00
|
|
|
|
Faster partition pruning
Add a new module backend/partitioning/partprune.c, implementing a more
sophisticated algorithm for partition pruning. The new module uses each
partition's "boundinfo" for pruning instead of constraint exclusion,
based on an idea proposed by Robert Haas of a "pruning program": a list
of steps generated from the query quals which are run iteratively to
obtain a list of partitions that must be scanned in order to satisfy
those quals.
At present, this targets planner-time partition pruning, but there exist
further patches to apply partition pruning at execution time as well.
This commit also moves some definitions from include/catalog/partition.h
to a new file include/partitioning/partbounds.h, in an attempt to
rationalize partitioning related code.
Authors: Amit Langote, David Rowley, Dilip Kumar
Reviewers: Robert Haas, Kyotaro Horiguchi, Ashutosh Bapat, Jesper Pedersen.
Discussion: https://postgr.es/m/098b9c71-1915-1a2a-8d52-1a7a50ce79e8@lab.ntt.co.jp
2018-04-06 21:23:04 +02:00
|
|
|
/* XXX see PartCollMatchesExprColl */
|
2011-09-29 06:43:42 +02:00
|
|
|
#define IndexCollMatchesExprColl(idxcollation, exprcollation) \
|
|
|
|
|
((idxcollation) == InvalidOid || (idxcollation) == (exprcollation))
|
|
|
|
|
|
2009-03-06 00:06:45 +01:00
|
|
|
/* Whether we are looking for plain indexscan, bitmap scan, or either */
|
|
|
|
|
typedef enum
|
|
|
|
|
{
|
|
|
|
|
ST_INDEXSCAN, /* must support amgettuple */
|
|
|
|
|
ST_BITMAPSCAN, /* must support amgetbitmap */
|
|
|
|
|
ST_ANYSCAN, /* either is okay */
|
|
|
|
|
} ScanTypeControl;
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* Data structure for collecting qual clauses that match an index */
|
|
|
|
|
typedef struct
|
|
|
|
|
{
|
|
|
|
|
bool nonempty; /* True if lists are not all empty */
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
/* Lists of IndexClause nodes, one list per index column */
|
2012-01-28 01:26:38 +01:00
|
|
|
List *indexclauses[INDEX_MAX_KEYS];
|
|
|
|
|
} IndexClauseSet;
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/* Per-path data used within choose_bitmap_and() */
|
|
|
|
|
typedef struct
|
|
|
|
|
{
|
|
|
|
|
Path *path; /* IndexPath, BitmapAndPath, or BitmapOrPath */
|
|
|
|
|
List *quals; /* the WHERE clauses it uses */
|
|
|
|
|
List *preds; /* predicates of its partial index(es) */
|
|
|
|
|
Bitmapset *clauseids; /* quals+preds represented as a bitmapset */
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
bool unclassifiable; /* has too many quals+preds to process? */
|
2007-04-17 22:03:03 +02:00
|
|
|
} PathClauseUsage;
|
|
|
|
|
|
2013-03-22 00:43:59 +01:00
|
|
|
/* Callback argument for ec_member_matches_indexcol */
|
|
|
|
|
typedef struct
|
|
|
|
|
{
|
|
|
|
|
IndexOptInfo *index; /* index we're considering */
|
|
|
|
|
int indexcol; /* index column we want to match to */
|
|
|
|
|
} ec_member_matches_arg;
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths);
|
2012-09-16 23:57:18 +02:00
|
|
|
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths,
|
|
|
|
|
List *indexjoinclauses,
|
2012-11-01 19:08:42 +01:00
|
|
|
int considered_clauses,
|
2012-09-16 23:57:18 +02:00
|
|
|
List **considered_relids);
|
|
|
|
|
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths,
|
|
|
|
|
Relids relids,
|
|
|
|
|
List **considered_relids);
|
2012-11-01 19:08:42 +01:00
|
|
|
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
|
|
|
|
|
List *indexjoinclauses);
|
2012-01-28 01:26:38 +01:00
|
|
|
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index, IndexClauseSet *clauses,
|
|
|
|
|
List **bitindexpaths);
|
|
|
|
|
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index, IndexClauseSet *clauses,
|
|
|
|
|
bool useful_predicate,
|
2014-10-26 21:12:22 +01:00
|
|
|
ScanTypeControl scantype,
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
bool *skip_nonnative_saop);
|
2012-01-28 01:26:38 +01:00
|
|
|
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
List *clauses, List *other_clauses);
|
2013-12-30 18:50:31 +01:00
|
|
|
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
List *clauses, List *other_clauses);
|
2006-07-22 17:41:56 +02:00
|
|
|
static Path *choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel,
|
2012-01-28 01:26:38 +01:00
|
|
|
List *paths);
|
2007-04-17 22:03:03 +02:00
|
|
|
static int path_usage_comparator(const void *a, const void *b);
|
|
|
|
|
static Cost bitmap_scan_cost_est(PlannerInfo *root, RelOptInfo *rel,
|
2012-01-28 01:26:38 +01:00
|
|
|
Path *ipath);
|
2006-07-22 17:41:56 +02:00
|
|
|
static Cost bitmap_and_cost_est(PlannerInfo *root, RelOptInfo *rel,
|
2012-01-28 01:26:38 +01:00
|
|
|
List *paths);
|
2007-04-17 22:03:03 +02:00
|
|
|
static PathClauseUsage *classify_index_clause_usage(Path *path,
|
|
|
|
|
List **clauselist);
|
2007-03-21 23:18:12 +01:00
|
|
|
static void find_indexpath_quals(Path *bitmapqual, List **quals, List **preds);
|
2007-04-17 22:03:03 +02:00
|
|
|
static int find_list_position(Node *node, List **nodelist);
|
2011-10-08 02:13:02 +02:00
|
|
|
static bool check_index_only(RelOptInfo *rel, IndexOptInfo *index);
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
static double get_loop_count(PlannerInfo *root, Index cur_relid, Relids outer_relids);
|
|
|
|
|
static double adjust_rowcount_for_semijoins(PlannerInfo *root,
|
|
|
|
|
Index cur_relid,
|
|
|
|
|
Index outer_relid,
|
|
|
|
|
double rowcount);
|
|
|
|
|
static double approximate_joinrel_size(PlannerInfo *root, Relids relids);
|
2019-02-12 03:26:08 +01:00
|
|
|
static void match_restriction_clauses_to_index(PlannerInfo *root,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *clauseset);
|
|
|
|
|
static void match_join_clauses_to_index(PlannerInfo *root,
|
|
|
|
|
RelOptInfo *rel, IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *clauseset,
|
|
|
|
|
List **joinorclauses);
|
|
|
|
|
static void match_eclass_clauses_to_index(PlannerInfo *root,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *clauseset);
|
2019-02-12 03:26:08 +01:00
|
|
|
static void match_clauses_to_index(PlannerInfo *root,
|
2012-01-28 01:26:38 +01:00
|
|
|
List *clauses,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexClauseSet *clauseset);
|
2019-02-12 03:26:08 +01:00
|
|
|
static void match_clause_to_index(PlannerInfo *root,
|
2012-01-28 01:26:38 +01:00
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexClauseSet *clauseset);
|
2019-02-12 03:26:08 +01:00
|
|
|
static IndexClause *match_clause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2010-11-20 19:57:27 +01:00
|
|
|
int indexcol,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index);
|
2022-09-02 23:01:51 +02:00
|
|
|
static bool IsBooleanOpfamily(Oid opfamily);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
static IndexClause *match_boolean_index_clause(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol, IndexOptInfo *index);
|
|
|
|
|
static IndexClause *match_opclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index);
|
|
|
|
|
static IndexClause *match_funcclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index);
|
|
|
|
|
static IndexClause *get_index_clause_from_support(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
Oid funcid,
|
|
|
|
|
int indexarg,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
static IndexClause *match_saopclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
static IndexClause *match_rowcompare_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2006-01-25 21:29:24 +01:00
|
|
|
int indexcol,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
static IndexClause *expand_indexqual_rowcompare(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
Oid expr_op,
|
|
|
|
|
bool var_on_left);
|
2011-12-25 01:03:21 +01:00
|
|
|
static void match_pathkeys_to_index(IndexOptInfo *index, List *pathkeys,
|
|
|
|
|
List **orderby_clauses_p,
|
|
|
|
|
List **clause_columns_p);
|
2010-12-03 02:50:48 +01:00
|
|
|
static Expr *match_clause_to_ordering_op(IndexOptInfo *index,
|
|
|
|
|
int indexcol, Expr *clause, Oid pk_opfamily);
|
2013-03-22 00:43:59 +01:00
|
|
|
static bool ec_member_matches_indexcol(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
EquivalenceClass *ec, EquivalenceMember *em,
|
|
|
|
|
void *arg);
|
1996-07-09 08:22:35 +02:00
|
|
|
|
|
|
|
|
|
1999-07-23 05:34:49 +02:00
|
|
|
/*
|
|
|
|
|
* create_index_paths()
|
|
|
|
|
* Generate all interesting index paths for the given relation.
|
2000-02-15 21:49:31 +01:00
|
|
|
* Candidate paths are added to the rel's pathlist (using add_path).
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
1999-07-30 06:07:25 +02:00
|
|
|
* To be considered for an index scan, an index must match one or more
|
1999-08-21 05:49:17 +02:00
|
|
|
* restriction clauses or join clauses from the query's qual condition,
|
2005-06-14 01:14:49 +02:00
|
|
|
* or match the query's ORDER BY condition, or have a predicate that
|
|
|
|
|
* matches the query's qual condition.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
1999-07-30 06:07:25 +02:00
|
|
|
* There are two basic kinds of index scans. A "plain" index scan uses
|
|
|
|
|
* only restriction clauses (possibly none at all) in its indexqual,
|
2012-01-28 01:26:38 +01:00
|
|
|
* so it can be applied in any context. A "parameterized" index scan uses
|
1999-07-30 06:07:25 +02:00
|
|
|
* join clauses (plus restriction clauses, if available) in its indexqual.
|
2012-01-28 01:26:38 +01:00
|
|
|
* When joining such a scan to one of the relations supplying the other
|
|
|
|
|
* variables used in its indexqual, the parameterized scan must appear as
|
|
|
|
|
* the inner relation of a nestloop join; it can't be used on the outer side,
|
|
|
|
|
* nor in a merge or hash join. In that context, values for the other rels'
|
1999-07-30 06:07:25 +02:00
|
|
|
* attributes are available and fixed during any one scan of the indexpath.
|
|
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* An IndexPath is generated and submitted to add_path() for each plain or
|
|
|
|
|
* parameterized index scan this routine deems potentially interesting for
|
|
|
|
|
* the current query.
|
1999-07-30 06:07:25 +02:00
|
|
|
*
|
1999-07-23 05:34:49 +02:00
|
|
|
* 'rel' is the relation for which we want to generate index paths
|
2004-01-05 06:07:36 +01:00
|
|
|
*
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
* Note: check_index_predicates() must have been run previously for this rel.
|
2012-08-27 04:48:55 +02:00
|
|
|
*
|
2013-08-18 02:22:37 +02:00
|
|
|
* Note: in cases involving LATERAL references in the relation's tlist, it's
|
|
|
|
|
* possible that rel->lateral_relids is nonempty. Currently, we include
|
|
|
|
|
* lateral_relids into the parameterization reported for each path, but don't
|
|
|
|
|
* take it into account otherwise. The fact that any such rels *must* be
|
|
|
|
|
* available as parameter sources perhaps should influence our choices of
|
|
|
|
|
* index quals ... but for now, it doesn't seem worth troubling over.
|
|
|
|
|
* In particular, comments below about "unparameterized" paths should be read
|
|
|
|
|
* as meaning "unparameterized so far as the indexquals are concerned".
|
1996-07-09 08:22:35 +02:00
|
|
|
*/
|
2000-02-15 21:49:31 +01:00
|
|
|
void
|
2005-06-06 00:32:58 +02:00
|
|
|
create_index_paths(PlannerInfo *root, RelOptInfo *rel)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2005-04-22 23:58:32 +02:00
|
|
|
List *indexpaths;
|
|
|
|
|
List *bitindexpaths;
|
2012-01-28 01:26:38 +01:00
|
|
|
List *bitjoinpaths;
|
|
|
|
|
List *joinorclauses;
|
|
|
|
|
IndexClauseSet rclauseset;
|
|
|
|
|
IndexClauseSet jclauseset;
|
|
|
|
|
IndexClauseSet eclauseset;
|
2012-08-30 20:32:22 +02:00
|
|
|
ListCell *lc;
|
2005-04-22 23:58:32 +02:00
|
|
|
|
|
|
|
|
/* Skip the whole mess if no indexes */
|
|
|
|
|
if (rel->indexlist == NIL)
|
|
|
|
|
return;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/* Bitmap paths are collected and then dealt with at the end */
|
|
|
|
|
bitindexpaths = bitjoinpaths = joinorclauses = NIL;
|
|
|
|
|
|
|
|
|
|
/* Examine each index in turn */
|
2012-08-30 20:32:22 +02:00
|
|
|
foreach(lc, rel->indexlist)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2012-08-30 20:32:22 +02:00
|
|
|
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/* Protect limited-size array in IndexClauseSets */
|
2019-02-13 00:38:32 +01:00
|
|
|
Assert(index->nkeycolumns <= INDEX_MAX_KEYS);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Ignore partial indexes that do not match the query.
|
|
|
|
|
* (generate_bitmap_or_paths() might be able to do something with
|
|
|
|
|
* them, but that's of no concern here.)
|
|
|
|
|
*/
|
|
|
|
|
if (index->indpred != NIL && !index->predOK)
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Identify the restriction clauses that can match the index.
|
|
|
|
|
*/
|
|
|
|
|
MemSet(&rclauseset, 0, sizeof(rclauseset));
|
2019-02-12 03:26:08 +01:00
|
|
|
match_restriction_clauses_to_index(root, index, &rclauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Build index paths from the restriction clauses. These will be
|
|
|
|
|
* non-parameterized paths. Plain paths go directly to add_path(),
|
|
|
|
|
* bitmap paths are added to bitindexpaths to be handled below.
|
|
|
|
|
*/
|
|
|
|
|
get_index_paths(root, rel, index, &rclauseset,
|
|
|
|
|
&bitindexpaths);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Identify the join clauses that can match the index. For the moment
|
|
|
|
|
* we keep them separate from the restriction clauses. Note that this
|
2012-09-16 23:57:18 +02:00
|
|
|
* step finds only "loose" join clauses that have not been merged into
|
2012-01-28 01:26:38 +01:00
|
|
|
* EquivalenceClasses. Also, collect join OR clauses for later.
|
|
|
|
|
*/
|
|
|
|
|
MemSet(&jclauseset, 0, sizeof(jclauseset));
|
2013-08-18 02:22:37 +02:00
|
|
|
match_join_clauses_to_index(root, rel, index,
|
2012-01-28 01:26:38 +01:00
|
|
|
&jclauseset, &joinorclauses);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Look for EquivalenceClasses that can generate joinclauses matching
|
|
|
|
|
* the index.
|
|
|
|
|
*/
|
|
|
|
|
MemSet(&eclauseset, 0, sizeof(eclauseset));
|
2013-08-18 02:22:37 +02:00
|
|
|
match_eclass_clauses_to_index(root, index,
|
2012-08-30 20:32:22 +02:00
|
|
|
&eclauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/*
|
2012-09-16 23:57:18 +02:00
|
|
|
* If we found any plain or eclass join clauses, build parameterized
|
|
|
|
|
* index paths using them.
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
if (jclauseset.nonempty || eclauseset.nonempty)
|
|
|
|
|
consider_index_join_clauses(root, rel, index,
|
|
|
|
|
&rclauseset,
|
|
|
|
|
&jclauseset,
|
|
|
|
|
&eclauseset,
|
|
|
|
|
&bitjoinpaths);
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Generate BitmapOrPaths for any suitable OR-clauses present in the
|
|
|
|
|
* restriction list. Add these to bitindexpaths.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
indexpaths = generate_bitmap_or_paths(root, rel,
|
2013-12-30 18:50:31 +01:00
|
|
|
rel->baserestrictinfo, NIL);
|
2012-01-28 01:26:38 +01:00
|
|
|
bitindexpaths = list_concat(bitindexpaths, indexpaths);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
|
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Likewise, generate BitmapOrPaths for any suitable OR-clauses present in
|
|
|
|
|
* the joinclause list. Add these to bitjoinpaths.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
indexpaths = generate_bitmap_or_paths(root, rel,
|
2013-12-30 18:50:31 +01:00
|
|
|
joinorclauses, rel->baserestrictinfo);
|
2012-01-28 01:26:38 +01:00
|
|
|
bitjoinpaths = list_concat(bitjoinpaths, indexpaths);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we found anything usable, generate a BitmapHeapPath for the most
|
|
|
|
|
* promising combination of restriction bitmap index paths. Note there
|
|
|
|
|
* will be only one such path no matter how many indexes exist. This
|
|
|
|
|
* should be sufficient since there's basically only one figure of merit
|
|
|
|
|
* (total cost) for such a path.
|
|
|
|
|
*/
|
|
|
|
|
if (bitindexpaths != NIL)
|
|
|
|
|
{
|
|
|
|
|
Path *bitmapqual;
|
|
|
|
|
BitmapHeapPath *bpath;
|
|
|
|
|
|
|
|
|
|
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
|
2012-08-27 04:48:55 +02:00
|
|
|
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
rel->lateral_relids, 1.0, 0);
|
2012-01-28 01:26:38 +01:00
|
|
|
add_path(rel, (Path *) bpath);
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
|
|
|
|
|
/* create a partial bitmap heap path */
|
|
|
|
|
if (rel->consider_parallel && rel->lateral_relids == NULL)
|
|
|
|
|
create_partial_bitmap_paths(root, rel, bitmapqual);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2012-08-16 19:03:54 +02:00
|
|
|
* Likewise, if we found anything usable, generate BitmapHeapPaths for the
|
|
|
|
|
* most promising combinations of join bitmap index paths. Our strategy
|
|
|
|
|
* is to generate one such path for each distinct parameterization seen
|
|
|
|
|
* among the available bitmap index paths. This may look pretty
|
|
|
|
|
* expensive, but usually there won't be very many distinct
|
2012-09-16 23:57:18 +02:00
|
|
|
* parameterizations. (This logic is quite similar to that in
|
|
|
|
|
* consider_index_join_clauses, but we're working with whole paths not
|
|
|
|
|
* individual clauses.)
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
if (bitjoinpaths != NIL)
|
|
|
|
|
{
|
2012-08-16 19:03:54 +02:00
|
|
|
List *all_path_outers;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
/* Identify each distinct parameterization seen in bitjoinpaths */
|
|
|
|
|
all_path_outers = NIL;
|
2012-08-16 19:03:54 +02:00
|
|
|
foreach(lc, bitjoinpaths)
|
|
|
|
|
{
|
|
|
|
|
Path *path = (Path *) lfirst(lc);
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
Relids required_outer = PATH_REQ_OUTER(path);
|
2012-08-16 19:03:54 +02:00
|
|
|
|
2022-11-13 16:22:45 +01:00
|
|
|
all_path_outers = list_append_unique(all_path_outers,
|
|
|
|
|
required_outer);
|
2012-08-16 19:03:54 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Now, for each distinct parameterization set ... */
|
|
|
|
|
foreach(lc, all_path_outers)
|
|
|
|
|
{
|
|
|
|
|
Relids max_outers = (Relids) lfirst(lc);
|
|
|
|
|
List *this_path_set;
|
|
|
|
|
Path *bitmapqual;
|
|
|
|
|
Relids required_outer;
|
|
|
|
|
double loop_count;
|
|
|
|
|
BitmapHeapPath *bpath;
|
|
|
|
|
ListCell *lcp;
|
|
|
|
|
|
|
|
|
|
/* Identify all the bitmap join paths needing no more than that */
|
|
|
|
|
this_path_set = NIL;
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
foreach(lcp, bitjoinpaths)
|
2012-08-16 19:03:54 +02:00
|
|
|
{
|
|
|
|
|
Path *path = (Path *) lfirst(lcp);
|
|
|
|
|
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
if (bms_is_subset(PATH_REQ_OUTER(path), max_outers))
|
2012-08-16 19:03:54 +02:00
|
|
|
this_path_set = lappend(this_path_set, path);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Add in restriction bitmap paths, since they can be used
|
|
|
|
|
* together with any join paths.
|
|
|
|
|
*/
|
|
|
|
|
this_path_set = list_concat(this_path_set, bitindexpaths);
|
|
|
|
|
|
|
|
|
|
/* Select best AND combination for this parameterization */
|
|
|
|
|
bitmapqual = choose_bitmap_and(root, rel, this_path_set);
|
|
|
|
|
|
|
|
|
|
/* And push that path into the mix */
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
required_outer = PATH_REQ_OUTER(bitmapqual);
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
loop_count = get_loop_count(root, rel->relid, required_outer);
|
2012-08-16 19:03:54 +02:00
|
|
|
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
required_outer, loop_count, 0);
|
2012-08-16 19:03:54 +02:00
|
|
|
add_path(rel, (Path *) bpath);
|
|
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* consider_index_join_clauses
|
|
|
|
|
* Given sets of join clauses for an index, decide which parameterized
|
|
|
|
|
* index paths to build.
|
|
|
|
|
*
|
|
|
|
|
* Plain indexpaths are sent directly to add_path, while potential
|
|
|
|
|
* bitmap indexpaths are added to *bitindexpaths for later processing.
|
|
|
|
|
*
|
|
|
|
|
* 'rel' is the index's heap relation
|
|
|
|
|
* 'index' is the index for which we want to generate paths
|
|
|
|
|
* 'rclauseset' is the collection of indexable restriction clauses
|
|
|
|
|
* 'jclauseset' is the collection of indexable simple join clauses
|
|
|
|
|
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
|
|
|
|
|
* '*bitindexpaths' is the list to add bitmap paths to
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths)
|
|
|
|
|
{
|
2012-11-01 19:08:42 +01:00
|
|
|
int considered_clauses = 0;
|
2012-09-16 23:57:18 +02:00
|
|
|
List *considered_relids = NIL;
|
2012-01-28 01:26:38 +01:00
|
|
|
int indexcol;
|
|
|
|
|
|
|
|
|
|
/*
|
2012-09-16 23:57:18 +02:00
|
|
|
* The strategy here is to identify every potentially useful set of outer
|
|
|
|
|
* rels that can provide indexable join clauses. For each such set,
|
|
|
|
|
* select all the join clauses available from those outer rels, add on all
|
|
|
|
|
* the indexable restriction clauses, and generate plain and/or bitmap
|
|
|
|
|
* index paths for that set of clauses. This is based on the assumption
|
|
|
|
|
* that it's always better to apply a clause as an indexqual than as a
|
|
|
|
|
* filter (qpqual); which is where an available clause would end up being
|
|
|
|
|
* applied if we omit it from the indexquals.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
2012-11-01 19:08:42 +01:00
|
|
|
* This looks expensive, but in most practical cases there won't be very
|
|
|
|
|
* many distinct sets of outer rels to consider. As a safety valve when
|
|
|
|
|
* that's not true, we use a heuristic: limit the number of outer rel sets
|
|
|
|
|
* considered to a multiple of the number of clauses considered. (We'll
|
|
|
|
|
* always consider using each individual join clause, though.)
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
2012-09-16 23:57:18 +02:00
|
|
|
* For simplicity in selecting relevant clauses, we represent each set of
|
|
|
|
|
* outer rels as a maximum set of clause_relids --- that is, the indexed
|
|
|
|
|
* relation itself is also included in the relids set. considered_relids
|
|
|
|
|
* lists all relids sets we've already tried.
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
2019-02-13 00:38:32 +01:00
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2012-09-16 23:57:18 +02:00
|
|
|
/* Consider each applicable simple join clause */
|
2012-11-01 19:08:42 +01:00
|
|
|
considered_clauses += list_length(jclauseset->indexclauses[indexcol]);
|
2012-09-16 23:57:18 +02:00
|
|
|
consider_index_join_outer_rels(root, rel, index,
|
|
|
|
|
rclauseset, jclauseset, eclauseset,
|
|
|
|
|
bitindexpaths,
|
|
|
|
|
jclauseset->indexclauses[indexcol],
|
2012-11-01 19:08:42 +01:00
|
|
|
considered_clauses,
|
2012-09-16 23:57:18 +02:00
|
|
|
&considered_relids);
|
|
|
|
|
/* Consider each applicable eclass join clause */
|
2012-11-01 19:08:42 +01:00
|
|
|
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
|
2012-09-16 23:57:18 +02:00
|
|
|
consider_index_join_outer_rels(root, rel, index,
|
|
|
|
|
rclauseset, jclauseset, eclauseset,
|
|
|
|
|
bitindexpaths,
|
|
|
|
|
eclauseset->indexclauses[indexcol],
|
2012-11-01 19:08:42 +01:00
|
|
|
considered_clauses,
|
2012-09-16 23:57:18 +02:00
|
|
|
&considered_relids);
|
|
|
|
|
}
|
|
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/*
|
|
|
|
|
* consider_index_join_outer_rels
|
|
|
|
|
* Generate parameterized paths based on clause relids in the clause list.
|
|
|
|
|
*
|
|
|
|
|
* Workhorse for consider_index_join_clauses; see notes therein for rationale.
|
|
|
|
|
*
|
|
|
|
|
* 'rel', 'index', 'rclauseset', 'jclauseset', 'eclauseset', and
|
|
|
|
|
* 'bitindexpaths' as above
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* 'indexjoinclauses' is a list of IndexClauses for join clauses
|
2012-11-01 19:08:42 +01:00
|
|
|
* 'considered_clauses' is the total number of clauses considered (so far)
|
2012-09-16 23:57:18 +02:00
|
|
|
* '*considered_relids' is a list of all relids sets already considered
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths,
|
|
|
|
|
List *indexjoinclauses,
|
2012-11-01 19:08:42 +01:00
|
|
|
int considered_clauses,
|
2012-09-16 23:57:18 +02:00
|
|
|
List **considered_relids)
|
|
|
|
|
{
|
|
|
|
|
ListCell *lc;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* Examine relids of each joinclause in the given list */
|
|
|
|
|
foreach(lc, indexjoinclauses)
|
|
|
|
|
{
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(lc);
|
|
|
|
|
Relids clause_relids = iclause->rinfo->clause_relids;
|
|
|
|
|
EquivalenceClass *parent_ec = iclause->rinfo->parent_ec;
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
int num_considered_relids;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* If we already tried its relids set, no need to do so again */
|
2022-11-13 16:22:45 +01:00
|
|
|
if (list_member(*considered_relids, clause_relids))
|
2012-09-16 23:57:18 +02:00
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Generate the union of this clause's relids set with each
|
|
|
|
|
* previously-tried set. This ensures we try this clause along with
|
2012-11-01 19:08:42 +01:00
|
|
|
* every interesting subset of previous clauses. However, to avoid
|
|
|
|
|
* exponential growth of planning time when there are many clauses,
|
|
|
|
|
* limit the number of relid sets accepted to 10 * considered_clauses.
|
2012-09-16 23:57:18 +02:00
|
|
|
*
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
* Note: get_join_index_paths appends entries to *considered_relids,
|
|
|
|
|
* but we do not need to visit such newly-added entries within this
|
|
|
|
|
* loop, so we don't use foreach() here. No real harm would be done
|
|
|
|
|
* if we did visit them, since the subset check would reject them; but
|
|
|
|
|
* it would waste some cycles.
|
2012-09-16 23:57:18 +02:00
|
|
|
*/
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
num_considered_relids = list_length(*considered_relids);
|
|
|
|
|
for (int pos = 0; pos < num_considered_relids; pos++)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
Relids oldrelids = (Relids) list_nth(*considered_relids, pos);
|
2012-09-16 23:57:18 +02:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If either is a subset of the other, no new set is possible.
|
|
|
|
|
* This isn't a complete test for redundancy, but it's easy and
|
|
|
|
|
* cheap. get_join_index_paths will check more carefully if we
|
|
|
|
|
* already generated the same relids set.
|
|
|
|
|
*/
|
|
|
|
|
if (bms_subset_compare(clause_relids, oldrelids) != BMS_DIFFERENT)
|
|
|
|
|
continue;
|
|
|
|
|
|
2012-11-01 19:08:42 +01:00
|
|
|
/*
|
|
|
|
|
* If this clause was derived from an equivalence class, the
|
|
|
|
|
* clause list may contain other clauses derived from the same
|
|
|
|
|
* eclass. We should not consider that combining this clause with
|
|
|
|
|
* one of those clauses generates a usefully different
|
|
|
|
|
* parameterization; so skip if any clause derived from the same
|
|
|
|
|
* eclass would already have been included when using oldrelids.
|
|
|
|
|
*/
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
if (parent_ec &&
|
|
|
|
|
eclass_already_used(parent_ec, oldrelids,
|
2012-11-01 19:08:42 +01:00
|
|
|
indexjoinclauses))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If the number of relid sets considered exceeds our heuristic
|
|
|
|
|
* limit, stop considering combinations of clauses. We'll still
|
|
|
|
|
* consider the current clause alone, though (below this loop).
|
|
|
|
|
*/
|
|
|
|
|
if (list_length(*considered_relids) >= 10 * considered_clauses)
|
|
|
|
|
break;
|
|
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* OK, try the union set */
|
|
|
|
|
get_join_index_paths(root, rel, index,
|
|
|
|
|
rclauseset, jclauseset, eclauseset,
|
|
|
|
|
bitindexpaths,
|
|
|
|
|
bms_union(clause_relids, oldrelids),
|
|
|
|
|
considered_relids);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
2012-09-16 23:57:18 +02:00
|
|
|
|
|
|
|
|
/* Also try this set of relids by itself */
|
|
|
|
|
get_join_index_paths(root, rel, index,
|
|
|
|
|
rclauseset, jclauseset, eclauseset,
|
|
|
|
|
bitindexpaths,
|
|
|
|
|
clause_relids,
|
|
|
|
|
considered_relids);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2012-09-16 23:57:18 +02:00
|
|
|
* get_join_index_paths
|
|
|
|
|
* Generate index paths using clauses from the specified outer relations.
|
|
|
|
|
* In addition to generating paths, relids is added to *considered_relids
|
|
|
|
|
* if not already present.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
|
|
|
|
* Workhorse for consider_index_join_clauses; see notes therein for rationale.
|
|
|
|
|
*
|
2012-09-16 23:57:18 +02:00
|
|
|
* 'rel', 'index', 'rclauseset', 'jclauseset', 'eclauseset',
|
|
|
|
|
* 'bitindexpaths', 'considered_relids' as above
|
|
|
|
|
* 'relids' is the current set of relids to consider (the target rel plus
|
|
|
|
|
* one or more outer rels)
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
static void
|
2012-09-16 23:57:18 +02:00
|
|
|
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *rclauseset,
|
|
|
|
|
IndexClauseSet *jclauseset,
|
|
|
|
|
IndexClauseSet *eclauseset,
|
|
|
|
|
List **bitindexpaths,
|
|
|
|
|
Relids relids,
|
|
|
|
|
List **considered_relids)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2012-09-16 23:57:18 +02:00
|
|
|
IndexClauseSet clauseset;
|
|
|
|
|
int indexcol;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* If we already considered this relids set, don't repeat the work */
|
2022-11-13 16:22:45 +01:00
|
|
|
if (list_member(*considered_relids, relids))
|
2012-01-28 01:26:38 +01:00
|
|
|
return;
|
|
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* Identify indexclauses usable with this relids set */
|
|
|
|
|
MemSet(&clauseset, 0, sizeof(clauseset));
|
|
|
|
|
|
2019-02-13 00:38:32 +01:00
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2012-09-16 23:57:18 +02:00
|
|
|
ListCell *lc;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* First find applicable simple join clauses */
|
|
|
|
|
foreach(lc, jclauseset->indexclauses[indexcol])
|
|
|
|
|
{
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(lc);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
if (bms_is_subset(iclause->rinfo->clause_relids, relids))
|
2012-09-16 23:57:18 +02:00
|
|
|
clauseset.indexclauses[indexcol] =
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
lappend(clauseset.indexclauses[indexcol], iclause);
|
2012-09-16 23:57:18 +02:00
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/*
|
|
|
|
|
* Add applicable eclass join clauses. The clauses generated for each
|
2013-03-22 00:43:59 +01:00
|
|
|
* column are redundant (cf generate_implied_equalities_for_column),
|
2012-09-16 23:57:18 +02:00
|
|
|
* so we need at most one. This is the only exception to the general
|
|
|
|
|
* rule of using all available index clauses.
|
|
|
|
|
*/
|
|
|
|
|
foreach(lc, eclauseset->indexclauses[indexcol])
|
|
|
|
|
{
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(lc);
|
2012-09-16 23:57:18 +02:00
|
|
|
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
if (bms_is_subset(iclause->rinfo->clause_relids, relids))
|
2012-09-16 23:57:18 +02:00
|
|
|
{
|
|
|
|
|
clauseset.indexclauses[indexcol] =
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
lappend(clauseset.indexclauses[indexcol], iclause);
|
2012-09-16 23:57:18 +02:00
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
/* Add restriction clauses */
|
2012-09-16 23:57:18 +02:00
|
|
|
clauseset.indexclauses[indexcol] =
|
|
|
|
|
list_concat(clauseset.indexclauses[indexcol],
|
|
|
|
|
rclauseset->indexclauses[indexcol]);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
if (clauseset.indexclauses[indexcol] != NIL)
|
|
|
|
|
clauseset.nonempty = true;
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
|
2012-09-16 23:57:18 +02:00
|
|
|
/* We should have found something, else caller passed silly relids */
|
|
|
|
|
Assert(clauseset.nonempty);
|
|
|
|
|
|
|
|
|
|
/* Build index path(s) using the collected set of clauses */
|
|
|
|
|
get_index_paths(root, rel, index, &clauseset, bitindexpaths);
|
|
|
|
|
|
|
|
|
|
/*
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
* Remember we considered paths for this set of relids.
|
2012-09-16 23:57:18 +02:00
|
|
|
*/
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
*considered_relids = lappend(*considered_relids, relids);
|
2012-09-16 23:57:18 +02:00
|
|
|
}
|
|
|
|
|
|
2012-11-01 19:08:42 +01:00
|
|
|
/*
|
|
|
|
|
* eclass_already_used
|
|
|
|
|
* True if any join clause usable with oldrelids was generated from
|
|
|
|
|
* the specified equivalence class.
|
|
|
|
|
*/
|
|
|
|
|
static bool
|
|
|
|
|
eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
|
|
|
|
|
List *indexjoinclauses)
|
|
|
|
|
{
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
foreach(lc, indexjoinclauses)
|
|
|
|
|
{
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(lc);
|
|
|
|
|
RestrictInfo *rinfo = iclause->rinfo;
|
2012-11-01 19:08:42 +01:00
|
|
|
|
|
|
|
|
if (rinfo->parent_ec == parent_ec &&
|
|
|
|
|
bms_is_subset(rinfo->clause_relids, oldrelids))
|
|
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* get_index_paths
|
|
|
|
|
* Given an index and a set of index clauses for it, construct IndexPaths.
|
|
|
|
|
*
|
|
|
|
|
* Plain indexpaths are sent directly to add_path, while potential
|
|
|
|
|
* bitmap indexpaths are added to *bitindexpaths for later processing.
|
|
|
|
|
*
|
|
|
|
|
* This is a fairly simple frontend to build_index_paths(). Its reason for
|
|
|
|
|
* existence is mainly to handle ScalarArrayOpExpr quals properly. If the
|
|
|
|
|
* index AM supports them natively, we should just include them in simple
|
|
|
|
|
* index paths. If not, we should exclude them while building simple index
|
|
|
|
|
* paths, and then make a separate attempt to include them in bitmap paths.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index, IndexClauseSet *clauses,
|
|
|
|
|
List **bitindexpaths)
|
|
|
|
|
{
|
|
|
|
|
List *indexpaths;
|
2014-10-26 21:12:22 +01:00
|
|
|
bool skip_nonnative_saop = false;
|
2012-01-28 01:26:38 +01:00
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
* clauses only if the index AM supports them natively.
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
indexpaths = build_index_paths(root, rel,
|
|
|
|
|
index, clauses,
|
|
|
|
|
index->predOK,
|
2014-10-26 21:12:22 +01:00
|
|
|
ST_ANYSCAN,
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
&skip_nonnative_saop);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
|
|
|
|
/*
|
2011-10-11 20:20:06 +02:00
|
|
|
* Submit all the ones that can form plain IndexScan plans to add_path. (A
|
2012-01-28 01:26:38 +01:00
|
|
|
* plain IndexPath can represent either a plain IndexScan or an
|
|
|
|
|
* IndexOnlyScan, but for our purposes here that distinction does not
|
2011-10-11 20:20:06 +02:00
|
|
|
* matter. However, some of the indexes might support only bitmap scans,
|
2012-01-28 01:26:38 +01:00
|
|
|
* and those we mustn't submit to add_path here.)
|
|
|
|
|
*
|
|
|
|
|
* Also, pick out the ones that are usable as bitmap scans. For that, we
|
|
|
|
|
* must discard indexes that don't support bitmap scans, and we also are
|
|
|
|
|
* only interested in paths that have some selectivity; we should discard
|
|
|
|
|
* anything that was generated solely for ordering purposes.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, indexpaths)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexPath *ipath = (IndexPath *) lfirst(lc);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
if (index->amhasgettuple)
|
2009-03-06 00:06:45 +01:00
|
|
|
add_path(rel, (Path *) ipath);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
if (index->amhasgetbitmap &&
|
2011-01-11 18:12:04 +01:00
|
|
|
(ipath->path.pathkeys == NIL ||
|
|
|
|
|
ipath->indexselectivity < 1.0))
|
2012-01-28 01:26:38 +01:00
|
|
|
*bitindexpaths = lappend(*bitindexpaths, ipath);
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2014-10-26 21:12:22 +01:00
|
|
|
* If there were ScalarArrayOpExpr clauses that the index can't handle
|
|
|
|
|
* natively, generate bitmap scan paths relying on executor-managed
|
|
|
|
|
* ScalarArrayOpExpr.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2014-10-26 21:12:22 +01:00
|
|
|
if (skip_nonnative_saop)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
|
|
|
|
indexpaths = build_index_paths(root, rel,
|
|
|
|
|
index, clauses,
|
|
|
|
|
false,
|
2014-10-26 21:12:22 +01:00
|
|
|
ST_BITMAPSCAN,
|
|
|
|
|
NULL);
|
2012-01-28 01:26:38 +01:00
|
|
|
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* build_index_paths
|
|
|
|
|
* Given an index and a set of index clauses for it, construct zero
|
2017-02-15 19:53:24 +01:00
|
|
|
* or more IndexPaths. It also constructs zero or more partial IndexPaths.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
|
|
|
|
* We return a list of paths because (1) this routine checks some cases
|
|
|
|
|
* that should cause us to not generate any IndexPath, and (2) in some
|
|
|
|
|
* cases we want to consider both a forward and a backward scan, so as
|
|
|
|
|
* to obtain both sort orders. Note that the paths are just returned
|
|
|
|
|
* to the caller and not immediately fed to add_path().
|
|
|
|
|
*
|
|
|
|
|
* At top level, useful_predicate should be exactly the index's predOK flag
|
|
|
|
|
* (ie, true if it has a predicate that was proven from the restriction
|
|
|
|
|
* clauses). When working on an arm of an OR clause, useful_predicate
|
|
|
|
|
* should be true if the predicate required the current OR list to be proven.
|
|
|
|
|
* Note that this routine should never be called at all if the index has an
|
|
|
|
|
* unprovable predicate.
|
|
|
|
|
*
|
|
|
|
|
* scantype indicates whether we want to create plain indexscans, bitmap
|
|
|
|
|
* indexscans, or both. When it's ST_BITMAPSCAN, we will not consider
|
|
|
|
|
* index ordering while deciding if a Path is worth generating.
|
|
|
|
|
*
|
2014-10-26 21:12:22 +01:00
|
|
|
* If skip_nonnative_saop is non-NULL, we ignore ScalarArrayOpExpr clauses
|
|
|
|
|
* unless the index AM supports them directly, and we set *skip_nonnative_saop
|
2017-08-16 06:22:32 +02:00
|
|
|
* to true if we found any such clauses (caller must initialize the variable
|
|
|
|
|
* to false). If it's NULL, we do not ignore ScalarArrayOpExpr clauses.
|
2014-10-26 21:12:22 +01:00
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* 'rel' is the index's heap relation
|
|
|
|
|
* 'index' is the index for which we want to generate paths
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* 'clauses' is the collection of indexable clauses (IndexClause nodes)
|
2012-01-28 01:26:38 +01:00
|
|
|
* 'useful_predicate' indicates whether the index has a useful predicate
|
|
|
|
|
* 'scantype' indicates whether we need plain or bitmap scan support
|
2014-10-26 21:12:22 +01:00
|
|
|
* 'skip_nonnative_saop' indicates whether to accept SAOP if index AM doesn't
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
static List *
|
|
|
|
|
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
IndexOptInfo *index, IndexClauseSet *clauses,
|
|
|
|
|
bool useful_predicate,
|
2014-10-26 21:12:22 +01:00
|
|
|
ScanTypeControl scantype,
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
bool *skip_nonnative_saop)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
|
|
|
|
List *result = NIL;
|
|
|
|
|
IndexPath *ipath;
|
|
|
|
|
List *index_clauses;
|
|
|
|
|
Relids outer_relids;
|
|
|
|
|
double loop_count;
|
|
|
|
|
List *orderbyclauses;
|
|
|
|
|
List *orderbyclausecols;
|
|
|
|
|
List *index_pathkeys;
|
|
|
|
|
List *useful_pathkeys;
|
|
|
|
|
bool pathkeys_possibly_useful;
|
|
|
|
|
bool index_is_ordered;
|
|
|
|
|
bool index_only_scan;
|
|
|
|
|
int indexcol;
|
2005-04-22 23:58:32 +02:00
|
|
|
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
Assert(skip_nonnative_saop != NULL || scantype == ST_BITMAPSCAN);
|
|
|
|
|
|
2006-05-18 19:12:10 +02:00
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Check that index supports the desired scan type(s)
|
2006-05-18 19:12:10 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
switch (scantype)
|
|
|
|
|
{
|
|
|
|
|
case ST_INDEXSCAN:
|
|
|
|
|
if (!index->amhasgettuple)
|
|
|
|
|
return NIL;
|
|
|
|
|
break;
|
|
|
|
|
case ST_BITMAPSCAN:
|
|
|
|
|
if (!index->amhasgetbitmap)
|
|
|
|
|
return NIL;
|
|
|
|
|
break;
|
|
|
|
|
case ST_ANYSCAN:
|
|
|
|
|
/* either or both are OK */
|
|
|
|
|
break;
|
|
|
|
|
}
|
2006-05-18 19:12:10 +02:00
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/*
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* 1. Combine the per-column IndexClause lists into an overall list.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* In the resulting list, clauses are ordered by index key, so that the
|
|
|
|
|
* column numbers form a nondecreasing sequence. (This order is depended
|
|
|
|
|
* on by btree and possibly other places.) The list can be empty, if the
|
|
|
|
|
* index AM allows that.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
|
|
|
|
* We also build a Relids set showing which outer rels are required by the
|
2012-08-27 04:48:55 +02:00
|
|
|
* selected clauses. Any lateral_relids are included in that, but not
|
|
|
|
|
* otherwise accounted for.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
index_clauses = NIL;
|
2012-08-27 04:48:55 +02:00
|
|
|
outer_relids = bms_copy(rel->lateral_relids);
|
2019-02-13 00:38:32 +01:00
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
ListCell *lc;
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, clauses->indexclauses[indexcol])
|
|
|
|
|
{
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(lc);
|
|
|
|
|
RestrictInfo *rinfo = iclause->rinfo;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
if (skip_nonnative_saop && !index->amsearcharray &&
|
|
|
|
|
IsA(rinfo->clause, ScalarArrayOpExpr))
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
/*
|
|
|
|
|
* Caller asked us to generate IndexPaths that omit any
|
|
|
|
|
* ScalarArrayOpExpr clauses when the underlying index AM
|
|
|
|
|
* lacks native support.
|
|
|
|
|
*
|
|
|
|
|
* We must omit this clause (and tell caller about it).
|
|
|
|
|
*/
|
|
|
|
|
*skip_nonnative_saop = true;
|
|
|
|
|
continue;
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
|
|
|
|
|
/* OK to include this clause */
|
|
|
|
|
index_clauses = lappend(index_clauses, iclause);
|
2012-01-28 01:26:38 +01:00
|
|
|
outer_relids = bms_add_members(outer_relids,
|
|
|
|
|
rinfo->clause_relids);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If no clauses match the first index column, check for amoptionalkey
|
|
|
|
|
* restriction. We can't generate a scan over an index with
|
|
|
|
|
* amoptionalkey = false unless there's at least one index clause.
|
|
|
|
|
* (When working on columns after the first, this test cannot fail. It
|
|
|
|
|
* is always okay for columns after the first to not have any
|
|
|
|
|
* clauses.)
|
|
|
|
|
*/
|
|
|
|
|
if (index_clauses == NIL && !index->amoptionalkey)
|
|
|
|
|
return NIL;
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* We do not want the index's rel itself listed in outer_relids */
|
|
|
|
|
outer_relids = bms_del_member(outer_relids, rel->relid);
|
|
|
|
|
|
|
|
|
|
/* Compute loop_count for cost estimation purposes */
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
loop_count = get_loop_count(root, rel->relid, outer_relids);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/*
|
|
|
|
|
* 2. Compute pathkeys describing index's ordering, if any, then see how
|
|
|
|
|
* many of them are actually useful for this query. This is not relevant
|
Enhance nbtree ScalarArrayOp execution.
Commit 9e8da0f7 taught nbtree to handle ScalarArrayOpExpr quals
natively. This works by pushing down the full context (the array keys)
to the nbtree index AM, enabling it to execute multiple primitive index
scans that the planner treats as one continuous index scan/index path.
This earlier enhancement enabled nbtree ScalarArrayOp index-only scans.
It also allowed scans with ScalarArrayOp quals to return ordered results
(with some notable restrictions, described further down).
Take this general approach a lot further: teach nbtree SAOP index scans
to decide how to execute ScalarArrayOp scans (when and where to start
the next primitive index scan) based on physical index characteristics.
This can be far more efficient. All SAOP scans will now reliably avoid
duplicative leaf page accesses (just like any other nbtree index scan).
SAOP scans whose array keys are naturally clustered together now require
far fewer index descents, since we'll reliably avoid starting a new
primitive scan just to get to a later offset from the same leaf page.
The scan's arrays now advance using binary searches for the array
element that best matches the next tuple's attribute value. Required
scan key arrays (i.e. arrays from scan keys that can terminate the scan)
ratchet forward in lockstep with the index scan. Non-required arrays
(i.e. arrays from scan keys that can only exclude non-matching tuples)
"advance" without the process ever rolling over to a higher-order array.
Naturally, only required SAOP scan keys trigger skipping over leaf pages
(non-required arrays cannot safely end or start primitive index scans).
Consequently, even index scans of a composite index with a high-order
inequality scan key (which we'll mark required) and a low-order SAOP
scan key (which we won't mark required) now avoid repeating leaf page
accesses -- that benefit isn't limited to simpler equality-only cases.
In general, all nbtree index scans now output tuples as if they were one
continuous index scan -- even scans that mix a high-order inequality
with lower-order SAOP equalities reliably output tuples in index order.
This allows us to remove a couple of special cases that were applied
when building index paths with SAOP clauses during planning.
Bugfix commit 807a40c5 taught the planner to avoid generating unsafe
path keys: path keys on a multicolumn index path, with a SAOP clause on
any attribute beyond the first/most significant attribute. These cases
are now all safe, so we go back to generating path keys without regard
for the presence of SAOP clauses (just like with any other clause type).
Affected queries can now exploit scan output order in all the usual ways
(e.g., certain "ORDER BY ... LIMIT n" queries can now terminate early).
Also undo changes from follow-up bugfix commit a4523c5a, which taught
the planner to produce alternative index paths, with path keys, but
without low-order SAOP index quals (filter quals were used instead).
We'll no longer generate these alternative paths, since they can no
longer offer any meaningful advantages over standard index qual paths.
Affected queries thereby avoid all of the disadvantages that come from
using filter quals within index scan nodes. They can avoid extra heap
page accesses from using filter quals to exclude non-matching tuples
(index quals will never have that problem). They can also skip over
irrelevant sections of the index in more cases (though only when nbtree
determines that starting another primitive scan actually makes sense).
There is a theoretical risk that removing restrictions on SAOP index
paths from the planner will break compatibility with amcanorder-based
index AMs maintained as extensions. Such an index AM could have the
same limitations around ordered SAOP scans as nbtree had up until now.
Adding a pro forma incompatibility item about the issue to the Postgres
17 release notes seems like a good idea.
Author: Peter Geoghegan <pg@bowt.ie>
Author: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Heikki Linnakangas <hlinnaka@iki.fi>
Reviewed-By: Matthias van de Meent <boekewurm+postgres@gmail.com>
Reviewed-By: Tomas Vondra <tomas.vondra@enterprisedb.com>
Discussion: https://postgr.es/m/CAH2-Wz=ksvN_sjcnD1+Bt-WtifRA5ok48aDYnq3pkKhxgMQpcw@mail.gmail.com
2024-04-06 17:47:10 +02:00
|
|
|
* if we are only trying to build bitmap indexscans.
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
|
|
|
|
pathkeys_possibly_useful = (scantype != ST_BITMAPSCAN &&
|
|
|
|
|
has_useful_pathkeys(root, rel));
|
|
|
|
|
index_is_ordered = (index->sortopfamily != NULL);
|
|
|
|
|
if (index_is_ordered && pathkeys_possibly_useful)
|
|
|
|
|
{
|
|
|
|
|
index_pathkeys = build_index_pathkeys(root, index,
|
|
|
|
|
ForwardScanDirection);
|
|
|
|
|
useful_pathkeys = truncate_useless_pathkeys(root, rel,
|
|
|
|
|
index_pathkeys);
|
|
|
|
|
orderbyclauses = NIL;
|
|
|
|
|
orderbyclausecols = NIL;
|
|
|
|
|
}
|
|
|
|
|
else if (index->amcanorderbyop && pathkeys_possibly_useful)
|
|
|
|
|
{
|
2023-07-04 13:08:52 +02:00
|
|
|
/*
|
|
|
|
|
* See if we can generate ordering operators for query_pathkeys or at
|
|
|
|
|
* least some prefix thereof. Matching to just a prefix of the
|
|
|
|
|
* query_pathkeys will allow an incremental sort to be considered on
|
|
|
|
|
* the index's partially sorted results.
|
|
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
match_pathkeys_to_index(index, root->query_pathkeys,
|
|
|
|
|
&orderbyclauses,
|
|
|
|
|
&orderbyclausecols);
|
2023-07-04 13:08:52 +02:00
|
|
|
if (list_length(root->query_pathkeys) == list_length(orderbyclauses))
|
2012-01-28 01:26:38 +01:00
|
|
|
useful_pathkeys = root->query_pathkeys;
|
|
|
|
|
else
|
2023-07-04 13:08:52 +02:00
|
|
|
useful_pathkeys = list_copy_head(root->query_pathkeys,
|
|
|
|
|
list_length(orderbyclauses));
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
useful_pathkeys = NIL;
|
|
|
|
|
orderbyclauses = NIL;
|
|
|
|
|
orderbyclausecols = NIL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* 3. Check if an index-only scan is possible. If we're not building
|
|
|
|
|
* plain indexscans, this isn't relevant since bitmap scans don't support
|
|
|
|
|
* index data retrieval anyway.
|
|
|
|
|
*/
|
|
|
|
|
index_only_scan = (scantype != ST_BITMAPSCAN &&
|
|
|
|
|
check_index_only(rel, index));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* 4. Generate an indexscan path if there are relevant restriction clauses
|
|
|
|
|
* in the current clauses, OR the index ordering is potentially useful for
|
|
|
|
|
* later merging or final output ordering, OR the index has a useful
|
|
|
|
|
* predicate, OR an index-only scan is possible.
|
|
|
|
|
*/
|
2014-10-26 21:12:22 +01:00
|
|
|
if (index_clauses != NIL || useful_pathkeys != NIL || useful_predicate ||
|
2012-01-28 01:26:38 +01:00
|
|
|
index_only_scan)
|
|
|
|
|
{
|
|
|
|
|
ipath = create_index_path(root, index,
|
|
|
|
|
index_clauses,
|
|
|
|
|
orderbyclauses,
|
|
|
|
|
orderbyclausecols,
|
|
|
|
|
useful_pathkeys,
|
2023-01-31 22:52:41 +01:00
|
|
|
ForwardScanDirection,
|
2012-01-28 01:26:38 +01:00
|
|
|
index_only_scan,
|
|
|
|
|
outer_relids,
|
2017-02-15 19:53:24 +01:00
|
|
|
loop_count,
|
|
|
|
|
false);
|
2012-01-28 01:26:38 +01:00
|
|
|
result = lappend(result, ipath);
|
2017-02-15 19:53:24 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If appropriate, consider parallel index scan. We don't allow
|
2017-02-19 11:23:59 +01:00
|
|
|
* parallel index scan for bitmap index scans.
|
2017-02-15 19:53:24 +01:00
|
|
|
*/
|
2017-02-19 11:23:59 +01:00
|
|
|
if (index->amcanparallel &&
|
2017-02-15 19:53:24 +01:00
|
|
|
rel->consider_parallel && outer_relids == NULL &&
|
|
|
|
|
scantype != ST_BITMAPSCAN)
|
|
|
|
|
{
|
|
|
|
|
ipath = create_index_path(root, index,
|
|
|
|
|
index_clauses,
|
|
|
|
|
orderbyclauses,
|
|
|
|
|
orderbyclausecols,
|
|
|
|
|
useful_pathkeys,
|
2023-01-31 22:52:41 +01:00
|
|
|
ForwardScanDirection,
|
2017-02-15 19:53:24 +01:00
|
|
|
index_only_scan,
|
|
|
|
|
outer_relids,
|
|
|
|
|
loop_count,
|
|
|
|
|
true);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* if, after costing the path, we find that it's not worth using
|
|
|
|
|
* parallel workers, just free it.
|
|
|
|
|
*/
|
|
|
|
|
if (ipath->path.parallel_workers > 0)
|
|
|
|
|
add_partial_path(rel, (Path *) ipath);
|
|
|
|
|
else
|
|
|
|
|
pfree(ipath);
|
|
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* 5. If the index is ordered, a backwards scan might be interesting.
|
|
|
|
|
*/
|
|
|
|
|
if (index_is_ordered && pathkeys_possibly_useful)
|
|
|
|
|
{
|
|
|
|
|
index_pathkeys = build_index_pathkeys(root, index,
|
|
|
|
|
BackwardScanDirection);
|
|
|
|
|
useful_pathkeys = truncate_useless_pathkeys(root, rel,
|
|
|
|
|
index_pathkeys);
|
|
|
|
|
if (useful_pathkeys != NIL)
|
|
|
|
|
{
|
|
|
|
|
ipath = create_index_path(root, index,
|
|
|
|
|
index_clauses,
|
|
|
|
|
NIL,
|
|
|
|
|
NIL,
|
|
|
|
|
useful_pathkeys,
|
|
|
|
|
BackwardScanDirection,
|
|
|
|
|
index_only_scan,
|
|
|
|
|
outer_relids,
|
2017-02-15 19:53:24 +01:00
|
|
|
loop_count,
|
|
|
|
|
false);
|
2012-01-28 01:26:38 +01:00
|
|
|
result = lappend(result, ipath);
|
2017-02-15 19:53:24 +01:00
|
|
|
|
|
|
|
|
/* If appropriate, consider parallel index scan */
|
2017-02-19 11:23:59 +01:00
|
|
|
if (index->amcanparallel &&
|
2017-02-15 19:53:24 +01:00
|
|
|
rel->consider_parallel && outer_relids == NULL &&
|
|
|
|
|
scantype != ST_BITMAPSCAN)
|
|
|
|
|
{
|
|
|
|
|
ipath = create_index_path(root, index,
|
|
|
|
|
index_clauses,
|
|
|
|
|
NIL,
|
|
|
|
|
NIL,
|
|
|
|
|
useful_pathkeys,
|
|
|
|
|
BackwardScanDirection,
|
|
|
|
|
index_only_scan,
|
|
|
|
|
outer_relids,
|
|
|
|
|
loop_count,
|
|
|
|
|
true);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* if, after costing the path, we find that it's not worth
|
|
|
|
|
* using parallel workers, just free it.
|
|
|
|
|
*/
|
|
|
|
|
if (ipath->path.parallel_workers > 0)
|
|
|
|
|
add_partial_path(rel, (Path *) ipath);
|
|
|
|
|
else
|
|
|
|
|
pfree(ipath);
|
|
|
|
|
}
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* build_paths_for_OR
|
|
|
|
|
* Given a list of restriction clauses from one arm of an OR clause,
|
|
|
|
|
* construct all matching IndexPaths for the relation.
|
|
|
|
|
*
|
|
|
|
|
* Here we must scan all indexes of the relation, since a bitmap OR tree
|
|
|
|
|
* can use multiple indexes.
|
2005-04-22 23:58:32 +02:00
|
|
|
*
|
|
|
|
|
* The caller actually supplies two lists of restriction clauses: some
|
2012-01-28 01:26:38 +01:00
|
|
|
* "current" ones and some "other" ones. Both lists can be used freely
|
2005-04-22 23:58:32 +02:00
|
|
|
* to match keys of the index, but an index must use at least one of the
|
|
|
|
|
* "current" clauses to be considered usable. The motivation for this is
|
|
|
|
|
* examples like
|
|
|
|
|
* WHERE (x = 42) AND (... OR (y = 52 AND z = 77) OR ....)
|
|
|
|
|
* While we are considering the y/z subclause of the OR, we can use "x = 42"
|
|
|
|
|
* as one of the available index conditions; but we shouldn't match the
|
|
|
|
|
* subclause to any index on x alone, because such a Path would already have
|
|
|
|
|
* been generated at the upper level. So we could use an index on x,y,z
|
|
|
|
|
* or an index on x,y for the OR subclause, but not an index on just x.
|
2005-07-28 22:26:22 +02:00
|
|
|
* When dealing with a partial index, a match of the index predicate to
|
|
|
|
|
* one of the "current" clauses also makes the index usable.
|
2005-04-22 23:58:32 +02:00
|
|
|
*
|
|
|
|
|
* 'rel' is the relation for which we want to generate index paths
|
|
|
|
|
* 'clauses' is the current list of clauses (RestrictInfo nodes)
|
2012-01-28 01:26:38 +01:00
|
|
|
* 'other_clauses' is the list of additional upper-level clauses
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
|
|
|
|
static List *
|
2012-01-28 01:26:38 +01:00
|
|
|
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
List *clauses, List *other_clauses)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
|
|
|
|
List *result = NIL;
|
|
|
|
|
List *all_clauses = NIL; /* not computed till needed */
|
2012-01-28 01:26:38 +01:00
|
|
|
ListCell *lc;
|
1998-08-14 18:13:07 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, rel->indexlist)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
|
|
|
|
|
IndexClauseSet clauseset;
|
|
|
|
|
List *indexpaths;
|
2005-07-28 22:26:22 +02:00
|
|
|
bool useful_predicate;
|
1998-08-14 18:13:07 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* Ignore index if it doesn't support bitmap scans */
|
|
|
|
|
if (!index->amhasgetbitmap)
|
2011-10-16 21:39:24 +02:00
|
|
|
continue;
|
|
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/*
|
|
|
|
|
* Ignore partial indexes that do not match the query. If a partial
|
2012-01-28 01:26:38 +01:00
|
|
|
* index is marked predOK then we know it's OK. Otherwise, we have to
|
|
|
|
|
* test whether the added clauses are sufficient to imply the
|
|
|
|
|
* predicate. If so, we can use the index in the current context.
|
2005-07-28 22:26:22 +02:00
|
|
|
*
|
|
|
|
|
* We set useful_predicate to true iff the predicate was proven using
|
|
|
|
|
* the current set of clauses. This is needed to prevent matching a
|
|
|
|
|
* predOK index to an arm of an OR, which would be a legal but
|
|
|
|
|
* pointlessly inefficient plan. (A better plan will be generated by
|
|
|
|
|
* just scanning the predOK index alone, no OR.)
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2005-07-28 22:26:22 +02:00
|
|
|
useful_predicate = false;
|
|
|
|
|
if (index->indpred != NIL)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
2005-07-28 22:26:22 +02:00
|
|
|
if (index->predOK)
|
|
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
/* Usable, but don't set useful_predicate */
|
2005-07-28 22:26:22 +02:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* Form all_clauses if not done already */
|
|
|
|
|
if (all_clauses == NIL)
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
all_clauses = list_concat_copy(clauses, other_clauses);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2017-06-14 19:13:11 +02:00
|
|
|
if (!predicate_implied_by(index->indpred, all_clauses, false))
|
2005-07-28 22:26:22 +02:00
|
|
|
continue; /* can't use it at all */
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2017-06-14 19:13:11 +02:00
|
|
|
if (!predicate_implied_by(index->indpred, other_clauses, false))
|
2005-07-28 22:26:22 +02:00
|
|
|
useful_predicate = true;
|
|
|
|
|
}
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
1998-08-14 18:13:07 +02:00
|
|
|
|
1998-08-02 00:12:13 +02:00
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Identify the restriction clauses that can match the index.
|
1998-08-02 00:12:13 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
MemSet(&clauseset, 0, sizeof(clauseset));
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clauses_to_index(root, clauses, index, &clauseset);
|
2005-07-28 22:26:22 +02:00
|
|
|
|
|
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* If no matches so far, and the index predicate isn't useful, we
|
|
|
|
|
* don't want it.
|
2005-07-28 22:26:22 +02:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
if (!clauseset.nonempty && !useful_predicate)
|
2005-07-28 22:26:22 +02:00
|
|
|
continue;
|
1999-07-30 06:07:25 +02:00
|
|
|
|
1998-08-02 00:12:13 +02:00
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Add "other" restriction clauses to the clauseset.
|
1999-08-16 04:17:58 +02:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clauses_to_index(root, other_clauses, index, &clauseset);
|
2000-12-14 23:30:45 +01:00
|
|
|
|
|
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Construct paths if possible.
|
2000-12-14 23:30:45 +01:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
indexpaths = build_index_paths(root, rel,
|
|
|
|
|
index, &clauseset,
|
|
|
|
|
useful_predicate,
|
2014-10-26 21:12:22 +01:00
|
|
|
ST_BITMAPSCAN,
|
|
|
|
|
NULL);
|
2012-01-28 01:26:38 +01:00
|
|
|
result = list_concat(result, indexpaths);
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* generate_bitmap_or_paths
|
2006-05-18 19:12:10 +02:00
|
|
|
* Look through the list of clauses to find OR clauses, and generate
|
|
|
|
|
* a BitmapOrPath for each one we can handle that way. Return a list
|
|
|
|
|
* of the generated BitmapOrPaths.
|
2005-04-22 23:58:32 +02:00
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* other_clauses is a list of additional clauses that can be assumed true
|
2005-04-22 23:58:32 +02:00
|
|
|
* for the purpose of generating indexquals, but are not to be searched for
|
2012-01-28 01:26:38 +01:00
|
|
|
* ORs. (See build_paths_for_OR() for motivation.)
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2013-12-30 18:50:31 +01:00
|
|
|
static List *
|
2005-06-06 00:32:58 +02:00
|
|
|
generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
|
2013-12-30 18:50:31 +01:00
|
|
|
List *clauses, List *other_clauses)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
|
|
|
|
List *result = NIL;
|
|
|
|
|
List *all_clauses;
|
2012-01-28 01:26:38 +01:00
|
|
|
ListCell *lc;
|
2005-04-22 23:58:32 +02:00
|
|
|
|
|
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* We can use both the current and other clauses as context for
|
|
|
|
|
* build_paths_for_OR; no need to remove ORs from the lists.
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
all_clauses = list_concat_copy(clauses, other_clauses);
|
2005-04-22 23:58:32 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, clauses)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
Improve castNode notation by introducing list-extraction-specific variants.
This extends the castNode() notation introduced by commit 5bcab1114 to
provide, in one step, extraction of a list cell's pointer and coercion to
a concrete node type. For example, "lfirst_node(Foo, lc)" is the same
as "castNode(Foo, lfirst(lc))". Almost half of the uses of castNode
that have appeared so far include a list extraction call, so this is
pretty widely useful, and it saves a few more keystrokes compared to the
old way.
As with the previous patch, back-patch the addition of these macros to
pg_list.h, so that the notation will be available when back-patching.
Patch by me, after an idea of Andrew Gierth's.
Discussion: https://postgr.es/m/14197.1491841216@sss.pgh.pa.us
2017-04-10 19:51:29 +02:00
|
|
|
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
|
2005-04-22 23:58:32 +02:00
|
|
|
List *pathlist;
|
|
|
|
|
Path *bitmapqual;
|
|
|
|
|
ListCell *j;
|
|
|
|
|
|
2006-05-18 19:12:10 +02:00
|
|
|
/* Ignore RestrictInfos that aren't ORs */
|
2005-04-22 23:58:32 +02:00
|
|
|
if (!restriction_is_or_clause(rinfo))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We must be able to match at least one index to each of the arms of
|
|
|
|
|
* the OR, else we can't use it.
|
|
|
|
|
*/
|
|
|
|
|
pathlist = NIL;
|
|
|
|
|
foreach(j, ((BoolExpr *) rinfo->orclause)->args)
|
|
|
|
|
{
|
|
|
|
|
Node *orarg = (Node *) lfirst(j);
|
|
|
|
|
List *indlist;
|
|
|
|
|
|
|
|
|
|
/* OR arguments should be ANDs or sub-RestrictInfos */
|
2019-01-29 21:26:44 +01:00
|
|
|
if (is_andclause(orarg))
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
|
|
|
|
List *andargs = ((BoolExpr *) orarg)->args;
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
indlist = build_paths_for_OR(root, rel,
|
|
|
|
|
andargs,
|
|
|
|
|
all_clauses);
|
|
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/* Recurse in case there are sub-ORs */
|
|
|
|
|
indlist = list_concat(indlist,
|
|
|
|
|
generate_bitmap_or_paths(root, rel,
|
|
|
|
|
andargs,
|
2013-12-30 18:50:31 +01:00
|
|
|
all_clauses));
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
2022-10-05 10:01:41 +02:00
|
|
|
RestrictInfo *ri = castNode(RestrictInfo, orarg);
|
2012-01-28 01:26:38 +01:00
|
|
|
List *orargs;
|
|
|
|
|
|
2022-10-05 10:01:41 +02:00
|
|
|
Assert(!restriction_is_or_clause(ri));
|
|
|
|
|
orargs = list_make1(ri);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
indlist = build_paths_for_OR(root, rel,
|
|
|
|
|
orargs,
|
|
|
|
|
all_clauses);
|
2005-04-22 23:58:32 +02:00
|
|
|
}
|
2005-10-15 04:49:52 +02:00
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/*
|
|
|
|
|
* If nothing matched this arm, we can't do anything with this OR
|
|
|
|
|
* clause.
|
|
|
|
|
*/
|
|
|
|
|
if (indlist == NIL)
|
|
|
|
|
{
|
|
|
|
|
pathlist = NIL;
|
|
|
|
|
break;
|
|
|
|
|
}
|
2005-10-15 04:49:52 +02:00
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/*
|
|
|
|
|
* OK, pick the most promising AND combination, and add it to
|
|
|
|
|
* pathlist.
|
|
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
bitmapqual = choose_bitmap_and(root, rel, indlist);
|
2005-04-22 23:58:32 +02:00
|
|
|
pathlist = lappend(pathlist, bitmapqual);
|
|
|
|
|
}
|
2005-10-15 04:49:52 +02:00
|
|
|
|
1999-08-16 04:17:58 +02:00
|
|
|
/*
|
2005-04-22 23:58:32 +02:00
|
|
|
* If we have a match for every arm, then turn them into a
|
|
|
|
|
* BitmapOrPath, and add to result list.
|
1998-08-02 00:12:13 +02:00
|
|
|
*/
|
2005-04-22 23:58:32 +02:00
|
|
|
if (pathlist != NIL)
|
|
|
|
|
{
|
|
|
|
|
bitmapqual = (Path *) create_bitmap_or_path(root, rel, pathlist);
|
|
|
|
|
result = lappend(result, bitmapqual);
|
|
|
|
|
}
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
2002-11-24 22:52:15 +01:00
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* choose_bitmap_and
|
|
|
|
|
* Given a nonempty list of bitmap paths, AND them into one path.
|
|
|
|
|
*
|
|
|
|
|
* This is a nontrivial decision since we can legally use any subset of the
|
|
|
|
|
* given path set. We want to choose a good tradeoff between selectivity
|
|
|
|
|
* and cost of computing the bitmap.
|
|
|
|
|
*
|
|
|
|
|
* The result is either a single one of the inputs, or a BitmapAndPath
|
|
|
|
|
* combining multiple inputs.
|
|
|
|
|
*/
|
|
|
|
|
static Path *
|
2012-01-28 01:26:38 +01:00
|
|
|
choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel, List *paths)
|
2005-04-22 23:58:32 +02:00
|
|
|
{
|
2005-04-23 03:57:34 +02:00
|
|
|
int npaths = list_length(paths);
|
2007-04-17 22:03:03 +02:00
|
|
|
PathClauseUsage **pathinfoarray;
|
|
|
|
|
PathClauseUsage *pathinfo;
|
|
|
|
|
List *clauselist;
|
|
|
|
|
List *bestpaths = NIL;
|
|
|
|
|
Cost bestcost = 0;
|
|
|
|
|
int i,
|
|
|
|
|
j;
|
2005-04-23 03:57:34 +02:00
|
|
|
ListCell *l;
|
|
|
|
|
|
|
|
|
|
Assert(npaths > 0); /* else caller error */
|
|
|
|
|
if (npaths == 1)
|
|
|
|
|
return (Path *) linitial(paths); /* easy case */
|
|
|
|
|
|
2005-04-22 23:58:32 +02:00
|
|
|
/*
|
2005-04-23 03:57:34 +02:00
|
|
|
* In theory we should consider every nonempty subset of the given paths.
|
|
|
|
|
* In practice that seems like overkill, given the crude nature of the
|
|
|
|
|
* estimates, not to mention the possible effects of higher-level AND and
|
2007-04-17 22:03:03 +02:00
|
|
|
* OR clauses. Moreover, it's completely impractical if there are a large
|
|
|
|
|
* number of paths, since the work would grow as O(2^N).
|
2005-04-25 05:58:30 +02:00
|
|
|
*
|
2007-04-17 22:03:03 +02:00
|
|
|
* As a heuristic, we first check for paths using exactly the same sets of
|
|
|
|
|
* WHERE clauses + index predicate conditions, and reject all but the
|
|
|
|
|
* cheapest-to-scan in any such group. This primarily gets rid of indexes
|
|
|
|
|
* that include the interesting columns but also irrelevant columns. (In
|
|
|
|
|
* situations where the DBA has gone overboard on creating variant
|
|
|
|
|
* indexes, this can make for a very large reduction in the number of
|
|
|
|
|
* paths considered further.)
|
2005-11-30 18:10:19 +01:00
|
|
|
*
|
2007-04-17 22:03:03 +02:00
|
|
|
* We then sort the surviving paths with the cheapest-to-scan first, and
|
|
|
|
|
* for each path, consider using that path alone as the basis for a bitmap
|
|
|
|
|
* scan. Then we consider bitmap AND scans formed from that path plus
|
|
|
|
|
* each subsequent (higher-cost) path, adding on a subsequent path if it
|
|
|
|
|
* results in a reduction in the estimated total scan cost. This means we
|
|
|
|
|
* consider about O(N^2) rather than O(2^N) path combinations, which is
|
|
|
|
|
* quite tolerable, especially given than N is usually reasonably small
|
|
|
|
|
* because of the prefiltering step. The cheapest of these is returned.
|
2007-03-21 23:18:12 +01:00
|
|
|
*
|
2007-04-17 22:03:03 +02:00
|
|
|
* We will only consider AND combinations in which no two indexes use the
|
|
|
|
|
* same WHERE clause. This is a bit of a kluge: it's needed because
|
|
|
|
|
* costsize.c and clausesel.c aren't very smart about redundant clauses.
|
|
|
|
|
* They will usually double-count the redundant clauses, producing a
|
|
|
|
|
* too-small selectivity that makes a redundant AND step look like it
|
|
|
|
|
* reduces the total cost. Perhaps someday that code will be smarter and
|
|
|
|
|
* we can remove this limitation. (But note that this also defends
|
|
|
|
|
* against flat-out duplicate input paths, which can happen because
|
2012-01-28 01:26:38 +01:00
|
|
|
* match_join_clauses_to_index will find the same OR join clauses that
|
Extract restriction OR clauses whether or not they are indexable.
It's possible to extract a restriction OR clause from a join clause that
has the form of an OR-of-ANDs, if each sub-AND includes a clause that
mentions only one specific relation. While PG has been aware of that idea
for many years, the code previously only did it if it could extract an
indexable OR clause. On reflection, though, that seems a silly limitation:
adding a restriction clause can be a win by reducing the number of rows
that have to be filtered at the join step, even if we have to test the
clause as a plain filter clause during the scan. This should be especially
useful for foreign tables, where the change can cut the number of rows that
have to be retrieved from the foreign server; but testing shows it can win
even on local tables. Per a suggestion from Robert Haas.
As a heuristic, I made the code accept an extracted restriction clause
if its estimated selectivity is less than 0.9, which will probably result
in accepting extracted clauses just about always. We might need to tweak
that later based on experience.
Since the code no longer has even a weak connection to Path creation,
remove orindxpath.c and create a new file optimizer/util/orclauses.c.
There's some additional janitorial cleanup of now-dead code that needs
to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
|
|
|
* extract_restriction_or_clauses has pulled OR restriction clauses out
|
|
|
|
|
* of.)
|
2005-08-29 00:47:20 +02:00
|
|
|
*
|
2007-04-17 22:03:03 +02:00
|
|
|
* For the same reason, we reject AND combinations in which an index
|
|
|
|
|
* predicate clause duplicates another clause. Here we find it necessary
|
|
|
|
|
* to be even stricter: we'll reject a partial index if any of its
|
|
|
|
|
* predicate clauses are implied by the set of WHERE clauses and predicate
|
|
|
|
|
* clauses used so far. This covers cases such as a condition "x = 42"
|
|
|
|
|
* used with a plain index, followed by a clauseless scan of a partial
|
|
|
|
|
* index "WHERE x >= 40 AND x < 50". The partial index has been accepted
|
|
|
|
|
* only because "x = 42" was present, and so allowing it would partially
|
|
|
|
|
* double-count selectivity. (We could use predicate_implied_by on
|
|
|
|
|
* regular qual clauses too, to have a more intelligent, but much more
|
|
|
|
|
* expensive, check for redundancy --- but in most cases simple equality
|
|
|
|
|
* seems to suffice.)
|
2005-04-22 23:58:32 +02:00
|
|
|
*/
|
2005-04-23 03:57:34 +02:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/*
|
|
|
|
|
* Extract clause usage info and detect any paths that use exactly the
|
|
|
|
|
* same set of clauses; keep only the cheapest-to-scan of any such groups.
|
|
|
|
|
* The surviving paths are put into an array for qsort'ing.
|
|
|
|
|
*/
|
|
|
|
|
pathinfoarray = (PathClauseUsage **)
|
|
|
|
|
palloc(npaths * sizeof(PathClauseUsage *));
|
|
|
|
|
clauselist = NIL;
|
|
|
|
|
npaths = 0;
|
2005-04-23 03:57:34 +02:00
|
|
|
foreach(l, paths)
|
|
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
Path *ipath = (Path *) lfirst(l);
|
|
|
|
|
|
|
|
|
|
pathinfo = classify_index_clause_usage(ipath, &clauselist);
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
|
|
|
|
|
/* If it's unclassifiable, treat it as distinct from all others */
|
|
|
|
|
if (pathinfo->unclassifiable)
|
|
|
|
|
{
|
|
|
|
|
pathinfoarray[npaths++] = pathinfo;
|
|
|
|
|
continue;
|
|
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
for (i = 0; i < npaths; i++)
|
|
|
|
|
{
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
if (!pathinfoarray[i]->unclassifiable &&
|
|
|
|
|
bms_equal(pathinfo->clauseids, pathinfoarray[i]->clauseids))
|
2007-04-17 22:03:03 +02:00
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
if (i < npaths)
|
|
|
|
|
{
|
|
|
|
|
/* duplicate clauseids, keep the cheaper one */
|
|
|
|
|
Cost ncost;
|
|
|
|
|
Cost ocost;
|
|
|
|
|
Selectivity nselec;
|
|
|
|
|
Selectivity oselec;
|
|
|
|
|
|
|
|
|
|
cost_bitmap_tree_node(pathinfo->path, &ncost, &nselec);
|
|
|
|
|
cost_bitmap_tree_node(pathinfoarray[i]->path, &ocost, &oselec);
|
|
|
|
|
if (ncost < ocost)
|
|
|
|
|
pathinfoarray[i] = pathinfo;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* not duplicate clauseids, add to array */
|
|
|
|
|
pathinfoarray[npaths++] = pathinfo;
|
|
|
|
|
}
|
2005-04-23 03:57:34 +02:00
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/* If only one surviving path, we're done */
|
|
|
|
|
if (npaths == 1)
|
|
|
|
|
return pathinfoarray[0]->path;
|
|
|
|
|
|
|
|
|
|
/* Sort the surviving paths by index access cost */
|
|
|
|
|
qsort(pathinfoarray, npaths, sizeof(PathClauseUsage *),
|
|
|
|
|
path_usage_comparator);
|
2005-04-23 03:57:34 +02:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/*
|
|
|
|
|
* For each surviving index, consider it as an "AND group leader", and see
|
|
|
|
|
* whether adding on any of the later indexes results in an AND path with
|
|
|
|
|
* cheaper total cost than before. Then take the cheapest AND group.
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
*
|
|
|
|
|
* Note: paths that are either clauseless or unclassifiable will have
|
|
|
|
|
* empty clauseids, so that they will not be rejected by the clauseids
|
|
|
|
|
* filter here, nor will they cause later paths to be rejected by it.
|
2007-04-17 22:03:03 +02:00
|
|
|
*/
|
|
|
|
|
for (i = 0; i < npaths; i++)
|
2005-04-23 03:57:34 +02:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
Cost costsofar;
|
|
|
|
|
List *qualsofar;
|
|
|
|
|
Bitmapset *clauseidsofar;
|
|
|
|
|
|
|
|
|
|
pathinfo = pathinfoarray[i];
|
|
|
|
|
paths = list_make1(pathinfo->path);
|
2012-01-28 01:26:38 +01:00
|
|
|
costsofar = bitmap_scan_cost_est(root, rel, pathinfo->path);
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
qualsofar = list_concat_copy(pathinfo->quals, pathinfo->preds);
|
2007-04-17 22:03:03 +02:00
|
|
|
clauseidsofar = bms_copy(pathinfo->clauseids);
|
|
|
|
|
|
|
|
|
|
for (j = i + 1; j < npaths; j++)
|
2007-03-21 23:18:12 +01:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
Cost newcost;
|
2007-03-21 23:18:12 +01:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
pathinfo = pathinfoarray[j];
|
|
|
|
|
/* Check for redundancy */
|
|
|
|
|
if (bms_overlap(pathinfo->clauseids, clauseidsofar))
|
|
|
|
|
continue; /* consider it redundant */
|
|
|
|
|
if (pathinfo->preds)
|
2007-03-21 23:18:12 +01:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
bool redundant = false;
|
2007-03-21 23:18:12 +01:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/* we check each predicate clause separately */
|
|
|
|
|
foreach(l, pathinfo->preds)
|
2007-03-21 23:18:12 +01:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
Node *np = (Node *) lfirst(l);
|
|
|
|
|
|
2017-06-14 19:13:11 +02:00
|
|
|
if (predicate_implied_by(list_make1(np), qualsofar, false))
|
2007-04-17 22:03:03 +02:00
|
|
|
{
|
|
|
|
|
redundant = true;
|
|
|
|
|
break; /* out of inner foreach loop */
|
|
|
|
|
}
|
2007-03-21 23:18:12 +01:00
|
|
|
}
|
2007-04-17 22:03:03 +02:00
|
|
|
if (redundant)
|
|
|
|
|
continue;
|
2007-03-21 23:18:12 +01:00
|
|
|
}
|
2007-04-17 22:03:03 +02:00
|
|
|
/* tentatively add new path to paths, so we can estimate cost */
|
|
|
|
|
paths = lappend(paths, pathinfo->path);
|
2012-01-28 01:26:38 +01:00
|
|
|
newcost = bitmap_and_cost_est(root, rel, paths);
|
2007-04-17 22:03:03 +02:00
|
|
|
if (newcost < costsofar)
|
|
|
|
|
{
|
|
|
|
|
/* keep new path in paths, update subsidiary variables */
|
|
|
|
|
costsofar = newcost;
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
qualsofar = list_concat(qualsofar, pathinfo->quals);
|
|
|
|
|
qualsofar = list_concat(qualsofar, pathinfo->preds);
|
2007-04-17 22:03:03 +02:00
|
|
|
clauseidsofar = bms_add_members(clauseidsofar,
|
|
|
|
|
pathinfo->clauseids);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/* reject new path, remove it from paths list */
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
paths = list_truncate(paths, list_length(paths) - 1);
|
2007-04-17 22:03:03 +02:00
|
|
|
}
|
2005-04-23 03:57:34 +02:00
|
|
|
}
|
2007-04-17 22:03:03 +02:00
|
|
|
|
|
|
|
|
/* Keep the cheapest AND-group (or singleton) */
|
|
|
|
|
if (i == 0 || costsofar < bestcost)
|
2005-04-23 03:57:34 +02:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
bestpaths = paths;
|
|
|
|
|
bestcost = costsofar;
|
2005-04-23 03:57:34 +02:00
|
|
|
}
|
2007-04-17 22:03:03 +02:00
|
|
|
|
|
|
|
|
/* some easy cleanup (we don't try real hard though) */
|
|
|
|
|
list_free(qualsofar);
|
2005-04-23 03:57:34 +02:00
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
if (list_length(bestpaths) == 1)
|
|
|
|
|
return (Path *) linitial(bestpaths); /* no need for AND */
|
|
|
|
|
return (Path *) create_bitmap_and_path(root, rel, bestpaths);
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/* qsort comparator to sort in increasing index access cost order */
|
2005-04-23 03:57:34 +02:00
|
|
|
static int
|
2007-04-17 22:03:03 +02:00
|
|
|
path_usage_comparator(const void *a, const void *b)
|
2005-04-23 03:57:34 +02:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
PathClauseUsage *pa = *(PathClauseUsage *const *) a;
|
|
|
|
|
PathClauseUsage *pb = *(PathClauseUsage *const *) b;
|
2005-04-23 03:57:34 +02:00
|
|
|
Cost acost;
|
|
|
|
|
Cost bcost;
|
|
|
|
|
Selectivity aselec;
|
|
|
|
|
Selectivity bselec;
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
cost_bitmap_tree_node(pa->path, &acost, &aselec);
|
|
|
|
|
cost_bitmap_tree_node(pb->path, &bcost, &bselec);
|
2005-04-23 03:57:34 +02:00
|
|
|
|
2005-11-30 18:10:19 +01:00
|
|
|
/*
|
2007-04-17 22:03:03 +02:00
|
|
|
* If costs are the same, sort by selectivity.
|
2005-11-30 18:10:19 +01:00
|
|
|
*/
|
2007-04-17 22:03:03 +02:00
|
|
|
if (acost < bcost)
|
2005-04-23 03:57:34 +02:00
|
|
|
return -1;
|
2007-04-17 22:03:03 +02:00
|
|
|
if (acost > bcost)
|
2005-04-23 03:57:34 +02:00
|
|
|
return 1;
|
2005-11-30 18:10:19 +01:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
if (aselec < bselec)
|
2005-04-23 03:57:34 +02:00
|
|
|
return -1;
|
2007-04-17 22:03:03 +02:00
|
|
|
if (aselec > bselec)
|
2005-04-23 03:57:34 +02:00
|
|
|
return 1;
|
2005-11-30 18:10:19 +01:00
|
|
|
|
2005-04-23 03:57:34 +02:00
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
2007-04-17 22:03:03 +02:00
|
|
|
* Estimate the cost of actually executing a bitmap scan with a single
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
* index path (which could be a BitmapAnd or BitmapOr node).
|
2007-04-17 22:03:03 +02:00
|
|
|
*/
|
|
|
|
|
static Cost
|
2012-01-28 01:26:38 +01:00
|
|
|
bitmap_scan_cost_est(PlannerInfo *root, RelOptInfo *rel, Path *ipath)
|
2007-04-17 22:03:03 +02:00
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
BitmapHeapPath bpath;
|
Revise parameterized-path mechanism to fix assorted issues.
This patch adjusts the treatment of parameterized paths so that all paths
with the same parameterization (same set of required outer rels) for the
same relation will have the same rowcount estimate. We cache the rowcount
estimates to ensure that property, and hopefully save a few cycles too.
Doing this makes it practical for add_path_precheck to operate without
a rowcount estimate: it need only assume that paths with different
parameterizations never dominate each other, which is close enough to
true anyway for coarse filtering, because normally a more-parameterized
path should yield fewer rows thanks to having more join clauses to apply.
In add_path, we do the full nine yards of comparing rowcount estimates
along with everything else, so that we can discard parameterized paths that
don't actually have an advantage. This fixes some issues I'd found with
add_path rejecting parameterized paths on the grounds that they were more
expensive than not-parameterized ones, even though they yielded many fewer
rows and hence would be cheaper once subsequent joining was considered.
To make the same-rowcounts assumption valid, we have to require that any
parameterized path enforce *all* join clauses that could be obtained from
the particular set of outer rels, even if not all of them are useful for
indexing. This is required at both base scans and joins. It's a good
thing anyway since the net impact is that join quals are checked at the
lowest practical level in the join tree. Hence, discard the original
rather ad-hoc mechanism for choosing parameterization joinquals, and build
a better one that has a more principled rule for when clauses can be moved.
The original rule was actually buggy anyway for lack of knowledge about
which relations are part of an outer join's outer side; getting this right
requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* Set up a dummy BitmapHeapPath */
|
|
|
|
|
bpath.path.type = T_BitmapHeapPath;
|
|
|
|
|
bpath.path.pathtype = T_BitmapHeapScan;
|
|
|
|
|
bpath.path.parent = rel;
|
2016-03-14 21:59:59 +01:00
|
|
|
bpath.path.pathtarget = rel->reltarget;
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
bpath.path.param_info = ipath->param_info;
|
2012-01-28 01:26:38 +01:00
|
|
|
bpath.path.pathkeys = NIL;
|
|
|
|
|
bpath.bitmapqual = ipath;
|
2007-04-17 22:03:03 +02:00
|
|
|
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
/*
|
|
|
|
|
* Check the cost of temporary path without considering parallelism.
|
|
|
|
|
* Parallel bitmap heap path will be considered at later stage.
|
|
|
|
|
*/
|
|
|
|
|
bpath.path.parallel_workers = 0;
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
|
|
|
|
|
/* Now we can do cost_bitmap_heap_scan */
|
Revise parameterized-path mechanism to fix assorted issues.
This patch adjusts the treatment of parameterized paths so that all paths
with the same parameterization (same set of required outer rels) for the
same relation will have the same rowcount estimate. We cache the rowcount
estimates to ensure that property, and hopefully save a few cycles too.
Doing this makes it practical for add_path_precheck to operate without
a rowcount estimate: it need only assume that paths with different
parameterizations never dominate each other, which is close enough to
true anyway for coarse filtering, because normally a more-parameterized
path should yield fewer rows thanks to having more join clauses to apply.
In add_path, we do the full nine yards of comparing rowcount estimates
along with everything else, so that we can discard parameterized paths that
don't actually have an advantage. This fixes some issues I'd found with
add_path rejecting parameterized paths on the grounds that they were more
expensive than not-parameterized ones, even though they yielded many fewer
rows and hence would be cheaper once subsequent joining was considered.
To make the same-rowcounts assumption valid, we have to require that any
parameterized path enforce *all* join clauses that could be obtained from
the particular set of outer rels, even if not all of them are useful for
indexing. This is required at both base scans and joins. It's a good
thing anyway since the net impact is that join quals are checked at the
lowest practical level in the join tree. Hence, discard the original
rather ad-hoc mechanism for choosing parameterization joinquals, and build
a better one that has a more principled rule for when clauses can be moved.
The original rule was actually buggy anyway for lack of knowledge about
which relations are part of an outer join's outer side; getting this right
requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
|
|
|
cost_bitmap_heap_scan(&bpath.path, root, rel,
|
|
|
|
|
bpath.path.param_info,
|
|
|
|
|
ipath,
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
get_loop_count(root, rel->relid,
|
|
|
|
|
PATH_REQ_OUTER(ipath)));
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
return bpath.path.total_cost;
|
2007-04-17 22:03:03 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Estimate the cost of actually executing a BitmapAnd scan with the given
|
2005-04-23 03:57:34 +02:00
|
|
|
* inputs.
|
|
|
|
|
*/
|
|
|
|
|
static Cost
|
2012-01-28 01:26:38 +01:00
|
|
|
bitmap_and_cost_est(PlannerInfo *root, RelOptInfo *rel, List *paths)
|
2005-04-23 03:57:34 +02:00
|
|
|
{
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
BitmapAndPath *apath;
|
2005-04-23 03:57:34 +02:00
|
|
|
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
/*
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
* Might as well build a real BitmapAndPath here, as the work is slightly
|
|
|
|
|
* too complicated to be worth repeating just to save one palloc.
|
Support parallel bitmap heap scans.
The index is scanned by a single process, but then all cooperating
processes can iterate jointly over the resulting set of heap blocks.
In the future, we might also want to support using a parallel bitmap
index scan to set up for a parallel bitmap heap scan, but that's a
job for another day.
Dilip Kumar, with some corrections and cosmetic changes by me. The
larger patch set of which this is a part has been reviewed and tested
by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia
Sabih, Haribabu Kommi, Thomas Munro, and me.
Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
|
|
|
*/
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
apath = create_bitmap_and_path(root, rel, paths);
|
2005-04-23 03:57:34 +02:00
|
|
|
|
Fix bitmap AND/OR scans on the inside of a nestloop partition-wise join.
reparameterize_path_by_child() failed to reparameterize BitmapAnd
and BitmapOr paths. This matters only if such a path is chosen as
the inside of a nestloop partition-wise join, where we have to pass
in parameters from the outside of the nestloop. If that did happen,
we generated a bad plan that would likely lead to crashes at execution.
This is not entirely reparameterize_path_by_child()'s fault though;
it's the victim of an ancient decision (my ancient decision, I think)
to not bother filling in param_info in BitmapAnd/Or path nodes. That
caused the function to believe that such nodes and their children
contain no parameter references and so need not be processed.
In hindsight that decision looks pretty penny-wise and pound-foolish:
while it saves a few cycles during path node setup, we do commonly
need the information later. In particular, by reversing the decision
and requiring valid param_info data in all nodes of a bitmap path
tree, we can get rid of indxpath.c's get_bitmap_tree_required_outer()
function, which computed the data on-demand. It's not unlikely that
that nets out as a savings of cycles in many scenarios. A couple
of other things in indxpath.c can be simplified as well.
While here, get rid of some cases in reparameterize_path_by_child()
that are visibly dead or useless, given that we only care about
reparameterizing paths that can be on the inside of a parameterized
nestloop. This case reminds one of the maxim that untested code
probably does not work, so I'm unwilling to leave unreachable code
in this function. (I did leave the T_Gather case in place even
though it's not reached in the regression tests. It's not very
clear to me when the planner might prefer to put Gather below
rather than above a nestloop, but at least in principle the case
might be interesting.)
Per bug #16536, originally from Arne Roland but with a test case
by Andrew Gierth. Back-patch to v11 where this code came in.
Discussion: https://postgr.es/m/16536-2213ee0b3aad41fd@postgresql.org
2020-07-15 00:56:49 +02:00
|
|
|
return bitmap_scan_cost_est(root, rel, (Path *) apath);
|
2005-04-23 03:57:34 +02:00
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* classify_index_clause_usage
|
|
|
|
|
* Construct a PathClauseUsage struct describing the WHERE clauses and
|
|
|
|
|
* index predicate clauses used by the given indexscan path.
|
|
|
|
|
* We consider two clauses the same if they are equal().
|
|
|
|
|
*
|
|
|
|
|
* At some point we might want to migrate this info into the Path data
|
|
|
|
|
* structure proper, but for the moment it's only needed within
|
|
|
|
|
* choose_bitmap_and().
|
|
|
|
|
*
|
|
|
|
|
* *clauselist is used and expanded as needed to identify all the distinct
|
|
|
|
|
* clauses seen across successive calls. Caller must initialize it to NIL
|
|
|
|
|
* before first call of a set.
|
|
|
|
|
*/
|
|
|
|
|
static PathClauseUsage *
|
|
|
|
|
classify_index_clause_usage(Path *path, List **clauselist)
|
|
|
|
|
{
|
|
|
|
|
PathClauseUsage *result;
|
|
|
|
|
Bitmapset *clauseids;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
result = (PathClauseUsage *) palloc(sizeof(PathClauseUsage));
|
|
|
|
|
result->path = path;
|
|
|
|
|
|
|
|
|
|
/* Recursively find the quals and preds used by the path */
|
|
|
|
|
result->quals = NIL;
|
|
|
|
|
result->preds = NIL;
|
|
|
|
|
find_indexpath_quals(path, &result->quals, &result->preds);
|
|
|
|
|
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
/*
|
|
|
|
|
* Some machine-generated queries have outlandish numbers of qual clauses.
|
|
|
|
|
* To avoid getting into O(N^2) behavior even in this preliminary
|
|
|
|
|
* classification step, we want to limit the number of entries we can
|
|
|
|
|
* accumulate in *clauselist. Treat any path with more than 100 quals +
|
|
|
|
|
* preds as unclassifiable, which will cause calling code to consider it
|
|
|
|
|
* distinct from all other paths.
|
|
|
|
|
*/
|
|
|
|
|
if (list_length(result->quals) + list_length(result->preds) > 100)
|
|
|
|
|
{
|
|
|
|
|
result->clauseids = NULL;
|
|
|
|
|
result->unclassifiable = true;
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
/* Build up a bitmapset representing the quals and preds */
|
|
|
|
|
clauseids = NULL;
|
|
|
|
|
foreach(lc, result->quals)
|
|
|
|
|
{
|
|
|
|
|
Node *node = (Node *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
clauseids = bms_add_member(clauseids,
|
|
|
|
|
find_list_position(node, clauselist));
|
|
|
|
|
}
|
|
|
|
|
foreach(lc, result->preds)
|
|
|
|
|
{
|
|
|
|
|
Node *node = (Node *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
clauseids = bms_add_member(clauseids,
|
|
|
|
|
find_list_position(node, clauselist));
|
|
|
|
|
}
|
|
|
|
|
result->clauseids = clauseids;
|
Limit the number of index clauses considered in choose_bitmap_and().
classify_index_clause_usage() is O(N^2) in the number of distinct index
qual clauses it considers, because of its use of a simple search list to
store them. For nearly all queries, that's fine because only a few clauses
will be considered. But Alexander Kuzmenkov reported a machine-generated
query with 80000 (!) index qual clauses, which caused this code to take
forever. Somewhat remarkably, this is the only O(N^2) behavior we now
have for such a query, so let's fix it.
We can get rid of the O(N^2) runtime for cases like this without much
damage to the functionality of choose_bitmap_and() by separating out
paths with "too many" qual or pred clauses, and deeming them to always
be nonredundant with other paths. Then their clauses needn't go into
the search list, so it doesn't get too long, but we don't lose the
ability to consider bitmap AND plans altogether. I set the threshold
for "too many" to be 100 clauses per path, which should be plenty to
ensure no change in planning behavior for normal queries.
There are other things we could do to make this go faster, but it's not
clear that it's worth any additional effort. 80000 qual clauses require
a whole lot of work in many other places, too.
The code's been like this for a long time, so back-patch to all supported
branches. The troublesome query only works back to 9.5 (in 9.4 it fails
with stack overflow in the parser); so I'm not sure that fixing this in
9.4 has any real-world benefit, but perhaps it does.
Discussion: https://postgr.es/m/90c5bdfa-d633-dabe-9889-3cf3e1acd443@postgrespro.ru
2018-11-12 17:19:04 +01:00
|
|
|
result->unclassifiable = false;
|
2007-04-17 22:03:03 +02:00
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
2006-04-09 20:18:41 +02:00
|
|
|
/*
|
2007-03-21 23:18:12 +01:00
|
|
|
* find_indexpath_quals
|
2006-04-09 20:18:41 +02:00
|
|
|
*
|
2007-03-21 23:18:12 +01:00
|
|
|
* Given the Path structure for a plain or bitmap indexscan, extract lists
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* of all the index clauses and index predicate conditions used in the Path.
|
2007-04-17 22:03:03 +02:00
|
|
|
* These are appended to the initial contents of *quals and *preds (hence
|
|
|
|
|
* caller should initialize those to NIL).
|
2006-04-09 20:18:41 +02:00
|
|
|
*
|
2013-12-30 18:50:31 +01:00
|
|
|
* Note we are not trying to produce an accurate representation of the AND/OR
|
2006-04-09 20:18:41 +02:00
|
|
|
* semantics of the Path, but just find out all the base conditions used.
|
|
|
|
|
*
|
2007-03-21 23:18:12 +01:00
|
|
|
* The result lists contain pointers to the expressions used in the Path,
|
2006-04-09 20:18:41 +02:00
|
|
|
* but all the list cells are freshly built, so it's safe to destructively
|
2007-03-21 23:18:12 +01:00
|
|
|
* modify the lists (eg, by concat'ing with other lists).
|
2006-04-09 20:18:41 +02:00
|
|
|
*/
|
2007-03-21 23:18:12 +01:00
|
|
|
static void
|
|
|
|
|
find_indexpath_quals(Path *bitmapqual, List **quals, List **preds)
|
2006-04-09 20:18:41 +02:00
|
|
|
{
|
|
|
|
|
if (IsA(bitmapqual, BitmapAndPath))
|
|
|
|
|
{
|
|
|
|
|
BitmapAndPath *apath = (BitmapAndPath *) bitmapqual;
|
2007-04-17 22:03:03 +02:00
|
|
|
ListCell *l;
|
2006-04-09 20:18:41 +02:00
|
|
|
|
|
|
|
|
foreach(l, apath->bitmapquals)
|
|
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
find_indexpath_quals((Path *) lfirst(l), quals, preds);
|
2006-04-09 20:18:41 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else if (IsA(bitmapqual, BitmapOrPath))
|
|
|
|
|
{
|
|
|
|
|
BitmapOrPath *opath = (BitmapOrPath *) bitmapqual;
|
2007-04-17 22:03:03 +02:00
|
|
|
ListCell *l;
|
2006-04-09 20:18:41 +02:00
|
|
|
|
|
|
|
|
foreach(l, opath->bitmapquals)
|
|
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
find_indexpath_quals((Path *) lfirst(l), quals, preds);
|
2006-04-09 20:18:41 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
else if (IsA(bitmapqual, IndexPath))
|
|
|
|
|
{
|
|
|
|
|
IndexPath *ipath = (IndexPath *) bitmapqual;
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
ListCell *l;
|
2006-04-09 20:18:41 +02:00
|
|
|
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
foreach(l, ipath->indexclauses)
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause = (IndexClause *) lfirst(l);
|
|
|
|
|
|
|
|
|
|
*quals = lappend(*quals, iclause->rinfo->clause);
|
|
|
|
|
}
|
Rationalize use of list_concat + list_copy combinations.
In the wake of commit 1cff1b95a, the result of list_concat no longer
shares the ListCells of the second input. Therefore, we can replace
"list_concat(x, list_copy(y))" with just "list_concat(x, y)".
To improve call sites that were list_copy'ing the first argument,
or both arguments, invent "list_concat_copy()" which produces a new
list sharing no ListCells with either input. (This is a bit faster
than "list_concat(list_copy(x), y)" because it makes the result list
the right size to start with.)
In call sites that were not list_copy'ing the second argument, the new
semantics mean that we are usually leaking the second List's storage,
since typically there is no remaining pointer to it. We considered
inventing another list_copy variant that would list_free the second
input, but concluded that for most call sites it isn't worth worrying
about, given the relative compactness of the new List representation.
(Note that in cases where such leakage would happen, the old code
already leaked the second List's header; so we're only discussing
the size of the leak not whether there is one. I did adjust two or
three places that had been troubling to free that header so that
they manually free the whole second List.)
Patch by me; thanks to David Rowley for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-08-12 17:20:18 +02:00
|
|
|
*preds = list_concat(*preds, ipath->indexinfo->indpred);
|
2006-04-09 20:18:41 +02:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
elog(ERROR, "unrecognized node type: %d", nodeTag(bitmapqual));
|
|
|
|
|
}
|
|
|
|
|
|
1996-07-09 08:22:35 +02:00
|
|
|
|
2005-11-30 18:10:19 +01:00
|
|
|
/*
|
2007-04-17 22:03:03 +02:00
|
|
|
* find_list_position
|
|
|
|
|
* Return the given node's position (counting from 0) in the given
|
|
|
|
|
* list of nodes. If it's not equal() to any existing list member,
|
|
|
|
|
* add it at the end, and return that position.
|
2005-11-30 18:10:19 +01:00
|
|
|
*/
|
2007-04-17 22:03:03 +02:00
|
|
|
static int
|
|
|
|
|
find_list_position(Node *node, List **nodelist)
|
2005-11-30 18:10:19 +01:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
int i;
|
|
|
|
|
ListCell *lc;
|
2005-11-30 18:10:19 +01:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
i = 0;
|
|
|
|
|
foreach(lc, *nodelist)
|
2005-11-30 18:10:19 +01:00
|
|
|
{
|
2007-04-17 22:03:03 +02:00
|
|
|
Node *oldnode = (Node *) lfirst(lc);
|
2005-11-30 18:10:19 +01:00
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
if (equal(node, oldnode))
|
|
|
|
|
return i;
|
|
|
|
|
i++;
|
2005-11-30 18:10:19 +01:00
|
|
|
}
|
|
|
|
|
|
2007-04-17 22:03:03 +02:00
|
|
|
*nodelist = lappend(*nodelist, node);
|
|
|
|
|
|
|
|
|
|
return i;
|
2005-11-30 18:10:19 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
2011-10-08 02:13:02 +02:00
|
|
|
/*
|
|
|
|
|
* check_index_only
|
|
|
|
|
* Determine whether an index-only scan is possible for this index.
|
|
|
|
|
*/
|
|
|
|
|
static bool
|
|
|
|
|
check_index_only(RelOptInfo *rel, IndexOptInfo *index)
|
|
|
|
|
{
|
|
|
|
|
bool result;
|
|
|
|
|
Bitmapset *attrs_used = NULL;
|
2015-03-26 18:12:00 +01:00
|
|
|
Bitmapset *index_canreturn_attrs = NULL;
|
2011-10-08 02:13:02 +02:00
|
|
|
ListCell *lc;
|
|
|
|
|
int i;
|
|
|
|
|
|
2015-03-26 18:12:00 +01:00
|
|
|
/* Index-only scans must be enabled */
|
2011-10-08 02:13:02 +02:00
|
|
|
if (!enable_indexonlyscan)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Check that all needed attributes of the relation are available from the
|
|
|
|
|
* index.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
/*
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
* First, identify all the attributes needed for joins or final output.
|
|
|
|
|
* Note: we must look at rel's targetlist, not the attr_needed data,
|
|
|
|
|
* because attr_needed isn't computed for inheritance child rels.
|
2011-10-08 02:13:02 +02:00
|
|
|
*/
|
2016-03-14 21:59:59 +01:00
|
|
|
pull_varattnos((Node *) rel->reltarget->exprs, rel->relid, &attrs_used);
|
2011-10-08 02:13:02 +02:00
|
|
|
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
/*
|
|
|
|
|
* Add all the attributes used by restriction clauses; but consider only
|
|
|
|
|
* those clauses not implied by the index predicate, since ones that are
|
|
|
|
|
* so implied don't need to be checked explicitly in the plan.
|
|
|
|
|
*
|
|
|
|
|
* Note: attributes used only in index quals would not be needed at
|
|
|
|
|
* runtime either, if we are certain that the index is not lossy. However
|
|
|
|
|
* it'd be complicated to account for that accurately, and it doesn't
|
|
|
|
|
* matter in most cases, since we'd conclude that such attributes are
|
|
|
|
|
* available from the index anyway.
|
|
|
|
|
*/
|
|
|
|
|
foreach(lc, index->indrestrictinfo)
|
2011-10-08 02:13:02 +02:00
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
pull_varattnos((Node *) rinfo->clause, rel->relid, &attrs_used);
|
|
|
|
|
}
|
|
|
|
|
|
2015-03-26 18:12:00 +01:00
|
|
|
/*
|
|
|
|
|
* Construct a bitmapset of columns that the index can return back in an
|
2022-01-03 22:12:11 +01:00
|
|
|
* index-only scan.
|
2015-03-26 18:12:00 +01:00
|
|
|
*/
|
2011-10-08 02:13:02 +02:00
|
|
|
for (i = 0; i < index->ncolumns; i++)
|
|
|
|
|
{
|
|
|
|
|
int attno = index->indexkeys[i];
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* For the moment, we just ignore index expressions. It might be nice
|
2011-10-11 20:20:06 +02:00
|
|
|
* to do something with them, later.
|
2011-10-08 02:13:02 +02:00
|
|
|
*/
|
2011-10-11 20:20:06 +02:00
|
|
|
if (attno == 0)
|
2011-10-08 02:13:02 +02:00
|
|
|
continue;
|
|
|
|
|
|
2015-03-26 18:12:00 +01:00
|
|
|
if (index->canreturn[i])
|
|
|
|
|
index_canreturn_attrs =
|
|
|
|
|
bms_add_member(index_canreturn_attrs,
|
|
|
|
|
attno - FirstLowInvalidHeapAttributeNumber);
|
2011-10-08 02:13:02 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Do we have all the necessary attributes? */
|
2015-03-26 18:12:00 +01:00
|
|
|
result = bms_is_subset(attrs_used, index_canreturn_attrs);
|
2011-10-08 02:13:02 +02:00
|
|
|
|
|
|
|
|
bms_free(attrs_used);
|
2015-03-26 18:12:00 +01:00
|
|
|
bms_free(index_canreturn_attrs);
|
2011-10-08 02:13:02 +02:00
|
|
|
|
|
|
|
|
return result;
|
|
|
|
|
}
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/*
|
|
|
|
|
* get_loop_count
|
|
|
|
|
* Choose the loop count estimate to use for costing a parameterized path
|
|
|
|
|
* with the given set of outer relids.
|
|
|
|
|
*
|
|
|
|
|
* Since we produce parameterized paths before we've begun to generate join
|
|
|
|
|
* relations, it's impossible to predict exactly how many times a parameterized
|
|
|
|
|
* path will be iterated; we don't know the size of the relation that will be
|
|
|
|
|
* on the outside of the nestloop. However, we should try to account for
|
|
|
|
|
* multiple iterations somehow in costing the path. The heuristic embodied
|
|
|
|
|
* here is to use the rowcount of the smallest other base relation needed in
|
|
|
|
|
* the join clauses used by the path. (We could alternatively consider the
|
|
|
|
|
* largest one, but that seems too optimistic.) This is of course the right
|
|
|
|
|
* answer for single-other-relation cases, and it seems like a reasonable
|
|
|
|
|
* zero-order approximation for multiway-join cases.
|
|
|
|
|
*
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
* In addition, we check to see if the other side of each join clause is on
|
|
|
|
|
* the inside of some semijoin that the current relation is on the outside of.
|
|
|
|
|
* If so, the only way that a parameterized path could be used is if the
|
|
|
|
|
* semijoin RHS has been unique-ified, so we should use the number of unique
|
|
|
|
|
* RHS rows rather than using the relation's raw rowcount.
|
|
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Note: for this to work, allpaths.c must establish all baserel size
|
|
|
|
|
* estimates before it begins to compute paths, or at least before it
|
|
|
|
|
* calls create_index_paths().
|
|
|
|
|
*/
|
|
|
|
|
static double
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
get_loop_count(PlannerInfo *root, Index cur_relid, Relids outer_relids)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
double result;
|
|
|
|
|
int outer_relid;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/* For a non-parameterized path, just return 1.0 quickly */
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
if (outer_relids == NULL)
|
|
|
|
|
return 1.0;
|
|
|
|
|
|
2015-03-12 03:53:32 +01:00
|
|
|
result = 0.0;
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
outer_relid = -1;
|
|
|
|
|
while ((outer_relid = bms_next_member(outer_relids, outer_relid)) >= 0)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
RelOptInfo *outer_rel;
|
|
|
|
|
double rowcount;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/* Paranoia: ignore bogus relid indexes */
|
|
|
|
|
if (outer_relid >= root->simple_rel_array_size)
|
|
|
|
|
continue;
|
|
|
|
|
outer_rel = root->simple_rel_array[outer_relid];
|
|
|
|
|
if (outer_rel == NULL)
|
|
|
|
|
continue;
|
|
|
|
|
Assert(outer_rel->relid == outer_relid); /* sanity check on array */
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/* Other relation could be proven empty, if so ignore */
|
|
|
|
|
if (IS_DUMMY_REL(outer_rel))
|
|
|
|
|
continue;
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/* Otherwise, rel's rows estimate should be valid by now */
|
|
|
|
|
Assert(outer_rel->rows > 0);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/* Check to see if rel is on the inside of any semijoins */
|
|
|
|
|
rowcount = adjust_rowcount_for_semijoins(root,
|
|
|
|
|
cur_relid,
|
|
|
|
|
outer_relid,
|
|
|
|
|
outer_rel->rows);
|
2012-01-28 01:26:38 +01:00
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/* Remember smallest row count estimate among the outer rels */
|
2015-03-12 03:53:32 +01:00
|
|
|
if (result == 0.0 || result > rowcount)
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
result = rowcount;
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
2015-03-12 03:53:32 +01:00
|
|
|
/* Return 1.0 if we found no valid relations (shouldn't happen) */
|
|
|
|
|
return (result > 0.0) ? result : 1.0;
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
/*
|
|
|
|
|
* Check to see if outer_relid is on the inside of any semijoin that cur_relid
|
|
|
|
|
* is on the outside of. If so, replace rowcount with the estimated number of
|
|
|
|
|
* unique rows from the semijoin RHS (assuming that's smaller, which it might
|
|
|
|
|
* not be). The estimate is crude but it's the best we can do at this stage
|
|
|
|
|
* of the proceedings.
|
|
|
|
|
*/
|
|
|
|
|
static double
|
|
|
|
|
adjust_rowcount_for_semijoins(PlannerInfo *root,
|
|
|
|
|
Index cur_relid,
|
|
|
|
|
Index outer_relid,
|
|
|
|
|
double rowcount)
|
|
|
|
|
{
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
foreach(lc, root->join_info_list)
|
|
|
|
|
{
|
|
|
|
|
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
if (sjinfo->jointype == JOIN_SEMI &&
|
|
|
|
|
bms_is_member(cur_relid, sjinfo->syn_lefthand) &&
|
|
|
|
|
bms_is_member(outer_relid, sjinfo->syn_righthand))
|
|
|
|
|
{
|
|
|
|
|
/* Estimate number of unique-ified rows */
|
|
|
|
|
double nraw;
|
|
|
|
|
double nunique;
|
|
|
|
|
|
|
|
|
|
nraw = approximate_joinrel_size(root, sjinfo->syn_righthand);
|
|
|
|
|
nunique = estimate_num_groups(root,
|
|
|
|
|
sjinfo->semi_rhs_exprs,
|
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
|
|
|
nraw,
|
2021-03-30 09:52:46 +02:00
|
|
|
NULL,
|
Support GROUPING SETS, CUBE and ROLLUP.
This SQL standard functionality allows to aggregate data by different
GROUP BY clauses at once. Each grouping set returns rows with columns
grouped by in other sets set to NULL.
This could previously be achieved by doing each grouping as a separate
query, conjoined by UNION ALLs. Besides being considerably more concise,
grouping sets will in many cases be faster, requiring only one scan over
the underlying data.
The current implementation of grouping sets only supports using sorting
for input. Individual sets that share a sort order are computed in one
pass. If there are sets that don't share a sort order, additional sort &
aggregation steps are performed. These additional passes are sourced by
the previous sort step; thus avoiding repeated scans of the source data.
The code is structured in a way that adding support for purely using
hash aggregation or a mix of hashing and sorting is possible. Sorting
was chosen to be supported first, as it is the most generic method of
implementation.
Instead of, as in an earlier versions of the patch, representing the
chain of sort and aggregation steps as full blown planner and executor
nodes, all but the first sort are performed inside the aggregation node
itself. This avoids the need to do some unusual gymnastics to handle
having to return aggregated and non-aggregated tuples from underlying
nodes, as well as having to shut down underlying nodes early to limit
memory usage. The optimizer still builds Sort/Agg node to describe each
phase, but they're not part of the plan tree, but instead additional
data for the aggregation node. They're a convenient and preexisting way
to describe aggregation and sorting. The first (and possibly only) sort
step is still performed as a separate execution step. That retains
similarity with existing group by plans, makes rescans fairly simple,
avoids very deep plans (leading to slow explains) and easily allows to
avoid the sorting step if the underlying data is sorted by other means.
A somewhat ugly side of this patch is having to deal with a grammar
ambiguity between the new CUBE keyword and the cube extension/functions
named cube (and rollup). To avoid breaking existing deployments of the
cube extension it has not been renamed, neither has cube been made a
reserved keyword. Instead precedence hacking is used to make GROUP BY
cube(..) refer to the CUBE grouping sets feature, and not the function
cube(). To actually group by a function cube(), unlikely as that might
be, the function name has to be quoted.
Needs a catversion bump because stored rules may change.
Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund
Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas
Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule
Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
|
|
|
NULL);
|
Improve planner's cost estimation in the presence of semijoins.
If we have a semijoin, say
SELECT * FROM x WHERE x1 IN (SELECT y1 FROM y)
and we're estimating the cost of a parameterized indexscan on x, the number
of repetitions of the indexscan should not be taken as the size of y; it'll
really only be the number of distinct values of y1, because the only valid
plan with y on the outside of a nestloop would require y to be unique-ified
before joining it to x. Most of the time this doesn't make that much
difference, but sometimes it can lead to drastically underestimating the
cost of the indexscan and hence choosing a bad plan, as pointed out by
David Kubečka.
Fixing this is a bit difficult because parameterized indexscans are costed
out quite early in the planning process, before we have the information
that would be needed to call estimate_num_groups() and thereby estimate the
number of distinct values of the join column(s). However we can move the
code that extracts a semijoin RHS's unique-ification columns, so that it's
done in initsplan.c rather than on-the-fly in create_unique_path(). That
shouldn't make any difference speed-wise and it's really a bit cleaner too.
The other bit of information we need is the size of the semijoin RHS,
which is easy if it's a single relation (we make those estimates before
considering indexscan costs) but problematic if it's a join relation.
The solution adopted here is just to use the product of the sizes of the
join component rels. That will generally be an overestimate, but since
estimate_num_groups() only uses this input as a clamp, an overestimate
shouldn't hurt us too badly. In any case we don't allow this new logic
to produce a value larger than we would have chosen before, so that at
worst an overestimate leaves us no wiser than we were before.
2015-03-12 02:21:00 +01:00
|
|
|
if (rowcount > nunique)
|
|
|
|
|
rowcount = nunique;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
return rowcount;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Make an approximate estimate of the size of a joinrel.
|
|
|
|
|
*
|
|
|
|
|
* We don't have enough info at this point to get a good estimate, so we
|
|
|
|
|
* just multiply the base relation sizes together. Fortunately, this is
|
|
|
|
|
* the right answer anyway for the most common case with a single relation
|
|
|
|
|
* on the RHS of a semijoin. Also, estimate_num_groups() has only a weak
|
|
|
|
|
* dependency on its input_rows argument (it basically uses it as a clamp).
|
|
|
|
|
* So we might be able to get a fairly decent end result even with a severe
|
|
|
|
|
* overestimate of the RHS's raw size.
|
|
|
|
|
*/
|
|
|
|
|
static double
|
|
|
|
|
approximate_joinrel_size(PlannerInfo *root, Relids relids)
|
|
|
|
|
{
|
|
|
|
|
double rowcount = 1.0;
|
|
|
|
|
int relid;
|
|
|
|
|
|
|
|
|
|
relid = -1;
|
|
|
|
|
while ((relid = bms_next_member(relids, relid)) >= 0)
|
|
|
|
|
{
|
|
|
|
|
RelOptInfo *rel;
|
|
|
|
|
|
|
|
|
|
/* Paranoia: ignore bogus relid indexes */
|
|
|
|
|
if (relid >= root->simple_rel_array_size)
|
|
|
|
|
continue;
|
|
|
|
|
rel = root->simple_rel_array[relid];
|
|
|
|
|
if (rel == NULL)
|
|
|
|
|
continue;
|
|
|
|
|
Assert(rel->relid == relid); /* sanity check on array */
|
|
|
|
|
|
|
|
|
|
/* Relation could be proven empty, if so ignore */
|
|
|
|
|
if (IS_DUMMY_REL(rel))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/* Otherwise, rel's rows estimate should be valid by now */
|
|
|
|
|
Assert(rel->rows > 0);
|
|
|
|
|
|
|
|
|
|
/* Accumulate product */
|
|
|
|
|
rowcount *= rel->rows;
|
|
|
|
|
}
|
|
|
|
|
return rowcount;
|
|
|
|
|
}
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
|
|
|
|
|
/****************************************************************************
|
|
|
|
|
* ---- ROUTINES TO CHECK QUERY CLAUSES ----
|
|
|
|
|
****************************************************************************/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_restriction_clauses_to_index
|
|
|
|
|
* Identify restriction clauses for the rel that match the index.
|
|
|
|
|
* Matching clauses are added to *clauseset.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
2019-02-12 03:26:08 +01:00
|
|
|
match_restriction_clauses_to_index(PlannerInfo *root,
|
|
|
|
|
IndexOptInfo *index,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexClauseSet *clauseset)
|
|
|
|
|
{
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
/* We can ignore clauses that are implied by the index predicate */
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clauses_to_index(root, index->indrestrictinfo, index, clauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_join_clauses_to_index
|
|
|
|
|
* Identify join clauses for the rel that match the index.
|
|
|
|
|
* Matching clauses are added to *clauseset.
|
|
|
|
|
* Also, add any potentially usable join OR clauses to *joinorclauses.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
match_join_clauses_to_index(PlannerInfo *root,
|
|
|
|
|
RelOptInfo *rel, IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *clauseset,
|
|
|
|
|
List **joinorclauses)
|
|
|
|
|
{
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
Revise parameterized-path mechanism to fix assorted issues.
This patch adjusts the treatment of parameterized paths so that all paths
with the same parameterization (same set of required outer rels) for the
same relation will have the same rowcount estimate. We cache the rowcount
estimates to ensure that property, and hopefully save a few cycles too.
Doing this makes it practical for add_path_precheck to operate without
a rowcount estimate: it need only assume that paths with different
parameterizations never dominate each other, which is close enough to
true anyway for coarse filtering, because normally a more-parameterized
path should yield fewer rows thanks to having more join clauses to apply.
In add_path, we do the full nine yards of comparing rowcount estimates
along with everything else, so that we can discard parameterized paths that
don't actually have an advantage. This fixes some issues I'd found with
add_path rejecting parameterized paths on the grounds that they were more
expensive than not-parameterized ones, even though they yielded many fewer
rows and hence would be cheaper once subsequent joining was considered.
To make the same-rowcounts assumption valid, we have to require that any
parameterized path enforce *all* join clauses that could be obtained from
the particular set of outer rels, even if not all of them are useful for
indexing. This is required at both base scans and joins. It's a good
thing anyway since the net impact is that join quals are checked at the
lowest practical level in the join tree. Hence, discard the original
rather ad-hoc mechanism for choosing parameterization joinquals, and build
a better one that has a more principled rule for when clauses can be moved.
The original rule was actually buggy anyway for lack of knowledge about
which relations are part of an outer join's outer side; getting this right
requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
|
|
|
/* Scan the rel's join clauses */
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, rel->joininfo)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
|
Revise parameterized-path mechanism to fix assorted issues.
This patch adjusts the treatment of parameterized paths so that all paths
with the same parameterization (same set of required outer rels) for the
same relation will have the same rowcount estimate. We cache the rowcount
estimates to ensure that property, and hopefully save a few cycles too.
Doing this makes it practical for add_path_precheck to operate without
a rowcount estimate: it need only assume that paths with different
parameterizations never dominate each other, which is close enough to
true anyway for coarse filtering, because normally a more-parameterized
path should yield fewer rows thanks to having more join clauses to apply.
In add_path, we do the full nine yards of comparing rowcount estimates
along with everything else, so that we can discard parameterized paths that
don't actually have an advantage. This fixes some issues I'd found with
add_path rejecting parameterized paths on the grounds that they were more
expensive than not-parameterized ones, even though they yielded many fewer
rows and hence would be cheaper once subsequent joining was considered.
To make the same-rowcounts assumption valid, we have to require that any
parameterized path enforce *all* join clauses that could be obtained from
the particular set of outer rels, even if not all of them are useful for
indexing. This is required at both base scans and joins. It's a good
thing anyway since the net impact is that join quals are checked at the
lowest practical level in the join tree. Hence, discard the original
rather ad-hoc mechanism for choosing parameterization joinquals, and build
a better one that has a more principled rule for when clauses can be moved.
The original rule was actually buggy anyway for lack of knowledge about
which relations are part of an outer join's outer side; getting this right
requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
|
|
|
/* Check if clause can be moved to this rel */
|
2013-08-18 02:22:37 +02:00
|
|
|
if (!join_clause_is_movable_to(rinfo, rel))
|
2012-08-30 20:32:22 +02:00
|
|
|
continue;
|
|
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* Potentially usable, so see if it matches the index or is an OR */
|
|
|
|
|
if (restriction_is_or_clause(rinfo))
|
|
|
|
|
*joinorclauses = lappend(*joinorclauses, rinfo);
|
|
|
|
|
else
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clause_to_index(root, rinfo, index, clauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_eclass_clauses_to_index
|
|
|
|
|
* Identify EquivalenceClass join clauses for the rel that match the index.
|
|
|
|
|
* Matching clauses are added to *clauseset.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
|
|
|
|
match_eclass_clauses_to_index(PlannerInfo *root, IndexOptInfo *index,
|
|
|
|
|
IndexClauseSet *clauseset)
|
|
|
|
|
{
|
|
|
|
|
int indexcol;
|
|
|
|
|
|
|
|
|
|
/* No work if rel is not in any such ECs */
|
|
|
|
|
if (!index->rel->has_eclass_joins)
|
|
|
|
|
return;
|
|
|
|
|
|
2018-04-07 22:00:39 +02:00
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2013-03-22 00:43:59 +01:00
|
|
|
ec_member_matches_arg arg;
|
2012-01-28 01:26:38 +01:00
|
|
|
List *clauses;
|
2011-10-08 02:13:02 +02:00
|
|
|
|
2012-08-30 20:32:22 +02:00
|
|
|
/* Generate clauses, skipping any that join to lateral_referencers */
|
2013-03-22 00:43:59 +01:00
|
|
|
arg.index = index;
|
|
|
|
|
arg.indexcol = indexcol;
|
|
|
|
|
clauses = generate_implied_equalities_for_column(root,
|
|
|
|
|
index->rel,
|
|
|
|
|
ec_member_matches_indexcol,
|
|
|
|
|
(void *) &arg,
|
2013-08-18 02:22:37 +02:00
|
|
|
index->rel->lateral_referencers);
|
1996-07-09 08:22:35 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/*
|
|
|
|
|
* We have to check whether the results actually do match the index,
|
|
|
|
|
* since for non-btree indexes the EC's equality operators might not
|
2013-03-22 00:43:59 +01:00
|
|
|
* be in the index opclass (cf ec_member_matches_indexcol).
|
2012-01-28 01:26:38 +01:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clauses_to_index(root, clauses, index, clauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
1996-07-09 08:22:35 +02:00
|
|
|
|
|
|
|
|
/*
|
2011-12-25 01:03:21 +01:00
|
|
|
* match_clauses_to_index
|
2012-01-28 01:26:38 +01:00
|
|
|
* Perform match_clause_to_index() for each clause in a list.
|
|
|
|
|
* Matching clauses are added to *clauseset.
|
1996-07-09 08:22:35 +02:00
|
|
|
*/
|
2011-12-25 01:03:21 +01:00
|
|
|
static void
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clauses_to_index(PlannerInfo *root,
|
2012-01-28 01:26:38 +01:00
|
|
|
List *clauses,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexClauseSet *clauseset)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2012-01-28 01:26:38 +01:00
|
|
|
ListCell *lc;
|
1996-07-09 08:22:35 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
foreach(lc, clauses)
|
1997-03-18 19:41:37 +01:00
|
|
|
{
|
Improve castNode notation by introducing list-extraction-specific variants.
This extends the castNode() notation introduced by commit 5bcab1114 to
provide, in one step, extraction of a list cell's pointer and coercion to
a concrete node type. For example, "lfirst_node(Foo, lc)" is the same
as "castNode(Foo, lfirst(lc))". Almost half of the uses of castNode
that have appeared so far include a list extraction call, so this is
pretty widely useful, and it saves a few more keystrokes compared to the
old way.
As with the previous patch, back-patch the addition of these macros to
pg_list.h, so that the notation will be available when back-patching.
Patch by me, after an idea of Andrew Gierth's.
Discussion: https://postgr.es/m/14197.1491841216@sss.pgh.pa.us
2017-04-10 19:51:29 +02:00
|
|
|
RestrictInfo *rinfo = lfirst_node(RestrictInfo, lc);
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clause_to_index(root, rinfo, index, clauseset);
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
|
|
|
|
}
|
1999-07-23 05:34:49 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/*
|
|
|
|
|
* match_clause_to_index
|
|
|
|
|
* Test whether a qual clause can be used with an index.
|
|
|
|
|
*
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
* If the clause is usable, add an IndexClause entry for it to the appropriate
|
|
|
|
|
* list in *clauseset. (*clauseset must be initialized to zeroes before first
|
|
|
|
|
* call.)
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
|
|
|
|
* Note: in some circumstances we may find the same RestrictInfos coming from
|
|
|
|
|
* multiple places. Defend against redundant outputs by refusing to add a
|
|
|
|
|
* clause twice (pointer equality should be a good enough check for this).
|
|
|
|
|
*
|
|
|
|
|
* Note: it's possible that a badly-defined index could have multiple matching
|
|
|
|
|
* columns. We always select the first match if so; this avoids scenarios
|
|
|
|
|
* wherein we get an inflated idea of the index's selectivity by using the
|
|
|
|
|
* same clause multiple times with different index columns.
|
|
|
|
|
*/
|
|
|
|
|
static void
|
2019-02-12 03:26:08 +01:00
|
|
|
match_clause_to_index(PlannerInfo *root,
|
2012-01-28 01:26:38 +01:00
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index,
|
2012-01-28 01:26:38 +01:00
|
|
|
IndexClauseSet *clauseset)
|
|
|
|
|
{
|
|
|
|
|
int indexcol;
|
1999-07-23 05:34:49 +02:00
|
|
|
|
Improve RLS planning by marking individual quals with security levels.
In an RLS query, we must ensure that security filter quals are evaluated
before ordinary query quals, in case the latter contain "leaky" functions
that could expose the contents of sensitive rows. The original
implementation of RLS planning ensured this by pushing the scan of a
secured table into a sub-query that it marked as a security-barrier view.
Unfortunately this results in very inefficient plans in many cases, because
the sub-query cannot be flattened and gets planned independently of the
rest of the query.
To fix, drop the use of sub-queries to enforce RLS qual order, and instead
mark each qual (RestrictInfo) with a security_level field establishing its
priority for evaluation. Quals must be evaluated in security_level order,
except that "leakproof" quals can be allowed to go ahead of quals of lower
security_level, if it's helpful to do so. This has to be enforced within
the ordering of any one list of quals to be evaluated at a table scan node,
and we also have to ensure that quals are not chosen for early evaluation
(i.e., use as an index qual or TID scan qual) if they're not allowed to go
ahead of other quals at the scan node.
This is sufficient to fix the problem for RLS quals, since we only support
RLS policies on simple tables and thus RLS quals will always exist at the
table scan level only. Eventually these qual ordering rules should be
enforced for join quals as well, which would permit improving planning for
explicit security-barrier views; but that's a task for another patch.
Note that FDWs would need to be aware of these rules --- and not, for
example, send an insecure qual for remote execution --- but since we do
not yet allow RLS policies on foreign tables, the case doesn't arise.
This will need to be addressed before we can allow such policies.
Patch by me, reviewed by Stephen Frost and Dean Rasheed.
Discussion: https://postgr.es/m/8185.1477432701@sss.pgh.pa.us
2017-01-18 18:58:20 +01:00
|
|
|
/*
|
|
|
|
|
* Never match pseudoconstants to indexes. (Normally a match could not
|
|
|
|
|
* happen anyway, since a pseudoconstant clause couldn't contain a Var,
|
|
|
|
|
* but what if someone builds an expression index on a constant? It's not
|
|
|
|
|
* totally unreasonable to do so with a partial index, either.)
|
|
|
|
|
*/
|
|
|
|
|
if (rinfo->pseudoconstant)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If clause can't be used as an indexqual because it must wait till after
|
|
|
|
|
* some lower-security-level restriction clause, reject it.
|
|
|
|
|
*/
|
|
|
|
|
if (!restriction_is_securely_promotable(rinfo, index->rel))
|
|
|
|
|
return;
|
|
|
|
|
|
2018-04-08 18:58:15 +02:00
|
|
|
/* OK, check each index key column for a match */
|
|
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2012-01-28 01:26:38 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexClause *iclause;
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/* Ignore duplicates */
|
|
|
|
|
foreach(lc, clauseset->indexclauses[indexcol])
|
|
|
|
|
{
|
2022-08-25 16:35:40 +02:00
|
|
|
iclause = (IndexClause *) lfirst(lc);
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
|
|
|
|
|
if (iclause->rinfo == rinfo)
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
/* OK, try to match the clause to the index column */
|
|
|
|
|
iclause = match_clause_to_indexcol(root,
|
|
|
|
|
rinfo,
|
|
|
|
|
indexcol,
|
|
|
|
|
index);
|
|
|
|
|
if (iclause)
|
2003-12-18 01:22:12 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
/* Success, so record it */
|
2012-01-28 01:26:38 +01:00
|
|
|
clauseset->indexclauses[indexcol] =
|
Refactor the representation of indexable clauses in IndexPaths.
In place of three separate but interrelated lists (indexclauses,
indexquals, and indexqualcols), an IndexPath now has one list
"indexclauses" of IndexClause nodes. This holds basically the same
information as before, but in a more useful format: in particular, there
is now a clear connection between an indexclause (an original restriction
clause from WHERE or JOIN/ON) and the indexquals (directly usable index
conditions) derived from it.
We also change the ground rules a bit by mandating that clause commutation,
if needed, be done up-front so that what is stored in the indexquals list
is always directly usable as an index condition. This gets rid of repeated
re-determination of which side of the clause is the indexkey during costing
and plan generation, as well as repeated lookups of the commutator
operator. To minimize the added up-front cost, the typical case of
commuting a plain OpExpr is handled by a new special-purpose function
commute_restrictinfo(). For RowCompareExprs, generating the new clause
properly commuted to begin with is not really any more complex than before,
it's just different --- and we can save doing that work twice, as the
pretty-klugy original implementation did.
Tracking the connection between original and derived clauses lets us
also track explicitly whether the derived clauses are an exact or lossy
translation of the original. This provides a cheap solution to getting
rid of unnecessary rechecks of boolean index clauses, which previously
seemed like it'd be more expensive than it was worth.
Another pleasant (IMO) side-effect is that EXPLAIN now always shows
index clauses with the indexkey on the left; this seems less confusing.
This commit leaves expand_indexqual_conditions() and some related
functions in a slightly messy state. I didn't bother to change them
any more than minimally necessary to work with the new data structure,
because all that code is going to be refactored out of existence in
a follow-on patch.
Discussion: https://postgr.es/m/22182.1549124950@sss.pgh.pa.us
2019-02-09 23:30:43 +01:00
|
|
|
lappend(clauseset->indexclauses[indexcol], iclause);
|
2012-01-28 01:26:38 +01:00
|
|
|
clauseset->nonempty = true;
|
2011-12-25 01:03:21 +01:00
|
|
|
return;
|
2012-01-28 01:26:38 +01:00
|
|
|
}
|
2010-11-20 21:07:16 +01:00
|
|
|
}
|
1997-03-18 19:41:37 +01:00
|
|
|
}
|
|
|
|
|
|
1997-01-22 07:25:42 +01:00
|
|
|
/*
|
2003-05-28 18:04:02 +02:00
|
|
|
* match_clause_to_indexcol()
|
2019-02-12 03:26:08 +01:00
|
|
|
* Determine whether a restriction clause matches a column of an index,
|
|
|
|
|
* and if so, build an IndexClause node describing the details.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2019-02-12 03:26:08 +01:00
|
|
|
* To match an index normally, an operator clause:
|
2000-08-13 04:50:35 +02:00
|
|
|
*
|
2002-11-24 22:52:15 +01:00
|
|
|
* (1) must be in the form (indexkey op const) or (const op indexkey);
|
|
|
|
|
* and
|
2019-02-12 03:26:08 +01:00
|
|
|
* (2) must contain an operator which is in the index's operator family
|
|
|
|
|
* for this column; and
|
2011-04-09 01:19:17 +02:00
|
|
|
* (3) must match the collation of the index, if collation is relevant.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Our definition of "const" is exceedingly liberal: we allow anything that
|
|
|
|
|
* doesn't involve a volatile function or a Var of the index's relation.
|
|
|
|
|
* In particular, Vars belonging to other relations of the query are
|
|
|
|
|
* accepted here, since a clause of that form can be used in a
|
|
|
|
|
* parameterized indexscan. It's the responsibility of higher code levels
|
|
|
|
|
* to manage restriction and join clauses appropriately.
|
2005-04-22 23:58:32 +02:00
|
|
|
*
|
2012-01-28 01:26:38 +01:00
|
|
|
* Note: we do need to check for Vars of the index's relation on the
|
|
|
|
|
* "const" side of the clause, since clauses like (a.f1 OP (b.f2 OP a.f3))
|
|
|
|
|
* are not processable by a parameterized indexscan on a.f1, whereas
|
|
|
|
|
* something like (a.f1 OP (b.f2 OP c.f3)) is.
|
2005-04-22 23:58:32 +02:00
|
|
|
*
|
1999-08-12 06:32:54 +02:00
|
|
|
* Presently, the executor can only deal with indexquals that have the
|
|
|
|
|
* indexkey on the left, so we can only use clauses that have the indexkey
|
|
|
|
|
* on the right if we can commute the clause to put the key on the left.
|
2019-02-12 03:26:08 +01:00
|
|
|
* We handle that by generating an IndexClause with the correctly-commuted
|
|
|
|
|
* opclause as a derived indexqual.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2011-09-29 06:43:42 +02:00
|
|
|
* If the index has a collation, the clause must have the same collation.
|
|
|
|
|
* For collation-less indexes, we assume it doesn't matter; this is
|
|
|
|
|
* necessary for cases like "hstore ? text", wherein hstore's operators
|
|
|
|
|
* don't care about collation but the clause will get marked with a
|
|
|
|
|
* collation anyway because of the text argument. (This logic is
|
|
|
|
|
* embodied in the macro IndexCollMatchesExprColl.)
|
|
|
|
|
*
|
2006-01-25 21:29:24 +01:00
|
|
|
* It is also possible to match RowCompareExpr clauses to indexes (but
|
2019-02-12 03:26:08 +01:00
|
|
|
* currently, only btree indexes handle this).
|
2006-01-25 21:29:24 +01:00
|
|
|
*
|
2005-11-25 20:47:50 +01:00
|
|
|
* It is also possible to match ScalarArrayOpExpr clauses to indexes, when
|
2012-01-28 01:26:38 +01:00
|
|
|
* the clause is of the form "indexkey op ANY (arrayconst)".
|
2005-11-25 20:47:50 +01:00
|
|
|
*
|
2005-03-27 00:29:20 +01:00
|
|
|
* For boolean indexes, it is also possible to match the clause directly
|
|
|
|
|
* to the indexkey; or perhaps the clause is (NOT indexkey).
|
|
|
|
|
*
|
2019-02-12 03:26:08 +01:00
|
|
|
* And, last but not least, some operators and functions can be processed
|
|
|
|
|
* to derive (typically lossy) indexquals from a clause that isn't in
|
|
|
|
|
* itself indexable. If we see that any operand of an OpExpr or FuncExpr
|
|
|
|
|
* matches the index key, and the function has a planner support function
|
|
|
|
|
* attached to it, we'll invoke the support function to see if such an
|
|
|
|
|
* indexqual can be built.
|
|
|
|
|
*
|
2004-01-04 01:07:32 +01:00
|
|
|
* 'rinfo' is the clause to be tested (as a RestrictInfo node).
|
2019-02-12 03:26:08 +01:00
|
|
|
* 'indexcol' is a column number of 'index' (counting from 0).
|
|
|
|
|
* 'index' is the index of interest.
|
1996-07-09 08:22:35 +02:00
|
|
|
*
|
2019-02-12 03:26:08 +01:00
|
|
|
* Returns an IndexClause if the clause can be used with this index key,
|
|
|
|
|
* or NULL if not.
|
1999-07-23 05:34:49 +02:00
|
|
|
*
|
2019-02-12 03:26:08 +01:00
|
|
|
* NOTE: returns NULL if clause is an OR or AND clause; it is the
|
2000-02-05 19:26:09 +01:00
|
|
|
* responsibility of higher-level routines to cope with those.
|
1996-07-09 08:22:35 +02:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
static IndexClause *
|
|
|
|
|
match_clause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2003-05-28 18:04:02 +02:00
|
|
|
int indexcol,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexClause *iclause;
|
2004-01-04 01:07:32 +01:00
|
|
|
Expr *clause = rinfo->clause;
|
2018-04-12 15:37:22 +02:00
|
|
|
Oid opfamily;
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2018-04-12 15:37:22 +02:00
|
|
|
Assert(indexcol < index->nkeycolumns);
|
|
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
/*
|
|
|
|
|
* Historically this code has coped with NULL clauses. That's probably
|
|
|
|
|
* not possible anymore, but we might as well continue to cope.
|
|
|
|
|
*/
|
|
|
|
|
if (clause == NULL)
|
|
|
|
|
return NULL;
|
2018-04-12 15:37:22 +02:00
|
|
|
|
2005-03-27 00:29:20 +01:00
|
|
|
/* First check for boolean-index cases. */
|
2019-02-12 03:26:08 +01:00
|
|
|
opfamily = index->opfamily[indexcol];
|
2006-12-23 01:43:13 +01:00
|
|
|
if (IsBooleanOpfamily(opfamily))
|
2005-03-27 00:29:20 +01:00
|
|
|
{
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
iclause = match_boolean_index_clause(root, rinfo, indexcol, index);
|
2019-02-12 03:26:08 +01:00
|
|
|
if (iclause)
|
|
|
|
|
return iclause;
|
2005-03-27 00:29:20 +01:00
|
|
|
}
|
|
|
|
|
|
2005-11-25 20:47:50 +01:00
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* Clause must be an opclause, funcclause, ScalarArrayOpExpr, or
|
|
|
|
|
* RowCompareExpr. Or, if the index supports it, we can handle IS
|
|
|
|
|
* NULL/NOT NULL clauses.
|
2005-11-25 20:47:50 +01:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
if (IsA(clause, OpExpr))
|
|
|
|
|
{
|
|
|
|
|
return match_opclause_to_indexcol(root, rinfo, indexcol, index);
|
|
|
|
|
}
|
|
|
|
|
else if (IsA(clause, FuncExpr))
|
2005-11-25 20:47:50 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
return match_funcclause_to_indexcol(root, rinfo, indexcol, index);
|
2005-11-25 20:47:50 +01:00
|
|
|
}
|
2019-02-12 03:26:08 +01:00
|
|
|
else if (IsA(clause, ScalarArrayOpExpr))
|
2005-11-25 20:47:50 +01:00
|
|
|
{
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
return match_saopclause_to_indexcol(root, rinfo, indexcol, index);
|
2005-11-25 20:47:50 +01:00
|
|
|
}
|
2019-02-12 03:26:08 +01:00
|
|
|
else if (IsA(clause, RowCompareExpr))
|
2006-01-25 21:29:24 +01:00
|
|
|
{
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
return match_rowcompare_to_indexcol(root, rinfo, indexcol, index);
|
2006-01-25 21:29:24 +01:00
|
|
|
}
|
2007-04-07 00:33:43 +02:00
|
|
|
else if (index->amsearchnulls && IsA(clause, NullTest))
|
|
|
|
|
{
|
|
|
|
|
NullTest *nt = (NullTest *) clause;
|
|
|
|
|
|
2010-01-02 00:03:10 +01:00
|
|
|
if (!nt->argisrow &&
|
|
|
|
|
match_index_to_operand((Node *) nt->arg, indexcol, index))
|
2019-02-12 03:26:08 +01:00
|
|
|
{
|
|
|
|
|
iclause = makeNode(IndexClause);
|
|
|
|
|
iclause->rinfo = rinfo;
|
2019-02-15 01:37:30 +01:00
|
|
|
iclause->indexquals = list_make1(rinfo);
|
2019-02-12 03:26:08 +01:00
|
|
|
iclause->lossy = false;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
2022-09-02 23:01:51 +02:00
|
|
|
/*
|
|
|
|
|
* IsBooleanOpfamily
|
|
|
|
|
* Detect whether an opfamily supports boolean equality as an operator.
|
|
|
|
|
*
|
|
|
|
|
* If the opfamily OID is in the range of built-in objects, we can rely
|
|
|
|
|
* on hard-wired knowledge of which built-in opfamilies support this.
|
|
|
|
|
* For extension opfamilies, there's no choice but to do a catcache lookup.
|
|
|
|
|
*/
|
|
|
|
|
static bool
|
|
|
|
|
IsBooleanOpfamily(Oid opfamily)
|
|
|
|
|
{
|
|
|
|
|
if (opfamily < FirstNormalObjectId)
|
|
|
|
|
return IsBuiltinBooleanOpfamily(opfamily);
|
|
|
|
|
else
|
|
|
|
|
return op_in_opfamily(BooleanEqualOperator, opfamily);
|
|
|
|
|
}
|
|
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
/*
|
|
|
|
|
* match_boolean_index_clause
|
|
|
|
|
* Recognize restriction clauses that can be matched to a boolean index.
|
|
|
|
|
*
|
|
|
|
|
* The idea here is that, for an index on a boolean column that supports the
|
|
|
|
|
* BooleanEqualOperator, we can transform a plain reference to the indexkey
|
|
|
|
|
* into "indexkey = true", or "NOT indexkey" into "indexkey = false", etc,
|
|
|
|
|
* so as to make the expression indexable using the index's "=" operator.
|
|
|
|
|
* Since Postgres 8.1, we must do this because constant simplification does
|
|
|
|
|
* the reverse transformation; without this code there'd be no way to use
|
|
|
|
|
* such an index at all.
|
|
|
|
|
*
|
|
|
|
|
* This should be called only when IsBooleanOpfamily() recognizes the
|
|
|
|
|
* index's operator family. We check to see if the clause matches the
|
|
|
|
|
* index's key, and if so, build a suitable IndexClause.
|
|
|
|
|
*/
|
|
|
|
|
static IndexClause *
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
match_boolean_index_clause(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index)
|
|
|
|
|
{
|
|
|
|
|
Node *clause = (Node *) rinfo->clause;
|
|
|
|
|
Expr *op = NULL;
|
|
|
|
|
|
|
|
|
|
/* Direct match? */
|
|
|
|
|
if (match_index_to_operand(clause, indexcol, index))
|
|
|
|
|
{
|
|
|
|
|
/* convert to indexkey = TRUE */
|
|
|
|
|
op = make_opclause(BooleanEqualOperator, BOOLOID, false,
|
|
|
|
|
(Expr *) clause,
|
|
|
|
|
(Expr *) makeBoolConst(true, false),
|
|
|
|
|
InvalidOid, InvalidOid);
|
|
|
|
|
}
|
|
|
|
|
/* NOT clause? */
|
|
|
|
|
else if (is_notclause(clause))
|
|
|
|
|
{
|
|
|
|
|
Node *arg = (Node *) get_notclausearg((Expr *) clause);
|
|
|
|
|
|
|
|
|
|
if (match_index_to_operand(arg, indexcol, index))
|
|
|
|
|
{
|
|
|
|
|
/* convert to indexkey = FALSE */
|
|
|
|
|
op = make_opclause(BooleanEqualOperator, BOOLOID, false,
|
|
|
|
|
(Expr *) arg,
|
|
|
|
|
(Expr *) makeBoolConst(false, false),
|
|
|
|
|
InvalidOid, InvalidOid);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Since we only consider clauses at top level of WHERE, we can convert
|
|
|
|
|
* indexkey IS TRUE and indexkey IS FALSE to index searches as well. The
|
|
|
|
|
* different meaning for NULL isn't important.
|
|
|
|
|
*/
|
|
|
|
|
else if (clause && IsA(clause, BooleanTest))
|
|
|
|
|
{
|
|
|
|
|
BooleanTest *btest = (BooleanTest *) clause;
|
|
|
|
|
Node *arg = (Node *) btest->arg;
|
|
|
|
|
|
|
|
|
|
if (btest->booltesttype == IS_TRUE &&
|
|
|
|
|
match_index_to_operand(arg, indexcol, index))
|
|
|
|
|
{
|
|
|
|
|
/* convert to indexkey = TRUE */
|
|
|
|
|
op = make_opclause(BooleanEqualOperator, BOOLOID, false,
|
|
|
|
|
(Expr *) arg,
|
|
|
|
|
(Expr *) makeBoolConst(true, false),
|
|
|
|
|
InvalidOid, InvalidOid);
|
|
|
|
|
}
|
|
|
|
|
else if (btest->booltesttype == IS_FALSE &&
|
|
|
|
|
match_index_to_operand(arg, indexcol, index))
|
|
|
|
|
{
|
|
|
|
|
/* convert to indexkey = FALSE */
|
|
|
|
|
op = make_opclause(BooleanEqualOperator, BOOLOID, false,
|
|
|
|
|
(Expr *) arg,
|
|
|
|
|
(Expr *) makeBoolConst(false, false),
|
|
|
|
|
InvalidOid, InvalidOid);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we successfully made an operator clause from the given qual, we must
|
|
|
|
|
* wrap it in an IndexClause. It's not lossy.
|
|
|
|
|
*/
|
|
|
|
|
if (op)
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause = makeNode(IndexClause);
|
|
|
|
|
|
|
|
|
|
iclause->rinfo = rinfo;
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
iclause->indexquals = list_make1(make_simple_restrictinfo(root, op));
|
2019-02-12 03:26:08 +01:00
|
|
|
iclause->lossy = false;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
return iclause;
|
2007-04-07 00:33:43 +02:00
|
|
|
}
|
2019-02-12 03:26:08 +01:00
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_opclause_to_indexcol()
|
|
|
|
|
* Handles the OpExpr case for match_clause_to_indexcol(),
|
|
|
|
|
* which see for comments.
|
|
|
|
|
*/
|
|
|
|
|
static IndexClause *
|
|
|
|
|
match_opclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index)
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause;
|
|
|
|
|
OpExpr *clause = (OpExpr *) rinfo->clause;
|
|
|
|
|
Node *leftop,
|
|
|
|
|
*rightop;
|
|
|
|
|
Oid expr_op;
|
|
|
|
|
Oid expr_coll;
|
|
|
|
|
Index index_relid;
|
|
|
|
|
Oid opfamily;
|
|
|
|
|
Oid idxcollation;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Only binary operators need apply. (In theory, a planner support
|
|
|
|
|
* function could do something with a unary operator, but it seems
|
|
|
|
|
* unlikely to be worth the cycles to check.)
|
|
|
|
|
*/
|
|
|
|
|
if (list_length(clause->args) != 2)
|
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
|
|
leftop = (Node *) linitial(clause->args);
|
|
|
|
|
rightop = (Node *) lsecond(clause->args);
|
|
|
|
|
expr_op = clause->opno;
|
|
|
|
|
expr_coll = clause->inputcollid;
|
|
|
|
|
|
|
|
|
|
index_relid = index->rel->relid;
|
|
|
|
|
opfamily = index->opfamily[indexcol];
|
|
|
|
|
idxcollation = index->indexcollations[indexcol];
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2002-11-24 22:52:15 +01:00
|
|
|
/*
|
|
|
|
|
* Check for clauses of the form: (indexkey operator constant) or
|
2019-02-12 03:26:08 +01:00
|
|
|
* (constant operator indexkey). See match_clause_to_indexcol's notes
|
|
|
|
|
* about const-ness.
|
|
|
|
|
*
|
|
|
|
|
* Note that we don't ask the support function about clauses that don't
|
|
|
|
|
* have one of these forms. Again, in principle it might be possible to
|
|
|
|
|
* do something, but it seems unlikely to be worth the cycles to check.
|
2002-11-24 22:52:15 +01:00
|
|
|
*/
|
2005-03-27 08:29:49 +02:00
|
|
|
if (match_index_to_operand(leftop, indexcol, index) &&
|
2019-02-12 03:26:08 +01:00
|
|
|
!bms_is_member(index_relid, rinfo->right_relids) &&
|
2005-04-22 23:58:32 +02:00
|
|
|
!contain_volatile_functions(rightop))
|
1997-03-18 19:41:37 +01:00
|
|
|
{
|
2011-09-29 06:43:42 +02:00
|
|
|
if (IndexCollMatchesExprColl(idxcollation, expr_coll) &&
|
2019-02-12 03:26:08 +01:00
|
|
|
op_in_opfamily(expr_op, opfamily))
|
|
|
|
|
{
|
|
|
|
|
iclause = makeNode(IndexClause);
|
|
|
|
|
iclause->rinfo = rinfo;
|
2019-02-15 01:37:30 +01:00
|
|
|
iclause->indexquals = list_make1(rinfo);
|
2019-02-12 03:26:08 +01:00
|
|
|
iclause->lossy = false;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
2002-11-24 22:52:15 +01:00
|
|
|
|
1999-07-23 05:34:49 +02:00
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* If we didn't find a member of the index's opfamily, try the support
|
|
|
|
|
* function for the operator's underlying function.
|
1999-07-23 05:34:49 +02:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
set_opfuncid(clause); /* make sure we have opfuncid */
|
|
|
|
|
return get_index_clause_from_support(root,
|
|
|
|
|
rinfo,
|
|
|
|
|
clause->opfuncid,
|
|
|
|
|
0, /* indexarg on left */
|
|
|
|
|
indexcol,
|
|
|
|
|
index);
|
2002-11-24 22:52:15 +01:00
|
|
|
}
|
1999-05-25 18:15:34 +02:00
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
if (match_index_to_operand(rightop, indexcol, index) &&
|
|
|
|
|
!bms_is_member(index_relid, rinfo->left_relids) &&
|
2005-04-22 23:58:32 +02:00
|
|
|
!contain_volatile_functions(leftop))
|
2002-11-24 22:52:15 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
if (IndexCollMatchesExprColl(idxcollation, expr_coll))
|
|
|
|
|
{
|
|
|
|
|
Oid comm_op = get_commutator(expr_op);
|
|
|
|
|
|
|
|
|
|
if (OidIsValid(comm_op) &&
|
|
|
|
|
op_in_opfamily(comm_op, opfamily))
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *commrinfo;
|
|
|
|
|
|
|
|
|
|
/* Build a commuted OpExpr and RestrictInfo */
|
|
|
|
|
commrinfo = commute_restrictinfo(rinfo, comm_op);
|
|
|
|
|
|
|
|
|
|
/* Make an IndexClause showing that as a derived qual */
|
|
|
|
|
iclause = makeNode(IndexClause);
|
|
|
|
|
iclause->rinfo = rinfo;
|
|
|
|
|
iclause->indexquals = list_make1(commrinfo);
|
|
|
|
|
iclause->lossy = false;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
|
|
|
|
}
|
2000-04-12 19:17:23 +02:00
|
|
|
|
2002-11-24 22:52:15 +01:00
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* If we didn't find a member of the index's opfamily, try the support
|
|
|
|
|
* function for the operator's underlying function.
|
2002-11-24 22:52:15 +01:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
set_opfuncid(clause); /* make sure we have opfuncid */
|
|
|
|
|
return get_index_clause_from_support(root,
|
|
|
|
|
rinfo,
|
|
|
|
|
clause->opfuncid,
|
|
|
|
|
1, /* indexarg on right */
|
|
|
|
|
indexcol,
|
|
|
|
|
index);
|
2002-11-24 22:52:15 +01:00
|
|
|
}
|
2000-04-12 19:17:23 +02:00
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
2002-11-24 22:52:15 +01:00
|
|
|
}
|
|
|
|
|
|
1999-08-12 06:32:54 +02:00
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* match_funcclause_to_indexcol()
|
|
|
|
|
* Handles the FuncExpr case for match_clause_to_indexcol(),
|
|
|
|
|
* which see for comments.
|
1999-08-12 06:32:54 +02:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
static IndexClause *
|
|
|
|
|
match_funcclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index)
|
1999-08-12 06:32:54 +02:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
FuncExpr *clause = (FuncExpr *) rinfo->clause;
|
|
|
|
|
int indexarg;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We have no built-in intelligence about function clauses, but if there's
|
|
|
|
|
* a planner support function, it might be able to do something. But, to
|
|
|
|
|
* cut down on wasted planning cycles, only call the support function if
|
|
|
|
|
* at least one argument matches the target index column.
|
|
|
|
|
*
|
|
|
|
|
* Note that we don't insist on the other arguments being pseudoconstants;
|
|
|
|
|
* the support function has to check that. This is to allow cases where
|
|
|
|
|
* only some of the other arguments need to be included in the indexqual.
|
|
|
|
|
*/
|
|
|
|
|
indexarg = 0;
|
|
|
|
|
foreach(lc, clause->args)
|
2005-11-25 20:47:50 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
Node *op = (Node *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
if (match_index_to_operand(op, indexcol, index))
|
|
|
|
|
{
|
|
|
|
|
return get_index_clause_from_support(root,
|
|
|
|
|
rinfo,
|
|
|
|
|
clause->funcid,
|
|
|
|
|
indexarg,
|
|
|
|
|
indexcol,
|
|
|
|
|
index);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
indexarg++;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* get_index_clause_from_support()
|
|
|
|
|
* If the function has a planner support function, try to construct
|
|
|
|
|
* an IndexClause using indexquals created by the support function.
|
|
|
|
|
*/
|
|
|
|
|
static IndexClause *
|
|
|
|
|
get_index_clause_from_support(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
|
|
|
|
Oid funcid,
|
|
|
|
|
int indexarg,
|
|
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index)
|
|
|
|
|
{
|
|
|
|
|
Oid prosupport = get_func_support(funcid);
|
|
|
|
|
SupportRequestIndexCondition req;
|
|
|
|
|
List *sresult;
|
|
|
|
|
|
|
|
|
|
if (!OidIsValid(prosupport))
|
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
|
|
req.type = T_SupportRequestIndexCondition;
|
|
|
|
|
req.root = root;
|
|
|
|
|
req.funcid = funcid;
|
|
|
|
|
req.node = (Node *) rinfo->clause;
|
|
|
|
|
req.indexarg = indexarg;
|
|
|
|
|
req.index = index;
|
|
|
|
|
req.indexcol = indexcol;
|
|
|
|
|
req.opfamily = index->opfamily[indexcol];
|
|
|
|
|
req.indexcollation = index->indexcollations[indexcol];
|
|
|
|
|
|
|
|
|
|
req.lossy = true; /* default assumption */
|
|
|
|
|
|
|
|
|
|
sresult = (List *)
|
|
|
|
|
DatumGetPointer(OidFunctionCall1(prosupport,
|
|
|
|
|
PointerGetDatum(&req)));
|
|
|
|
|
|
|
|
|
|
if (sresult != NIL)
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause = makeNode(IndexClause);
|
|
|
|
|
List *indexquals = NIL;
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The support function API says it should just give back bare
|
|
|
|
|
* clauses, so here we must wrap each one in a RestrictInfo.
|
|
|
|
|
*/
|
|
|
|
|
foreach(lc, sresult)
|
|
|
|
|
{
|
|
|
|
|
Expr *clause = (Expr *) lfirst(lc);
|
|
|
|
|
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
indexquals = lappend(indexquals,
|
|
|
|
|
make_simple_restrictinfo(root, clause));
|
2019-02-12 03:26:08 +01:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
iclause->rinfo = rinfo;
|
|
|
|
|
iclause->indexquals = indexquals;
|
|
|
|
|
iclause->lossy = req.lossy;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return NULL;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_saopclause_to_indexcol()
|
|
|
|
|
* Handles the ScalarArrayOpExpr case for match_clause_to_indexcol(),
|
|
|
|
|
* which see for comments.
|
|
|
|
|
*/
|
|
|
|
|
static IndexClause *
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
match_saopclause_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index)
|
|
|
|
|
{
|
|
|
|
|
ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) rinfo->clause;
|
|
|
|
|
Node *leftop,
|
|
|
|
|
*rightop;
|
|
|
|
|
Relids right_relids;
|
|
|
|
|
Oid expr_op;
|
|
|
|
|
Oid expr_coll;
|
|
|
|
|
Index index_relid;
|
|
|
|
|
Oid opfamily;
|
|
|
|
|
Oid idxcollation;
|
|
|
|
|
|
|
|
|
|
/* We only accept ANY clauses, not ALL */
|
|
|
|
|
if (!saop->useOr)
|
|
|
|
|
return NULL;
|
|
|
|
|
leftop = (Node *) linitial(saop->args);
|
|
|
|
|
rightop = (Node *) lsecond(saop->args);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
right_relids = pull_varnos(root, rightop);
|
2019-02-12 03:26:08 +01:00
|
|
|
expr_op = saop->opno;
|
|
|
|
|
expr_coll = saop->inputcollid;
|
|
|
|
|
|
|
|
|
|
index_relid = index->rel->relid;
|
|
|
|
|
opfamily = index->opfamily[indexcol];
|
|
|
|
|
idxcollation = index->indexcollations[indexcol];
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We must have indexkey on the left and a pseudo-constant array argument.
|
|
|
|
|
*/
|
|
|
|
|
if (match_index_to_operand(leftop, indexcol, index) &&
|
|
|
|
|
!bms_is_member(index_relid, right_relids) &&
|
|
|
|
|
!contain_volatile_functions(rightop))
|
|
|
|
|
{
|
|
|
|
|
if (IndexCollMatchesExprColl(idxcollation, expr_coll) &&
|
|
|
|
|
op_in_opfamily(expr_op, opfamily))
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause = makeNode(IndexClause);
|
|
|
|
|
|
|
|
|
|
iclause->rinfo = rinfo;
|
2019-02-15 01:37:30 +01:00
|
|
|
iclause->indexquals = list_make1(rinfo);
|
2019-02-12 03:26:08 +01:00
|
|
|
iclause->lossy = false;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We do not currently ask support functions about ScalarArrayOpExprs,
|
|
|
|
|
* though in principle we could.
|
|
|
|
|
*/
|
2005-11-25 20:47:50 +01:00
|
|
|
}
|
1999-08-12 06:32:54 +02:00
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
|
|
|
|
|
2006-01-25 21:29:24 +01:00
|
|
|
/*
|
|
|
|
|
* match_rowcompare_to_indexcol()
|
|
|
|
|
* Handles the RowCompareExpr case for match_clause_to_indexcol(),
|
|
|
|
|
* which see for comments.
|
2019-02-12 03:26:08 +01:00
|
|
|
*
|
|
|
|
|
* In this routine we check whether the first column of the row comparison
|
|
|
|
|
* matches the target index column. This is sufficient to guarantee that some
|
|
|
|
|
* index condition can be constructed from the RowCompareExpr --- the rest
|
|
|
|
|
* is handled by expand_indexqual_rowcompare().
|
2006-01-25 21:29:24 +01:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
static IndexClause *
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
match_rowcompare_to_indexcol(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2006-01-25 21:29:24 +01:00
|
|
|
int indexcol,
|
2019-02-12 03:26:08 +01:00
|
|
|
IndexOptInfo *index)
|
2006-01-25 21:29:24 +01:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
RowCompareExpr *clause = (RowCompareExpr *) rinfo->clause;
|
|
|
|
|
Index index_relid;
|
|
|
|
|
Oid opfamily;
|
|
|
|
|
Oid idxcollation;
|
2006-01-25 21:29:24 +01:00
|
|
|
Node *leftop,
|
|
|
|
|
*rightop;
|
2019-02-12 03:26:08 +01:00
|
|
|
bool var_on_left;
|
2006-01-25 21:29:24 +01:00
|
|
|
Oid expr_op;
|
2011-04-09 01:19:17 +02:00
|
|
|
Oid expr_coll;
|
2006-01-25 21:29:24 +01:00
|
|
|
|
|
|
|
|
/* Forget it if we're not dealing with a btree index */
|
|
|
|
|
if (index->relam != BTREE_AM_OID)
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
|
|
|
|
|
|
|
|
|
index_relid = index->rel->relid;
|
|
|
|
|
opfamily = index->opfamily[indexcol];
|
|
|
|
|
idxcollation = index->indexcollations[indexcol];
|
2006-01-25 21:29:24 +01:00
|
|
|
|
|
|
|
|
/*
|
2006-12-23 01:43:13 +01:00
|
|
|
* We could do the matching on the basis of insisting that the opfamily
|
|
|
|
|
* shown in the RowCompareExpr be the same as the index column's opfamily,
|
|
|
|
|
* but that could fail in the presence of reverse-sort opfamilies: it'd be
|
|
|
|
|
* a matter of chance whether RowCompareExpr had picked the forward or
|
|
|
|
|
* reverse-sort family. So look only at the operator, and match if it is
|
|
|
|
|
* a member of the index's opfamily (after commutation, if the indexkey is
|
|
|
|
|
* on the right). We'll worry later about whether any additional
|
|
|
|
|
* operators are matchable to the index.
|
2006-01-25 21:29:24 +01:00
|
|
|
*/
|
|
|
|
|
leftop = (Node *) linitial(clause->largs);
|
|
|
|
|
rightop = (Node *) linitial(clause->rargs);
|
|
|
|
|
expr_op = linitial_oid(clause->opnos);
|
2011-04-09 01:19:17 +02:00
|
|
|
expr_coll = linitial_oid(clause->inputcollids);
|
|
|
|
|
|
2011-09-29 06:43:42 +02:00
|
|
|
/* Collations must match, if relevant */
|
|
|
|
|
if (!IndexCollMatchesExprColl(idxcollation, expr_coll))
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
2006-01-25 21:29:24 +01:00
|
|
|
|
|
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* These syntactic tests are the same as in match_opclause_to_indexcol()
|
2006-01-25 21:29:24 +01:00
|
|
|
*/
|
|
|
|
|
if (match_index_to_operand(leftop, indexcol, index) &&
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
!bms_is_member(index_relid, pull_varnos(root, rightop)) &&
|
2006-01-25 21:29:24 +01:00
|
|
|
!contain_volatile_functions(rightop))
|
|
|
|
|
{
|
|
|
|
|
/* OK, indexkey is on left */
|
2019-02-12 03:26:08 +01:00
|
|
|
var_on_left = true;
|
2006-01-25 21:29:24 +01:00
|
|
|
}
|
|
|
|
|
else if (match_index_to_operand(rightop, indexcol, index) &&
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
!bms_is_member(index_relid, pull_varnos(root, leftop)) &&
|
2006-01-25 21:29:24 +01:00
|
|
|
!contain_volatile_functions(leftop))
|
|
|
|
|
{
|
|
|
|
|
/* indexkey is on right, so commute the operator */
|
|
|
|
|
expr_op = get_commutator(expr_op);
|
|
|
|
|
if (expr_op == InvalidOid)
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
|
|
|
|
var_on_left = false;
|
2006-01-25 21:29:24 +01:00
|
|
|
}
|
|
|
|
|
else
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
2006-01-25 21:29:24 +01:00
|
|
|
|
2006-12-23 01:43:13 +01:00
|
|
|
/* We're good if the operator is the right type of opfamily member */
|
|
|
|
|
switch (get_op_opfamily_strategy(expr_op, opfamily))
|
2006-01-25 21:29:24 +01:00
|
|
|
{
|
|
|
|
|
case BTLessStrategyNumber:
|
|
|
|
|
case BTLessEqualStrategyNumber:
|
|
|
|
|
case BTGreaterEqualStrategyNumber:
|
|
|
|
|
case BTGreaterStrategyNumber:
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
return expand_indexqual_rowcompare(root,
|
|
|
|
|
rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
indexcol,
|
|
|
|
|
index,
|
|
|
|
|
expr_op,
|
|
|
|
|
var_on_left);
|
2006-01-25 21:29:24 +01:00
|
|
|
}
|
|
|
|
|
|
2019-02-12 03:26:08 +01:00
|
|
|
return NULL;
|
2006-01-25 21:29:24 +01:00
|
|
|
}
|
|
|
|
|
|
2010-12-03 02:50:48 +01:00
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* expand_indexqual_rowcompare --- expand a single indexqual condition
|
|
|
|
|
* that is a RowCompareExpr
|
2010-12-03 02:50:48 +01:00
|
|
|
*
|
2019-02-12 03:26:08 +01:00
|
|
|
* It's already known that the first column of the row comparison matches
|
|
|
|
|
* the specified column of the index. We can use additional columns of the
|
|
|
|
|
* row comparison as index qualifications, so long as they match the index
|
|
|
|
|
* in the "same direction", ie, the indexkeys are all on the same side of the
|
|
|
|
|
* clause and the operators are all the same-type members of the opfamilies.
|
|
|
|
|
*
|
|
|
|
|
* If all the columns of the RowCompareExpr match in this way, we just use it
|
|
|
|
|
* as-is, except for possibly commuting it to put the indexkeys on the left.
|
|
|
|
|
*
|
|
|
|
|
* Otherwise, we build a shortened RowCompareExpr (if more than one
|
|
|
|
|
* column matches) or a simple OpExpr (if the first-column match is all
|
|
|
|
|
* there is). In these cases the modified clause is always "<=" or ">="
|
|
|
|
|
* even when the original was "<" or ">" --- this is necessary to match all
|
|
|
|
|
* the rows that could match the original. (We are building a lossy version
|
|
|
|
|
* of the row comparison when we do this, so we set lossy = true.)
|
|
|
|
|
*
|
|
|
|
|
* Note: this is really just the last half of match_rowcompare_to_indexcol,
|
|
|
|
|
* but we split it out for comprehensibility.
|
|
|
|
|
*/
|
|
|
|
|
static IndexClause *
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
expand_indexqual_rowcompare(PlannerInfo *root,
|
|
|
|
|
RestrictInfo *rinfo,
|
2019-02-12 03:26:08 +01:00
|
|
|
int indexcol,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
Oid expr_op,
|
|
|
|
|
bool var_on_left)
|
|
|
|
|
{
|
|
|
|
|
IndexClause *iclause = makeNode(IndexClause);
|
|
|
|
|
RowCompareExpr *clause = (RowCompareExpr *) rinfo->clause;
|
|
|
|
|
int op_strategy;
|
|
|
|
|
Oid op_lefttype;
|
|
|
|
|
Oid op_righttype;
|
|
|
|
|
int matching_cols;
|
|
|
|
|
List *expr_ops;
|
|
|
|
|
List *opfamilies;
|
|
|
|
|
List *lefttypes;
|
|
|
|
|
List *righttypes;
|
|
|
|
|
List *new_ops;
|
|
|
|
|
List *var_args;
|
|
|
|
|
List *non_var_args;
|
|
|
|
|
|
|
|
|
|
iclause->rinfo = rinfo;
|
|
|
|
|
iclause->indexcol = indexcol;
|
|
|
|
|
|
|
|
|
|
if (var_on_left)
|
|
|
|
|
{
|
|
|
|
|
var_args = clause->largs;
|
|
|
|
|
non_var_args = clause->rargs;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
var_args = clause->rargs;
|
|
|
|
|
non_var_args = clause->largs;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
get_op_opfamily_properties(expr_op, index->opfamily[indexcol], false,
|
|
|
|
|
&op_strategy,
|
|
|
|
|
&op_lefttype,
|
|
|
|
|
&op_righttype);
|
|
|
|
|
|
|
|
|
|
/* Initialize returned list of which index columns are used */
|
|
|
|
|
iclause->indexcols = list_make1_int(indexcol);
|
|
|
|
|
|
|
|
|
|
/* Build lists of ops, opfamilies and operator datatypes in case needed */
|
|
|
|
|
expr_ops = list_make1_oid(expr_op);
|
|
|
|
|
opfamilies = list_make1_oid(index->opfamily[indexcol]);
|
|
|
|
|
lefttypes = list_make1_oid(op_lefttype);
|
|
|
|
|
righttypes = list_make1_oid(op_righttype);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* See how many of the remaining columns match some index column in the
|
|
|
|
|
* same way. As in match_clause_to_indexcol(), the "other" side of any
|
|
|
|
|
* potential index condition is OK as long as it doesn't use Vars from the
|
|
|
|
|
* indexed relation.
|
|
|
|
|
*/
|
|
|
|
|
matching_cols = 1;
|
|
|
|
|
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
while (matching_cols < list_length(var_args))
|
2019-02-12 03:26:08 +01:00
|
|
|
{
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
Node *varop = (Node *) list_nth(var_args, matching_cols);
|
|
|
|
|
Node *constop = (Node *) list_nth(non_var_args, matching_cols);
|
2019-02-12 03:26:08 +01:00
|
|
|
int i;
|
|
|
|
|
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
expr_op = list_nth_oid(clause->opnos, matching_cols);
|
2019-02-12 03:26:08 +01:00
|
|
|
if (!var_on_left)
|
|
|
|
|
{
|
|
|
|
|
/* indexkey is on right, so commute the operator */
|
|
|
|
|
expr_op = get_commutator(expr_op);
|
|
|
|
|
if (expr_op == InvalidOid)
|
|
|
|
|
break; /* operator is not usable */
|
|
|
|
|
}
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
if (bms_is_member(index->rel->relid, pull_varnos(root, constop)))
|
2019-02-12 03:26:08 +01:00
|
|
|
break; /* no good, Var on wrong side */
|
|
|
|
|
if (contain_volatile_functions(constop))
|
|
|
|
|
break; /* no good, volatile comparison value */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The Var side can match any key column of the index.
|
|
|
|
|
*/
|
|
|
|
|
for (i = 0; i < index->nkeycolumns; i++)
|
|
|
|
|
{
|
|
|
|
|
if (match_index_to_operand(varop, i, index) &&
|
|
|
|
|
get_op_opfamily_strategy(expr_op,
|
|
|
|
|
index->opfamily[i]) == op_strategy &&
|
|
|
|
|
IndexCollMatchesExprColl(index->indexcollations[i],
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
list_nth_oid(clause->inputcollids,
|
|
|
|
|
matching_cols)))
|
2019-02-12 03:26:08 +01:00
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
if (i >= index->nkeycolumns)
|
|
|
|
|
break; /* no match found */
|
|
|
|
|
|
|
|
|
|
/* Add column number to returned list */
|
|
|
|
|
iclause->indexcols = lappend_int(iclause->indexcols, i);
|
|
|
|
|
|
|
|
|
|
/* Add operator info to lists */
|
|
|
|
|
get_op_opfamily_properties(expr_op, index->opfamily[i], false,
|
|
|
|
|
&op_strategy,
|
|
|
|
|
&op_lefttype,
|
|
|
|
|
&op_righttype);
|
|
|
|
|
expr_ops = lappend_oid(expr_ops, expr_op);
|
|
|
|
|
opfamilies = lappend_oid(opfamilies, index->opfamily[i]);
|
|
|
|
|
lefttypes = lappend_oid(lefttypes, op_lefttype);
|
|
|
|
|
righttypes = lappend_oid(righttypes, op_righttype);
|
|
|
|
|
|
|
|
|
|
/* This column matches, keep scanning */
|
|
|
|
|
matching_cols++;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* Result is non-lossy if all columns are usable as index quals */
|
|
|
|
|
iclause->lossy = (matching_cols != list_length(clause->opnos));
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We can use rinfo->clause as-is if we have var on left and it's all
|
|
|
|
|
* usable as index quals.
|
|
|
|
|
*/
|
|
|
|
|
if (var_on_left && !iclause->lossy)
|
2019-02-15 01:37:30 +01:00
|
|
|
iclause->indexquals = list_make1(rinfo);
|
2019-02-12 03:26:08 +01:00
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
/*
|
|
|
|
|
* We have to generate a modified rowcompare (possibly just one
|
|
|
|
|
* OpExpr). The painful part of this is changing < to <= or > to >=,
|
|
|
|
|
* so deal with that first.
|
|
|
|
|
*/
|
|
|
|
|
if (!iclause->lossy)
|
|
|
|
|
{
|
|
|
|
|
/* very easy, just use the commuted operators */
|
|
|
|
|
new_ops = expr_ops;
|
|
|
|
|
}
|
|
|
|
|
else if (op_strategy == BTLessEqualStrategyNumber ||
|
|
|
|
|
op_strategy == BTGreaterEqualStrategyNumber)
|
|
|
|
|
{
|
|
|
|
|
/* easy, just use the same (possibly commuted) operators */
|
|
|
|
|
new_ops = list_truncate(expr_ops, matching_cols);
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
ListCell *opfamilies_cell;
|
|
|
|
|
ListCell *lefttypes_cell;
|
|
|
|
|
ListCell *righttypes_cell;
|
|
|
|
|
|
|
|
|
|
if (op_strategy == BTLessStrategyNumber)
|
|
|
|
|
op_strategy = BTLessEqualStrategyNumber;
|
|
|
|
|
else if (op_strategy == BTGreaterStrategyNumber)
|
|
|
|
|
op_strategy = BTGreaterEqualStrategyNumber;
|
|
|
|
|
else
|
|
|
|
|
elog(ERROR, "unexpected strategy number %d", op_strategy);
|
|
|
|
|
new_ops = NIL;
|
|
|
|
|
forthree(opfamilies_cell, opfamilies,
|
|
|
|
|
lefttypes_cell, lefttypes,
|
|
|
|
|
righttypes_cell, righttypes)
|
|
|
|
|
{
|
|
|
|
|
Oid opfam = lfirst_oid(opfamilies_cell);
|
|
|
|
|
Oid lefttype = lfirst_oid(lefttypes_cell);
|
|
|
|
|
Oid righttype = lfirst_oid(righttypes_cell);
|
|
|
|
|
|
|
|
|
|
expr_op = get_opfamily_member(opfam, lefttype, righttype,
|
|
|
|
|
op_strategy);
|
|
|
|
|
if (!OidIsValid(expr_op)) /* should not happen */
|
|
|
|
|
elog(ERROR, "missing operator %d(%u,%u) in opfamily %u",
|
|
|
|
|
op_strategy, lefttype, righttype, opfam);
|
|
|
|
|
new_ops = lappend_oid(new_ops, expr_op);
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/* If we have more than one matching col, create a subset rowcompare */
|
|
|
|
|
if (matching_cols > 1)
|
|
|
|
|
{
|
|
|
|
|
RowCompareExpr *rc = makeNode(RowCompareExpr);
|
|
|
|
|
|
|
|
|
|
rc->rctype = (RowCompareType) op_strategy;
|
|
|
|
|
rc->opnos = new_ops;
|
2022-07-13 05:03:47 +02:00
|
|
|
rc->opfamilies = list_copy_head(clause->opfamilies,
|
|
|
|
|
matching_cols);
|
|
|
|
|
rc->inputcollids = list_copy_head(clause->inputcollids,
|
|
|
|
|
matching_cols);
|
|
|
|
|
rc->largs = list_copy_head(var_args, matching_cols);
|
|
|
|
|
rc->rargs = list_copy_head(non_var_args, matching_cols);
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
iclause->indexquals = list_make1(make_simple_restrictinfo(root,
|
|
|
|
|
(Expr *) rc));
|
2019-02-12 03:26:08 +01:00
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
{
|
|
|
|
|
Expr *op;
|
|
|
|
|
|
|
|
|
|
/* We don't report an index column list in this case */
|
|
|
|
|
iclause->indexcols = NIL;
|
|
|
|
|
|
|
|
|
|
op = make_opclause(linitial_oid(new_ops), BOOLOID, false,
|
|
|
|
|
copyObject(linitial(var_args)),
|
|
|
|
|
copyObject(linitial(non_var_args)),
|
|
|
|
|
InvalidOid,
|
|
|
|
|
linitial_oid(clause->inputcollids));
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
iclause->indexquals = list_make1(make_simple_restrictinfo(root, op));
|
2019-02-12 03:26:08 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return iclause;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/****************************************************************************
|
|
|
|
|
* ---- ROUTINES TO CHECK ORDERING OPERATORS ----
|
|
|
|
|
****************************************************************************/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_pathkeys_to_index
|
2023-07-04 13:08:52 +02:00
|
|
|
* For the given 'index' and 'pathkeys', output a list of suitable ORDER
|
|
|
|
|
* BY expressions, each of the form "indexedcol operator pseudoconstant",
|
|
|
|
|
* along with an integer list of the index column numbers (zero based)
|
|
|
|
|
* that each clause would be used with.
|
|
|
|
|
*
|
|
|
|
|
* This attempts to find an ORDER BY and index column number for all items in
|
|
|
|
|
* the pathkey list, however, if we're unable to match any given pathkey to an
|
|
|
|
|
* index column, we return just the ones matched by the function so far. This
|
|
|
|
|
* allows callers who are interested in partial matches to get them. Callers
|
|
|
|
|
* can determine a partial match vs a full match by checking the outputted
|
|
|
|
|
* list lengths. A full match will have one item in the output lists for each
|
|
|
|
|
* item in the given 'pathkeys' list.
|
2010-12-03 02:50:48 +01:00
|
|
|
*/
|
2011-12-25 01:03:21 +01:00
|
|
|
static void
|
|
|
|
|
match_pathkeys_to_index(IndexOptInfo *index, List *pathkeys,
|
|
|
|
|
List **orderby_clauses_p,
|
|
|
|
|
List **clause_columns_p)
|
2010-12-03 02:50:48 +01:00
|
|
|
{
|
|
|
|
|
ListCell *lc1;
|
|
|
|
|
|
2011-12-25 01:03:21 +01:00
|
|
|
*orderby_clauses_p = NIL; /* set default results */
|
|
|
|
|
*clause_columns_p = NIL;
|
|
|
|
|
|
2010-12-03 02:50:48 +01:00
|
|
|
/* Only indexes with the amcanorderbyop property are interesting here */
|
|
|
|
|
if (!index->amcanorderbyop)
|
2011-12-25 01:03:21 +01:00
|
|
|
return;
|
2010-12-03 02:50:48 +01:00
|
|
|
|
|
|
|
|
foreach(lc1, pathkeys)
|
|
|
|
|
{
|
|
|
|
|
PathKey *pathkey = (PathKey *) lfirst(lc1);
|
|
|
|
|
bool found = false;
|
|
|
|
|
ListCell *lc2;
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
/* Pathkey must request default sort order for the target opfamily */
|
|
|
|
|
if (pathkey->pk_strategy != BTLessStrategyNumber ||
|
|
|
|
|
pathkey->pk_nulls_first)
|
2011-12-25 01:03:21 +01:00
|
|
|
return;
|
2010-12-03 02:50:48 +01:00
|
|
|
|
|
|
|
|
/* If eclass is volatile, no hope of using an indexscan */
|
|
|
|
|
if (pathkey->pk_eclass->ec_has_volatile)
|
2011-12-25 01:03:21 +01:00
|
|
|
return;
|
2010-12-03 02:50:48 +01:00
|
|
|
|
Revisit handling of UNION ALL subqueries with non-Var output columns.
In commit 57664ed25e5dea117158a2e663c29e60b3546e1c I tried to fix a bug
reported by Teodor Sigaev by making non-simple-Var output columns distinct
(by wrapping their expressions with dummy PlaceHolderVar nodes). This did
not work too well. Commit b28ffd0fcc583c1811e5295279e7d4366c3cae6c fixed
some ensuing problems with matching to child indexes, but per a recent
report from Claus Stadler, constraint exclusion of UNION ALL subqueries was
still broken, because constant-simplification didn't handle the injected
PlaceHolderVars well either. On reflection, the original patch was quite
misguided: there is no reason to expect that EquivalenceClass child members
will be distinct. So instead of trying to make them so, we should ensure
that we can cope with the situation when they're not.
Accordingly, this patch reverts the code changes in the above-mentioned
commits (though the regression test cases they added stay). Instead, I've
added assorted defenses to make sure that duplicate EC child members don't
cause any problems. Teodor's original problem ("MergeAppend child's
targetlist doesn't match MergeAppend") is addressed more directly by
revising prepare_sort_from_pathkeys to let the parent MergeAppend's sort
list guide creation of each child's sort list.
In passing, get rid of add_sort_column; as far as I can tell, testing for
duplicate sort keys at this stage is dead code. Certainly it doesn't
trigger often enough to be worth expending cycles on in ordinary queries.
And keeping the test would've greatly complicated the new logic in
prepare_sort_from_pathkeys, because comparing pathkey list entries against
a previous output array requires that we not skip any entries in the list.
Back-patch to 9.1, like the previous patches. The only known issue in
this area that wasn't caused by the ill-advised previous patches was the
MergeAppend planning failure, which of course is not relevant before 9.1.
It's possible that we need some of the new defenses against duplicate child
EC entries in older branches, but until there's some clear evidence of that
I'm going to refrain from back-patching further.
2012-03-16 18:11:12 +01:00
|
|
|
/*
|
|
|
|
|
* Try to match eclass member expression(s) to index. Note that child
|
|
|
|
|
* EC members are considered, but only when they belong to the target
|
|
|
|
|
* relation. (Unlike regular members, the same expression could be a
|
|
|
|
|
* child member of more than one EC. Therefore, the same index could
|
|
|
|
|
* be considered to match more than one pathkey list, which is OK
|
|
|
|
|
* here. See also get_eclass_for_sort_expr.)
|
|
|
|
|
*/
|
2010-12-03 02:50:48 +01:00
|
|
|
foreach(lc2, pathkey->pk_eclass->ec_members)
|
|
|
|
|
{
|
|
|
|
|
EquivalenceMember *member = (EquivalenceMember *) lfirst(lc2);
|
|
|
|
|
int indexcol;
|
|
|
|
|
|
|
|
|
|
/* No possibility of match if it references other relations */
|
|
|
|
|
if (!bms_equal(member->em_relids, index->rel->relids))
|
|
|
|
|
continue;
|
|
|
|
|
|
2012-03-16 19:07:21 +01:00
|
|
|
/*
|
|
|
|
|
* We allow any column of the index to match each pathkey; they
|
|
|
|
|
* don't have to match left-to-right as you might expect. This is
|
2019-02-13 00:38:32 +01:00
|
|
|
* correct for GiST, and it doesn't matter for SP-GiST because
|
|
|
|
|
* that doesn't handle multiple columns anyway, and no other
|
|
|
|
|
* existing AMs support amcanorderbyop. We might need different
|
|
|
|
|
* logic in future for other implementations.
|
2012-03-16 19:07:21 +01:00
|
|
|
*/
|
2019-02-13 00:38:32 +01:00
|
|
|
for (indexcol = 0; indexcol < index->nkeycolumns; indexcol++)
|
2010-12-03 02:50:48 +01:00
|
|
|
{
|
|
|
|
|
Expr *expr;
|
|
|
|
|
|
|
|
|
|
expr = match_clause_to_ordering_op(index,
|
|
|
|
|
indexcol,
|
|
|
|
|
member->em_expr,
|
|
|
|
|
pathkey->pk_opfamily);
|
|
|
|
|
if (expr)
|
|
|
|
|
{
|
2023-07-04 13:08:52 +02:00
|
|
|
*orderby_clauses_p = lappend(*orderby_clauses_p, expr);
|
|
|
|
|
*clause_columns_p = lappend_int(*clause_columns_p, indexcol);
|
2010-12-03 02:50:48 +01:00
|
|
|
found = true;
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (found) /* don't want to look at remaining members */
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
|
2023-07-04 13:08:52 +02:00
|
|
|
/*
|
|
|
|
|
* Return the matches found so far when this pathkey couldn't be
|
|
|
|
|
* matched to the index.
|
|
|
|
|
*/
|
|
|
|
|
if (!found)
|
2011-12-25 01:03:21 +01:00
|
|
|
return;
|
2010-12-03 02:50:48 +01:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_clause_to_ordering_op
|
|
|
|
|
* Determines whether an ordering operator expression matches an
|
|
|
|
|
* index column.
|
|
|
|
|
*
|
|
|
|
|
* This is similar to, but simpler than, match_clause_to_indexcol.
|
|
|
|
|
* We only care about simple OpExpr cases. The input is a bare
|
|
|
|
|
* expression that is being ordered by, which must be of the form
|
|
|
|
|
* (indexkey op const) or (const op indexkey) where op is an ordering
|
|
|
|
|
* operator for the column's opfamily.
|
|
|
|
|
*
|
|
|
|
|
* 'index' is the index of interest.
|
|
|
|
|
* 'indexcol' is a column number of 'index' (counting from 0).
|
|
|
|
|
* 'clause' is the ordering expression to be tested.
|
|
|
|
|
* 'pk_opfamily' is the btree opfamily describing the required sort order.
|
|
|
|
|
*
|
2011-04-09 01:19:17 +02:00
|
|
|
* Note that we currently do not consider the collation of the ordering
|
|
|
|
|
* operator's result. In practical cases the result type will be numeric
|
|
|
|
|
* and thus have no collation, and it's not very clear what to match to
|
|
|
|
|
* if it did have a collation. The index's collation should match the
|
|
|
|
|
* ordering operator's input collation, not its result.
|
|
|
|
|
*
|
2010-12-03 02:50:48 +01:00
|
|
|
* If successful, return 'clause' as-is if the indexkey is on the left,
|
|
|
|
|
* otherwise a commuted copy of 'clause'. If no match, return NULL.
|
|
|
|
|
*/
|
|
|
|
|
static Expr *
|
|
|
|
|
match_clause_to_ordering_op(IndexOptInfo *index,
|
|
|
|
|
int indexcol,
|
|
|
|
|
Expr *clause,
|
|
|
|
|
Oid pk_opfamily)
|
|
|
|
|
{
|
2018-04-12 15:37:22 +02:00
|
|
|
Oid opfamily;
|
|
|
|
|
Oid idxcollation;
|
2010-12-03 02:50:48 +01:00
|
|
|
Node *leftop,
|
|
|
|
|
*rightop;
|
|
|
|
|
Oid expr_op;
|
2011-04-09 01:19:17 +02:00
|
|
|
Oid expr_coll;
|
2010-12-03 02:50:48 +01:00
|
|
|
Oid sortfamily;
|
|
|
|
|
bool commuted;
|
|
|
|
|
|
2018-04-12 15:37:22 +02:00
|
|
|
Assert(indexcol < index->nkeycolumns);
|
|
|
|
|
|
|
|
|
|
opfamily = index->opfamily[indexcol];
|
|
|
|
|
idxcollation = index->indexcollations[indexcol];
|
2018-04-26 20:47:16 +02:00
|
|
|
|
2010-12-03 02:50:48 +01:00
|
|
|
/*
|
|
|
|
|
* Clause must be a binary opclause.
|
|
|
|
|
*/
|
|
|
|
|
if (!is_opclause(clause))
|
|
|
|
|
return NULL;
|
|
|
|
|
leftop = get_leftop(clause);
|
|
|
|
|
rightop = get_rightop(clause);
|
|
|
|
|
if (!leftop || !rightop)
|
|
|
|
|
return NULL;
|
|
|
|
|
expr_op = ((OpExpr *) clause)->opno;
|
2011-04-09 01:19:17 +02:00
|
|
|
expr_coll = ((OpExpr *) clause)->inputcollid;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* We can forget the whole thing right away if wrong collation.
|
|
|
|
|
*/
|
2011-09-29 06:43:42 +02:00
|
|
|
if (!IndexCollMatchesExprColl(idxcollation, expr_coll))
|
2011-04-09 01:19:17 +02:00
|
|
|
return NULL;
|
2010-12-03 02:50:48 +01:00
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Check for clauses of the form: (indexkey operator constant) or
|
|
|
|
|
* (constant operator indexkey).
|
|
|
|
|
*/
|
|
|
|
|
if (match_index_to_operand(leftop, indexcol, index) &&
|
|
|
|
|
!contain_var_clause(rightop) &&
|
|
|
|
|
!contain_volatile_functions(rightop))
|
|
|
|
|
{
|
|
|
|
|
commuted = false;
|
|
|
|
|
}
|
|
|
|
|
else if (match_index_to_operand(rightop, indexcol, index) &&
|
|
|
|
|
!contain_var_clause(leftop) &&
|
|
|
|
|
!contain_volatile_functions(leftop))
|
|
|
|
|
{
|
|
|
|
|
/* Might match, but we need a commuted operator */
|
|
|
|
|
expr_op = get_commutator(expr_op);
|
|
|
|
|
if (expr_op == InvalidOid)
|
|
|
|
|
return NULL;
|
|
|
|
|
commuted = true;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Is the (commuted) operator an ordering operator for the opfamily? And
|
|
|
|
|
* if so, does it yield the right sorting semantics?
|
|
|
|
|
*/
|
|
|
|
|
sortfamily = get_op_opfamily_sortfamily(expr_op, opfamily);
|
|
|
|
|
if (sortfamily != pk_opfamily)
|
|
|
|
|
return NULL;
|
|
|
|
|
|
|
|
|
|
/* We have a match. Return clause or a commuted version thereof. */
|
|
|
|
|
if (commuted)
|
|
|
|
|
{
|
|
|
|
|
OpExpr *newclause = makeNode(OpExpr);
|
|
|
|
|
|
|
|
|
|
/* flat-copy all the fields of clause */
|
|
|
|
|
memcpy(newclause, clause, sizeof(OpExpr));
|
|
|
|
|
|
|
|
|
|
/* commute it */
|
|
|
|
|
newclause->opno = expr_op;
|
|
|
|
|
newclause->opfuncid = InvalidOid;
|
|
|
|
|
newclause->args = list_make2(rightop, leftop);
|
|
|
|
|
|
|
|
|
|
clause = (Expr *) newclause;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return clause;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
1996-07-09 08:22:35 +02:00
|
|
|
/****************************************************************************
|
|
|
|
|
* ---- ROUTINES TO DO PARTIAL INDEX PREDICATE TESTS ----
|
|
|
|
|
****************************************************************************/
|
|
|
|
|
|
2004-01-05 06:07:36 +01:00
|
|
|
/*
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
* check_index_predicates
|
|
|
|
|
* Set the predicate-derived IndexOptInfo fields for each index
|
|
|
|
|
* of the specified relation.
|
|
|
|
|
*
|
|
|
|
|
* predOK is set true if the index is partial and its predicate is satisfied
|
|
|
|
|
* for this query, ie the query's WHERE clauses imply the predicate.
|
|
|
|
|
*
|
|
|
|
|
* indrestrictinfo is set to the relation's baserestrictinfo list less any
|
|
|
|
|
* conditions that are implied by the index's predicate. (Obviously, for a
|
|
|
|
|
* non-partial index, this is the same as baserestrictinfo.) Such conditions
|
|
|
|
|
* can be dropped from the plan when using the index, in certain cases.
|
|
|
|
|
*
|
|
|
|
|
* At one time it was possible for this to get re-run after adding more
|
|
|
|
|
* restrictions to the rel, thus possibly letting us prove more indexes OK.
|
|
|
|
|
* That doesn't happen any more (at least not in the core code's usage),
|
|
|
|
|
* but this code still supports it in case extensions want to mess with the
|
|
|
|
|
* baserestrictinfo list. We assume that adding more restrictions can't make
|
|
|
|
|
* an index not predOK. We must recompute indrestrictinfo each time, though,
|
|
|
|
|
* to make sure any newly-added restrictions get into it if needed.
|
2004-01-05 06:07:36 +01:00
|
|
|
*/
|
|
|
|
|
void
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
check_index_predicates(PlannerInfo *root, RelOptInfo *rel)
|
2004-01-05 06:07:36 +01:00
|
|
|
{
|
2012-11-16 01:29:05 +01:00
|
|
|
List *clauselist;
|
|
|
|
|
bool have_partial;
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
bool is_target_rel;
|
2012-11-16 01:29:05 +01:00
|
|
|
Relids otherrels;
|
|
|
|
|
ListCell *lc;
|
2004-01-05 06:07:36 +01:00
|
|
|
|
Abstract logic to allow for multiple kinds of child rels.
Currently, the only type of child relation is an "other member rel",
which is the child of a baserel, but in the future joins and even
upper relations may have child rels. To facilitate that, introduce
macros that test to test for particular RelOptKind values, and use
them in various places where they help to clarify the sense of a test.
(For example, a test may allow RELOPT_OTHER_MEMBER_REL either because
it intends to allow child rels, or because it intends to allow simple
rels.)
Also, remove find_childrel_top_parent, which will not work for a
child rel that is not a baserel. Instead, add a new RelOptInfo
member top_parent_relids to track the same kind of information in a
more generic manner.
Ashutosh Bapat, slightly tweaked by me. Review and testing of the
patch set from which this was taken by Rajkumar Raghuwanshi and Rafia
Sabih.
Discussion: http://postgr.es/m/CA+TgmoagTnF2yqR3PT2rv=om=wJiZ4-A+ATwdnriTGku1CLYxA@mail.gmail.com
2017-04-04 04:41:31 +02:00
|
|
|
/* Indexes are available only on base or "other" member relations. */
|
|
|
|
|
Assert(IS_SIMPLE_REL(rel));
|
|
|
|
|
|
2012-11-16 01:29:05 +01:00
|
|
|
/*
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
* Initialize the indrestrictinfo lists to be identical to
|
|
|
|
|
* baserestrictinfo, and check whether there are any partial indexes. If
|
|
|
|
|
* not, this is all we need to do.
|
2012-11-16 01:29:05 +01:00
|
|
|
*/
|
|
|
|
|
have_partial = false;
|
|
|
|
|
foreach(lc, rel->indexlist)
|
2001-08-21 18:36:06 +02:00
|
|
|
{
|
2012-11-16 01:29:05 +01:00
|
|
|
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
|
|
|
|
|
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
index->indrestrictinfo = rel->baserestrictinfo;
|
|
|
|
|
if (index->indpred)
|
|
|
|
|
have_partial = true;
|
2012-11-16 01:29:05 +01:00
|
|
|
}
|
|
|
|
|
if (!have_partial)
|
|
|
|
|
return;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Construct a list of clauses that we can assume true for the purpose of
|
|
|
|
|
* proving the index(es) usable. Restriction clauses for the rel are
|
|
|
|
|
* always usable, and so are any join clauses that are "movable to" this
|
|
|
|
|
* rel. Also, we can consider any EC-derivable join clauses (which must
|
|
|
|
|
* be "movable to" this rel, by definition).
|
|
|
|
|
*/
|
|
|
|
|
clauselist = list_copy(rel->baserestrictinfo);
|
|
|
|
|
|
|
|
|
|
/* Scan the rel's join clauses */
|
|
|
|
|
foreach(lc, rel->joininfo)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
/* Check if clause can be moved to this rel */
|
2013-08-18 02:22:37 +02:00
|
|
|
if (!join_clause_is_movable_to(rinfo, rel))
|
2012-11-16 01:29:05 +01:00
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
clauselist = lappend(clauselist, rinfo);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Add on any equivalence-derivable join clauses. Computing the correct
|
|
|
|
|
* relid sets for generate_join_implied_equalities is slightly tricky
|
|
|
|
|
* because the rel could be a child rel rather than a true baserel, and in
|
Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees. This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.
To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle. PlaceHolderVars
receive similar decoration for the same reason.
In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos. This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.
This change affects FDWs that want to plan foreign joins. They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.
Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup. (The README additions
and comments mention some stuff that will appear in the follow-up.)
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:16:20 +01:00
|
|
|
* that case we must subtract its parents' relid(s) from all_query_rels.
|
Fix mis-handling of outer join quals generated by EquivalenceClasses.
It's possible, in admittedly-rather-contrived cases, for an eclass
to generate a derived "join" qual that constrains the post-outer-join
value(s) of some RHS variable(s) without mentioning the LHS at all.
While the mechanisms were set up to work for this, we fell foul of
the "get_common_eclass_indexes" filter installed by commit 3373c7155:
it could decide that such an eclass wasn't relevant to the join, so
that the required qual clause wouldn't get emitted there or anywhere
else.
To fix, apply get_common_eclass_indexes only at inner joins, where
its rule is still valid. At an outer join, fall back to examining all
eclasses that mention either input (or the OJ relid, though it should
be impossible for an eclass to mention that without mentioning either
input). Perhaps we can improve on that later, but the cost/benefit of
adding more complexity to skip some irrelevant eclasses is dubious.
To allow cheaply distinguishing outer from inner joins, pass the
ojrelid to generate_join_implied_equalities as a separate argument.
This also allows cleaning up some sloppiness that had crept into
the definition of its join_relids argument, and it allows accurate
calculation of nominal_join_relids for a child outer join. (The
latter oversight seems not to have been a live bug, but it certainly
could have caused problems in future.)
Also fix what might be a live bug in check_index_predicates: it was
being sloppy about what it passed to generate_join_implied_equalities.
Per report from Richard Guo.
Discussion: https://postgr.es/m/CAMbWs4-DsTBfOvXuw64GdFss2=M5cwtEhY=0DCS7t2gT7P6hSA@mail.gmail.com
2023-02-23 17:05:58 +01:00
|
|
|
* Additionally, we mustn't consider clauses that are only computable
|
|
|
|
|
* after outer joins that can null the rel.
|
2012-11-16 01:29:05 +01:00
|
|
|
*/
|
|
|
|
|
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
|
Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees. This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.
To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle. PlaceHolderVars
receive similar decoration for the same reason.
In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos. This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.
This change affects FDWs that want to plan foreign joins. They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.
Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup. (The README additions
and comments mention some stuff that will appear in the follow-up.)
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:16:20 +01:00
|
|
|
otherrels = bms_difference(root->all_query_rels,
|
Fix some more problems with nested append relations.
As of commit a87c72915 (which later got backpatched as far as 9.1),
we're explicitly supporting the notion that append relations can be
nested; this can occur when UNION ALL constructs are nested, or when
a UNION ALL contains a table with inheritance children.
Bug #11457 from Nelson Page, as well as an earlier report from Elvis
Pranskevichus, showed that there were still nasty bugs associated with such
cases: in particular the EquivalenceClass mechanism could try to generate
"join" clauses connecting an appendrel child to some grandparent appendrel,
which would result in assertion failures or bogus plans.
Upon investigation I concluded that all current callers of
find_childrel_appendrelinfo() need to be fixed to explicitly consider
multiple levels of parent appendrels. The most complex fix was in
processing of "broken" EquivalenceClasses, which are ECs for which we have
been unable to generate all the derived equality clauses we would like to
because of missing cross-type equality operators in the underlying btree
operator family. That code path is more or less entirely untested by
the regression tests to date, because no standard opfamilies have such
holes in them. So I wrote a new regression test script to try to exercise
it a bit, which turned out to be quite a worthwhile activity as it exposed
existing bugs in all supported branches.
The present patch is essentially the same as far back as 9.2, which is
where parameterized paths were introduced. In 9.0 and 9.1, we only need
to back-patch a small fragment of commit 5b7b5518d, which fixes failure to
propagate out the original WHERE clauses when a broken EC contains constant
members. (The regression test case results show that these older branches
are noticeably stupider than 9.2+ in terms of the quality of the plans
generated; but we don't really care about plan quality in such cases,
only that the plan not be outright wrong. A more invasive fix in the
older branches would not be a good idea anyway from a plan-stability
standpoint.)
2014-10-02 01:30:24 +02:00
|
|
|
find_childrel_parents(root, rel));
|
2012-11-16 01:29:05 +01:00
|
|
|
else
|
Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees. This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.
To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle. PlaceHolderVars
receive similar decoration for the same reason.
In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos. This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.
This change affects FDWs that want to plan foreign joins. They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.
Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup. (The README additions
and comments mention some stuff that will appear in the follow-up.)
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:16:20 +01:00
|
|
|
otherrels = bms_difference(root->all_query_rels, rel->relids);
|
Fix mis-handling of outer join quals generated by EquivalenceClasses.
It's possible, in admittedly-rather-contrived cases, for an eclass
to generate a derived "join" qual that constrains the post-outer-join
value(s) of some RHS variable(s) without mentioning the LHS at all.
While the mechanisms were set up to work for this, we fell foul of
the "get_common_eclass_indexes" filter installed by commit 3373c7155:
it could decide that such an eclass wasn't relevant to the join, so
that the required qual clause wouldn't get emitted there or anywhere
else.
To fix, apply get_common_eclass_indexes only at inner joins, where
its rule is still valid. At an outer join, fall back to examining all
eclasses that mention either input (or the OJ relid, though it should
be impossible for an eclass to mention that without mentioning either
input). Perhaps we can improve on that later, but the cost/benefit of
adding more complexity to skip some irrelevant eclasses is dubious.
To allow cheaply distinguishing outer from inner joins, pass the
ojrelid to generate_join_implied_equalities as a separate argument.
This also allows cleaning up some sloppiness that had crept into
the definition of its join_relids argument, and it allows accurate
calculation of nominal_join_relids for a child outer join. (The
latter oversight seems not to have been a live bug, but it certainly
could have caused problems in future.)
Also fix what might be a live bug in check_index_predicates: it was
being sloppy about what it passed to generate_join_implied_equalities.
Per report from Richard Guo.
Discussion: https://postgr.es/m/CAMbWs4-DsTBfOvXuw64GdFss2=M5cwtEhY=0DCS7t2gT7P6hSA@mail.gmail.com
2023-02-23 17:05:58 +01:00
|
|
|
otherrels = bms_del_members(otherrels, rel->nulling_relids);
|
2012-11-16 01:29:05 +01:00
|
|
|
|
|
|
|
|
if (!bms_is_empty(otherrels))
|
|
|
|
|
clauselist =
|
|
|
|
|
list_concat(clauselist,
|
|
|
|
|
generate_join_implied_equalities(root,
|
|
|
|
|
bms_union(rel->relids,
|
|
|
|
|
otherrels),
|
|
|
|
|
otherrels,
|
Fix mis-handling of outer join quals generated by EquivalenceClasses.
It's possible, in admittedly-rather-contrived cases, for an eclass
to generate a derived "join" qual that constrains the post-outer-join
value(s) of some RHS variable(s) without mentioning the LHS at all.
While the mechanisms were set up to work for this, we fell foul of
the "get_common_eclass_indexes" filter installed by commit 3373c7155:
it could decide that such an eclass wasn't relevant to the join, so
that the required qual clause wouldn't get emitted there or anywhere
else.
To fix, apply get_common_eclass_indexes only at inner joins, where
its rule is still valid. At an outer join, fall back to examining all
eclasses that mention either input (or the OJ relid, though it should
be impossible for an eclass to mention that without mentioning either
input). Perhaps we can improve on that later, but the cost/benefit of
adding more complexity to skip some irrelevant eclasses is dubious.
To allow cheaply distinguishing outer from inner joins, pass the
ojrelid to generate_join_implied_equalities as a separate argument.
This also allows cleaning up some sloppiness that had crept into
the definition of its join_relids argument, and it allows accurate
calculation of nominal_join_relids for a child outer join. (The
latter oversight seems not to have been a live bug, but it certainly
could have caused problems in future.)
Also fix what might be a live bug in check_index_predicates: it was
being sloppy about what it passed to generate_join_implied_equalities.
Per report from Richard Guo.
Discussion: https://postgr.es/m/CAMbWs4-DsTBfOvXuw64GdFss2=M5cwtEhY=0DCS7t2gT7P6hSA@mail.gmail.com
2023-02-23 17:05:58 +01:00
|
|
|
rel,
|
2023-05-17 17:13:52 +02:00
|
|
|
NULL));
|
2012-11-16 01:29:05 +01:00
|
|
|
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
/*
|
|
|
|
|
* Normally we remove quals that are implied by a partial index's
|
|
|
|
|
* predicate from indrestrictinfo, indicating that they need not be
|
|
|
|
|
* checked explicitly by an indexscan plan using this index. However, if
|
2022-10-24 12:52:43 +02:00
|
|
|
* the rel is a target relation of UPDATE/DELETE/MERGE/SELECT FOR UPDATE,
|
|
|
|
|
* we cannot remove such quals from the plan, because they need to be in
|
|
|
|
|
* the plan so that they will be properly rechecked by EvalPlanQual
|
|
|
|
|
* testing. Some day we might want to remove such quals from the main
|
|
|
|
|
* plan anyway and pass them through to EvalPlanQual via a side channel;
|
|
|
|
|
* but for now, we just don't remove implied quals at all for target
|
|
|
|
|
* relations.
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
*/
|
Rework planning and execution of UPDATE and DELETE.
This patch makes two closely related sets of changes:
1. For UPDATE, the subplan of the ModifyTable node now only delivers
the new values of the changed columns (i.e., the expressions computed
in the query's SET clause) plus row identity information such as CTID.
ModifyTable must re-fetch the original tuple to merge in the old
values of any unchanged columns. The core advantage of this is that
the changed columns are uniform across all tables of an inherited or
partitioned target relation, whereas the other columns might not be.
A secondary advantage, when the UPDATE involves joins, is that less
data needs to pass through the plan tree. The disadvantage of course
is an extra fetch of each tuple to be updated. However, that seems to
be very nearly free in context; even worst-case tests don't show it to
add more than a couple percent to the total query cost. At some point
it might be interesting to combine the re-fetch with the tuple access
that ModifyTable must do anyway to mark the old tuple dead; but that
would require a good deal of refactoring and it seems it wouldn't buy
all that much, so this patch doesn't attempt it.
2. For inherited UPDATE/DELETE, instead of generating a separate
subplan for each target relation, we now generate a single subplan
that is just exactly like a SELECT's plan, then stick ModifyTable
on top of that. To let ModifyTable know which target relation a
given incoming row refers to, a tableoid junk column is added to
the row identity information. This gets rid of the horrid hack
that was inheritance_planner(), eliminating O(N^2) planning cost
and memory consumption in cases where there were many unprunable
target relations.
Point 2 of course requires point 1, so that there is a uniform
definition of the non-junk columns to be returned by the subplan.
We can't insist on uniform definition of the row identity junk
columns however, if we want to keep the ability to have both
plain and foreign tables in a partitioning hierarchy. Since
it wouldn't scale very far to have every child table have its
own row identity column, this patch includes provisions to merge
similar row identity columns into one column of the subplan result.
In particular, we can merge the whole-row Vars typically used as
row identity by FDWs into one column by pretending they are type
RECORD. (It's still okay for the actual composite Datums to be
labeled with the table's rowtype OID, though.)
There is more that can be done to file down residual inefficiencies
in this patch, but it seems to be committable now.
FDW authors should note several API changes:
* The argument list for AddForeignUpdateTargets() has changed, and so
has the method it must use for adding junk columns to the query. Call
add_row_identity_var() instead of manipulating the parse tree directly.
You might want to reconsider exactly what you're adding, too.
* PlanDirectModify() must now work a little harder to find the
ForeignScan plan node; if the foreign table is part of a partitioning
hierarchy then the ForeignScan might not be the direct child of
ModifyTable. See postgres_fdw for sample code.
* To check whether a relation is a target relation, it's no
longer sufficient to compare its relid to root->parse->resultRelation.
Instead, check it against all_result_relids or leaf_result_relids,
as appropriate.
Amit Langote and Tom Lane
Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
|
|
|
is_target_rel = (bms_is_member(rel->relid, root->all_result_relids) ||
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
get_plan_rowmark(root->rowMarks, rel->relid) != NULL);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Now try to prove each index predicate true, and compute the
|
|
|
|
|
* indrestrictinfo lists for partial indexes. Note that we compute the
|
|
|
|
|
* indrestrictinfo list even for non-predOK indexes; this might seem
|
|
|
|
|
* wasteful, but we may be able to use such indexes in OR clauses, cf
|
|
|
|
|
* generate_bitmap_or_paths().
|
|
|
|
|
*/
|
2012-11-16 01:29:05 +01:00
|
|
|
foreach(lc, rel->indexlist)
|
|
|
|
|
{
|
|
|
|
|
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
ListCell *lcr;
|
2002-12-15 17:17:59 +01:00
|
|
|
|
2005-06-11 00:25:37 +02:00
|
|
|
if (index->indpred == NIL)
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
continue; /* ignore non-partial indexes here */
|
2000-08-24 05:29:15 +02:00
|
|
|
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
if (!index->predOK) /* don't repeat work if already proven OK */
|
2017-06-14 19:13:11 +02:00
|
|
|
index->predOK = predicate_implied_by(index->indpred, clauselist,
|
|
|
|
|
false);
|
2009-02-15 21:16:21 +01:00
|
|
|
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
/* If rel is an update target, leave indrestrictinfo as set above */
|
|
|
|
|
if (is_target_rel)
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/* Else compute indrestrictinfo as the non-implied quals */
|
|
|
|
|
index->indrestrictinfo = NIL;
|
|
|
|
|
foreach(lcr, rel->baserestrictinfo)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lcr);
|
|
|
|
|
|
|
|
|
|
/* predicate_implied_by() assumes first arg is immutable */
|
|
|
|
|
if (contain_mutable_functions((Node *) rinfo->clause) ||
|
|
|
|
|
!predicate_implied_by(list_make1(rinfo->clause),
|
2017-06-14 19:13:11 +02:00
|
|
|
index->indpred, false))
|
Support using index-only scans with partial indexes in more cases.
Previously, the planner would reject an index-only scan if any restriction
clause for its table used a column not available from the index, even
if that restriction clause would later be dropped from the plan entirely
because it's implied by the index's predicate. This is a fairly common
situation for partial indexes because predicates using columns not included
in the index are often the most useful kind of predicate, and we have to
duplicate (or at least imply) the predicate in the WHERE clause in order
to get the index to be considered at all. So index-only scans were
essentially unavailable with such partial indexes.
To fix, we have to do detection of implied-by-predicate clauses much
earlier in the planner. This patch puts it in check_index_predicates
(nee check_partial_indexes), meaning it gets done for every partial index,
whereas we previously only considered this issue at createplan time,
so that the work was only done for an index actually selected for use.
That could result in a noticeable planning slowdown for queries against
tables with many partial indexes. However, testing suggested that there
isn't really a significant cost, especially not with reasonable numbers
of partial indexes. We do get a small additional benefit, which is that
cost_index is more accurate since it correctly discounts the evaluation
cost of clauses that will be removed. We can also avoid considering such
clauses as potential indexquals, which saves useless matching cycles in
the case where the predicate columns aren't in the index, and prevents
generating bogus plans that double-count the clause's selectivity when
the columns are in the index.
Tomas Vondra and Kyotaro Horiguchi, reviewed by Kevin Grittner and
Konstantin Knizhnik, and whacked around a little by me
2016-03-31 20:48:56 +02:00
|
|
|
index->indrestrictinfo = lappend(index->indrestrictinfo, rinfo);
|
|
|
|
|
}
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/****************************************************************************
|
2012-01-28 01:26:38 +01:00
|
|
|
* ---- ROUTINES TO CHECK EXTERNALLY-VISIBLE CONDITIONS ----
|
1996-07-09 08:22:35 +02:00
|
|
|
****************************************************************************/
|
|
|
|
|
|
|
|
|
|
/*
|
2013-03-22 00:43:59 +01:00
|
|
|
* ec_member_matches_indexcol
|
2012-01-28 01:26:38 +01:00
|
|
|
* Test whether an EquivalenceClass member matches an index column.
|
2007-01-20 21:45:41 +01:00
|
|
|
*
|
2013-03-22 00:43:59 +01:00
|
|
|
* This is a callback for use by generate_implied_equalities_for_column.
|
2007-01-20 21:45:41 +01:00
|
|
|
*/
|
2013-03-22 00:43:59 +01:00
|
|
|
static bool
|
|
|
|
|
ec_member_matches_indexcol(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
EquivalenceClass *ec, EquivalenceMember *em,
|
|
|
|
|
void *arg)
|
2002-11-24 22:52:15 +01:00
|
|
|
{
|
2013-03-22 00:43:59 +01:00
|
|
|
IndexOptInfo *index = ((ec_member_matches_arg *) arg)->index;
|
|
|
|
|
int indexcol = ((ec_member_matches_arg *) arg)->indexcol;
|
2018-04-12 15:37:22 +02:00
|
|
|
Oid curFamily;
|
|
|
|
|
Oid curCollation;
|
|
|
|
|
|
|
|
|
|
Assert(indexcol < index->nkeycolumns);
|
|
|
|
|
|
|
|
|
|
curFamily = index->opfamily[indexcol];
|
|
|
|
|
curCollation = index->indexcollations[indexcol];
|
2002-11-24 22:52:15 +01:00
|
|
|
|
2007-01-20 21:45:41 +01:00
|
|
|
/*
|
2012-01-28 01:26:38 +01:00
|
|
|
* If it's a btree index, we can reject it if its opfamily isn't
|
|
|
|
|
* compatible with the EC, since no clause generated from the EC could be
|
|
|
|
|
* used with the index. For non-btree indexes, we can't easily tell
|
|
|
|
|
* whether clauses generated from the EC could be used with the index, so
|
|
|
|
|
* don't check the opfamily. This might mean we return "true" for a
|
|
|
|
|
* useless EC, so we have to recheck the results of
|
2013-03-22 00:43:59 +01:00
|
|
|
* generate_implied_equalities_for_column; see
|
2012-01-28 01:26:38 +01:00
|
|
|
* match_eclass_clauses_to_index.
|
2007-01-20 21:45:41 +01:00
|
|
|
*/
|
2012-01-28 01:26:38 +01:00
|
|
|
if (index->relam == BTREE_AM_OID &&
|
|
|
|
|
!list_member_oid(ec->ec_opfamilies, curFamily))
|
|
|
|
|
return false;
|
2002-11-24 22:52:15 +01:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
/* We insist on collation match for all index types, though */
|
|
|
|
|
if (!IndexCollMatchesExprColl(curCollation, ec->ec_collation))
|
|
|
|
|
return false;
|
2006-04-09 20:18:41 +02:00
|
|
|
|
2012-01-28 01:26:38 +01:00
|
|
|
return match_index_to_operand((Node *) em->em_expr, indexcol, index);
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
|
|
|
|
|
2009-09-17 22:49:29 +02:00
|
|
|
/*
|
|
|
|
|
* relation_has_unique_index_for
|
|
|
|
|
* Determine whether the relation provably has at most one row satisfying
|
|
|
|
|
* a set of equality conditions, because the conditions constrain all
|
|
|
|
|
* columns of some unique index.
|
|
|
|
|
*
|
2011-10-26 23:52:02 +02:00
|
|
|
* The conditions can be represented in either or both of two ways:
|
|
|
|
|
* 1. A list of RestrictInfo nodes, where the caller has already determined
|
|
|
|
|
* that each condition is a mergejoinable equality with an expression in
|
|
|
|
|
* this relation on one side, and an expression not involving this relation
|
|
|
|
|
* on the other. The transient outer_is_left flag is used to identify which
|
|
|
|
|
* side we should look at: left side if outer_is_left is false, right side
|
|
|
|
|
* if it is true.
|
|
|
|
|
* 2. A list of expressions in this relation, and a corresponding list of
|
|
|
|
|
* equality operators. The caller must have already checked that the operators
|
|
|
|
|
* represent equality. (Note: the operators could be cross-type; the
|
|
|
|
|
* expressions should correspond to their RHS inputs.)
|
|
|
|
|
*
|
|
|
|
|
* The caller need only supply equality conditions arising from joins;
|
|
|
|
|
* this routine automatically adds in any usable baserestrictinfo clauses.
|
|
|
|
|
* (Note that the passed-in restrictlist will be destructively modified!)
|
2009-09-17 22:49:29 +02:00
|
|
|
*/
|
|
|
|
|
bool
|
|
|
|
|
relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
|
2011-10-26 23:52:02 +02:00
|
|
|
List *restrictlist,
|
|
|
|
|
List *exprlist, List *oprlist)
|
Remove useless self-joins
The Self Join Elimination (SJE) feature removes an inner join of a plain table
to itself in the query tree if is proved that the join can be replaced with
a scan without impacting the query result. Self join and inner relation are
replaced with the outer in query, equivalence classes, and planner info
structures. Also, inner restrictlist moves to the outer one with removing
duplicated clauses. Thus, this optimization reduces the length of the range
table list (this especially makes sense for partitioned relations), reduces
the number of restriction clauses === selectivity estimations, and potentially
can improve total planner prediction for the query.
The SJE proof is based on innerrel_is_unique machinery.
We can remove a self-join when for each outer row:
1. At most one inner row matches the join clause.
2. Each matched inner row must be (physically) the same row as the outer one.
In this patch we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x
2. Add to the list above the baseretrictinfo of the inner table.
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables.
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove
the possibility of self-join elimination inner and outer clauses must have
an exact match.
The relation replacement procedure is not trivial and it is partly combined
with the one, used to remove useless left joins. Tests, covering this feature,
were added to join.sql. Some regression tests changed due to self-join removal
logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov, Alexander Kuzmenkov
Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz
Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas
Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu
Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina
Reviewed-by: Alexander Korotkov
2023-10-25 11:46:22 +02:00
|
|
|
{
|
|
|
|
|
return relation_has_unique_index_ext(root, rel, restrictlist,
|
|
|
|
|
exprlist, oprlist, NULL);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* relation_has_unique_index_ext
|
|
|
|
|
* Same as relation_has_unique_index_for(), but supports extra_clauses
|
|
|
|
|
* parameter. If extra_clauses isn't NULL, return baserestrictinfo clauses
|
|
|
|
|
* which were used to derive uniqueness.
|
|
|
|
|
*/
|
|
|
|
|
bool
|
|
|
|
|
relation_has_unique_index_ext(PlannerInfo *root, RelOptInfo *rel,
|
|
|
|
|
List *restrictlist,
|
|
|
|
|
List *exprlist, List *oprlist,
|
|
|
|
|
List **extra_clauses)
|
2009-09-17 22:49:29 +02:00
|
|
|
{
|
|
|
|
|
ListCell *ic;
|
|
|
|
|
|
2011-10-26 23:52:02 +02:00
|
|
|
Assert(list_length(exprlist) == list_length(oprlist));
|
|
|
|
|
|
|
|
|
|
/* Short-circuit if no indexes... */
|
|
|
|
|
if (rel->indexlist == NIL)
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Examine the rel's restriction clauses for usable var = const clauses
|
|
|
|
|
* that we can add to the restrictlist.
|
|
|
|
|
*/
|
|
|
|
|
foreach(ic, rel->baserestrictinfo)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(ic);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Note: can_join won't be set for a restriction clause, but
|
|
|
|
|
* mergeopfamilies will be if it has a mergejoinable operator and
|
|
|
|
|
* doesn't contain volatile functions.
|
|
|
|
|
*/
|
|
|
|
|
if (restrictinfo->mergeopfamilies == NIL)
|
|
|
|
|
continue; /* not mergejoinable */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The clause certainly doesn't refer to anything but the given rel.
|
|
|
|
|
* If either side is pseudoconstant then we can use it.
|
|
|
|
|
*/
|
|
|
|
|
if (bms_is_empty(restrictinfo->left_relids))
|
|
|
|
|
{
|
|
|
|
|
/* righthand side is inner */
|
|
|
|
|
restrictinfo->outer_is_left = true;
|
|
|
|
|
}
|
|
|
|
|
else if (bms_is_empty(restrictinfo->right_relids))
|
|
|
|
|
{
|
|
|
|
|
/* lefthand side is inner */
|
|
|
|
|
restrictinfo->outer_is_left = false;
|
|
|
|
|
}
|
|
|
|
|
else
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/* OK, add to list */
|
|
|
|
|
restrictlist = lappend(restrictlist, restrictinfo);
|
|
|
|
|
}
|
|
|
|
|
|
2009-09-17 22:49:29 +02:00
|
|
|
/* Short-circuit the easy case */
|
2011-10-26 23:52:02 +02:00
|
|
|
if (restrictlist == NIL && exprlist == NIL)
|
2009-09-17 22:49:29 +02:00
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/* Examine each index of the relation ... */
|
|
|
|
|
foreach(ic, rel->indexlist)
|
|
|
|
|
{
|
|
|
|
|
IndexOptInfo *ind = (IndexOptInfo *) lfirst(ic);
|
|
|
|
|
int c;
|
Remove useless self-joins
The Self Join Elimination (SJE) feature removes an inner join of a plain table
to itself in the query tree if is proved that the join can be replaced with
a scan without impacting the query result. Self join and inner relation are
replaced with the outer in query, equivalence classes, and planner info
structures. Also, inner restrictlist moves to the outer one with removing
duplicated clauses. Thus, this optimization reduces the length of the range
table list (this especially makes sense for partitioned relations), reduces
the number of restriction clauses === selectivity estimations, and potentially
can improve total planner prediction for the query.
The SJE proof is based on innerrel_is_unique machinery.
We can remove a self-join when for each outer row:
1. At most one inner row matches the join clause.
2. Each matched inner row must be (physically) the same row as the outer one.
In this patch we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x
2. Add to the list above the baseretrictinfo of the inner table.
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables.
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove
the possibility of self-join elimination inner and outer clauses must have
an exact match.
The relation replacement procedure is not trivial and it is partly combined
with the one, used to remove useless left joins. Tests, covering this feature,
were added to join.sql. Some regression tests changed due to self-join removal
logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov, Alexander Kuzmenkov
Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz
Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas
Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu
Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina
Reviewed-by: Alexander Korotkov
2023-10-25 11:46:22 +02:00
|
|
|
List *exprs = NIL;
|
2009-09-17 22:49:29 +02:00
|
|
|
|
|
|
|
|
/*
|
2011-10-23 06:43:39 +02:00
|
|
|
* If the index is not unique, or not immediately enforced, or if it's
|
2023-06-19 03:00:42 +02:00
|
|
|
* a partial index, it's useless here. We're unable to make use of
|
|
|
|
|
* predOK partial unique indexes due to the fact that
|
|
|
|
|
* check_index_predicates() also makes use of join predicates to
|
|
|
|
|
* determine if the partial index is usable. Here we need proofs that
|
|
|
|
|
* hold true before any joins are evaluated.
|
2009-09-17 22:49:29 +02:00
|
|
|
*/
|
2023-06-19 03:00:42 +02:00
|
|
|
if (!ind->unique || !ind->immediate || ind->indpred != NIL)
|
2009-09-17 22:49:29 +02:00
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
2011-10-26 23:52:02 +02:00
|
|
|
* Try to find each index column in the lists of conditions. This is
|
|
|
|
|
* O(N^2) or worse, but we expect all the lists to be short.
|
2009-09-17 22:49:29 +02:00
|
|
|
*/
|
2019-02-13 00:38:32 +01:00
|
|
|
for (c = 0; c < ind->nkeycolumns; c++)
|
2009-09-17 22:49:29 +02:00
|
|
|
{
|
2011-10-26 23:52:02 +02:00
|
|
|
bool matched = false;
|
2009-09-17 22:49:29 +02:00
|
|
|
ListCell *lc;
|
2011-10-26 23:52:02 +02:00
|
|
|
ListCell *lc2;
|
2009-09-17 22:49:29 +02:00
|
|
|
|
|
|
|
|
foreach(lc, restrictlist)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
Node *rexpr;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The condition's equality operator must be a member of the
|
|
|
|
|
* index opfamily, else it is not asserting the right kind of
|
|
|
|
|
* equality behavior for this index. We check this first
|
|
|
|
|
* since it's probably cheaper than match_index_to_operand().
|
|
|
|
|
*/
|
|
|
|
|
if (!list_member_oid(rinfo->mergeopfamilies, ind->opfamily[c]))
|
|
|
|
|
continue;
|
|
|
|
|
|
2011-04-09 01:19:17 +02:00
|
|
|
/*
|
|
|
|
|
* XXX at some point we may need to check collations here too.
|
|
|
|
|
* For the moment we assume all collations reduce to the same
|
|
|
|
|
* notion of equality.
|
|
|
|
|
*/
|
|
|
|
|
|
2009-09-17 22:49:29 +02:00
|
|
|
/* OK, see if the condition operand matches the index key */
|
|
|
|
|
if (rinfo->outer_is_left)
|
|
|
|
|
rexpr = get_rightop(rinfo->clause);
|
|
|
|
|
else
|
|
|
|
|
rexpr = get_leftop(rinfo->clause);
|
|
|
|
|
|
|
|
|
|
if (match_index_to_operand(rexpr, c, ind))
|
2011-10-26 23:52:02 +02:00
|
|
|
{
|
|
|
|
|
matched = true; /* column is unique */
|
Remove useless self-joins
The Self Join Elimination (SJE) feature removes an inner join of a plain table
to itself in the query tree if is proved that the join can be replaced with
a scan without impacting the query result. Self join and inner relation are
replaced with the outer in query, equivalence classes, and planner info
structures. Also, inner restrictlist moves to the outer one with removing
duplicated clauses. Thus, this optimization reduces the length of the range
table list (this especially makes sense for partitioned relations), reduces
the number of restriction clauses === selectivity estimations, and potentially
can improve total planner prediction for the query.
The SJE proof is based on innerrel_is_unique machinery.
We can remove a self-join when for each outer row:
1. At most one inner row matches the join clause.
2. Each matched inner row must be (physically) the same row as the outer one.
In this patch we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x
2. Add to the list above the baseretrictinfo of the inner table.
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables.
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove
the possibility of self-join elimination inner and outer clauses must have
an exact match.
The relation replacement procedure is not trivial and it is partly combined
with the one, used to remove useless left joins. Tests, covering this feature,
were added to join.sql. Some regression tests changed due to self-join removal
logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov, Alexander Kuzmenkov
Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz
Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas
Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu
Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina
Reviewed-by: Alexander Korotkov
2023-10-25 11:46:22 +02:00
|
|
|
|
|
|
|
|
if (bms_membership(rinfo->clause_relids) == BMS_SINGLETON)
|
|
|
|
|
{
|
|
|
|
|
MemoryContext oldMemCtx =
|
|
|
|
|
MemoryContextSwitchTo(root->planner_cxt);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Add filter clause into a list allowing caller to
|
|
|
|
|
* know if uniqueness have made not only by join
|
|
|
|
|
* clauses.
|
|
|
|
|
*/
|
|
|
|
|
Assert(bms_is_empty(rinfo->left_relids) ||
|
|
|
|
|
bms_is_empty(rinfo->right_relids));
|
|
|
|
|
if (extra_clauses)
|
|
|
|
|
exprs = lappend(exprs, rinfo);
|
|
|
|
|
MemoryContextSwitchTo(oldMemCtx);
|
|
|
|
|
}
|
|
|
|
|
|
2011-10-26 23:52:02 +02:00
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
if (matched)
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
forboth(lc, exprlist, lc2, oprlist)
|
|
|
|
|
{
|
|
|
|
|
Node *expr = (Node *) lfirst(lc);
|
|
|
|
|
Oid opr = lfirst_oid(lc2);
|
|
|
|
|
|
|
|
|
|
/* See if the expression matches the index key */
|
|
|
|
|
if (!match_index_to_operand(expr, c, ind))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* The equality operator must be a member of the index
|
|
|
|
|
* opfamily, else it is not asserting the right kind of
|
|
|
|
|
* equality behavior for this index. We assume the caller
|
|
|
|
|
* determined it is an equality operator, so we don't need to
|
|
|
|
|
* check any more tightly than this.
|
|
|
|
|
*/
|
|
|
|
|
if (!op_in_opfamily(opr, ind->opfamily[c]))
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* XXX at some point we may need to check collations here too.
|
|
|
|
|
* For the moment we assume all collations reduce to the same
|
|
|
|
|
* notion of equality.
|
|
|
|
|
*/
|
|
|
|
|
|
|
|
|
|
matched = true; /* column is unique */
|
|
|
|
|
break;
|
2009-09-17 22:49:29 +02:00
|
|
|
}
|
|
|
|
|
|
2011-10-26 23:52:02 +02:00
|
|
|
if (!matched)
|
2009-09-17 22:49:29 +02:00
|
|
|
break; /* no match; this index doesn't help us */
|
|
|
|
|
}
|
|
|
|
|
|
2019-02-13 00:38:32 +01:00
|
|
|
/* Matched all key columns of this index? */
|
|
|
|
|
if (c == ind->nkeycolumns)
|
Remove useless self-joins
The Self Join Elimination (SJE) feature removes an inner join of a plain table
to itself in the query tree if is proved that the join can be replaced with
a scan without impacting the query result. Self join and inner relation are
replaced with the outer in query, equivalence classes, and planner info
structures. Also, inner restrictlist moves to the outer one with removing
duplicated clauses. Thus, this optimization reduces the length of the range
table list (this especially makes sense for partitioned relations), reduces
the number of restriction clauses === selectivity estimations, and potentially
can improve total planner prediction for the query.
The SJE proof is based on innerrel_is_unique machinery.
We can remove a self-join when for each outer row:
1. At most one inner row matches the join clause.
2. Each matched inner row must be (physically) the same row as the outer one.
In this patch we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x
2. Add to the list above the baseretrictinfo of the inner table.
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables.
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove
the possibility of self-join elimination inner and outer clauses must have
an exact match.
The relation replacement procedure is not trivial and it is partly combined
with the one, used to remove useless left joins. Tests, covering this feature,
were added to join.sql. Some regression tests changed due to self-join removal
logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov, Alexander Kuzmenkov
Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz
Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas
Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu
Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina
Reviewed-by: Alexander Korotkov
2023-10-25 11:46:22 +02:00
|
|
|
{
|
|
|
|
|
if (extra_clauses)
|
|
|
|
|
*extra_clauses = exprs;
|
2009-09-17 22:49:29 +02:00
|
|
|
return true;
|
Remove useless self-joins
The Self Join Elimination (SJE) feature removes an inner join of a plain table
to itself in the query tree if is proved that the join can be replaced with
a scan without impacting the query result. Self join and inner relation are
replaced with the outer in query, equivalence classes, and planner info
structures. Also, inner restrictlist moves to the outer one with removing
duplicated clauses. Thus, this optimization reduces the length of the range
table list (this especially makes sense for partitioned relations), reduces
the number of restriction clauses === selectivity estimations, and potentially
can improve total planner prediction for the query.
The SJE proof is based on innerrel_is_unique machinery.
We can remove a self-join when for each outer row:
1. At most one inner row matches the join clause.
2. Each matched inner row must be (physically) the same row as the outer one.
In this patch we use the next approach to identify a self-join:
1. Collect all merge-joinable join quals which look like a.x = b.x
2. Add to the list above the baseretrictinfo of the inner table.
3. Check innerrel_is_unique() for the qual list. If it returns false, skip
this pair of joining tables.
4. Check uniqueness, proved by the baserestrictinfo clauses. To prove
the possibility of self-join elimination inner and outer clauses must have
an exact match.
The relation replacement procedure is not trivial and it is partly combined
with the one, used to remove useless left joins. Tests, covering this feature,
were added to join.sql. Some regression tests changed due to self-join removal
logic.
Discussion: https://postgr.es/m/flat/64486b0b-0404-e39e-322d-0801154901f3%40postgrespro.ru
Author: Andrey Lepikhov, Alexander Kuzmenkov
Reviewed-by: Tom Lane, Robert Haas, Andres Freund, Simon Riggs, Jonathan S. Katz
Reviewed-by: David Rowley, Thomas Munro, Konstantin Knizhnik, Heikki Linnakangas
Reviewed-by: Hywel Carver, Laurenz Albe, Ronan Dunklau, vignesh C, Zhihong Yu
Reviewed-by: Greg Stark, Jaime Casanova, Michał Kłeczek, Alena Rybakina
Reviewed-by: Alexander Korotkov
2023-10-25 11:46:22 +02:00
|
|
|
}
|
2009-09-17 22:49:29 +02:00
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
2017-01-15 20:09:35 +01:00
|
|
|
/*
|
|
|
|
|
* indexcol_is_bool_constant_for_query
|
|
|
|
|
*
|
|
|
|
|
* If an index column is constrained to have a constant value by the query's
|
|
|
|
|
* WHERE conditions, then it's irrelevant for sort-order considerations.
|
|
|
|
|
* Usually that means we have a restriction clause WHERE indexcol = constant,
|
|
|
|
|
* which gets turned into an EquivalenceClass containing a constant, which
|
|
|
|
|
* is recognized as redundant by build_index_pathkeys(). But if the index
|
|
|
|
|
* column is a boolean variable (or expression), then we are not going to
|
|
|
|
|
* see WHERE indexcol = constant, because expression preprocessing will have
|
|
|
|
|
* simplified that to "WHERE indexcol" or "WHERE NOT indexcol". So we are not
|
|
|
|
|
* going to have a matching EquivalenceClass (unless the query also contains
|
|
|
|
|
* "ORDER BY indexcol"). To allow such cases to work the same as they would
|
|
|
|
|
* for non-boolean values, this function is provided to detect whether the
|
|
|
|
|
* specified index column matches a boolean restriction clause.
|
|
|
|
|
*/
|
|
|
|
|
bool
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
indexcol_is_bool_constant_for_query(PlannerInfo *root,
|
|
|
|
|
IndexOptInfo *index,
|
|
|
|
|
int indexcol)
|
2017-01-15 20:09:35 +01:00
|
|
|
{
|
|
|
|
|
ListCell *lc;
|
|
|
|
|
|
|
|
|
|
/* If the index isn't boolean, we can't possibly get a match */
|
|
|
|
|
if (!IsBooleanOpfamily(index->opfamily[indexcol]))
|
|
|
|
|
return false;
|
|
|
|
|
|
|
|
|
|
/* Check each restriction clause for the index's rel */
|
|
|
|
|
foreach(lc, index->rel->baserestrictinfo)
|
|
|
|
|
{
|
|
|
|
|
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* As in match_clause_to_indexcol, never match pseudoconstants to
|
|
|
|
|
* indexes. (It might be semantically okay to do so here, but the
|
|
|
|
|
* odds of getting a match are negligible, so don't waste the cycles.)
|
|
|
|
|
*/
|
|
|
|
|
if (rinfo->pseudoconstant)
|
|
|
|
|
continue;
|
|
|
|
|
|
|
|
|
|
/* See if we can match the clause's expression to the index column */
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
if (match_boolean_index_clause(root, rinfo, indexcol, index))
|
2017-01-15 20:09:35 +01:00
|
|
|
return true;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
2005-06-14 06:04:30 +02:00
|
|
|
|
1999-07-23 05:34:49 +02:00
|
|
|
/****************************************************************************
|
|
|
|
|
* ---- ROUTINES TO CHECK OPERANDS ----
|
|
|
|
|
****************************************************************************/
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* match_index_to_operand()
|
|
|
|
|
* Generalized test for a match between an index's key
|
|
|
|
|
* and the operand on one side of a restriction or join clause.
|
2003-05-28 18:04:02 +02:00
|
|
|
*
|
|
|
|
|
* operand: the nodetree to be compared to the index
|
|
|
|
|
* indexcol: the column number of the index (counting from 0)
|
|
|
|
|
* index: the index of interest
|
2011-04-09 01:19:17 +02:00
|
|
|
*
|
|
|
|
|
* Note that we aren't interested in collations here; the caller must check
|
|
|
|
|
* for a collation match, if it's dealing with an operator where that matters.
|
2012-01-28 01:26:38 +01:00
|
|
|
*
|
|
|
|
|
* This is exported for use in selfuncs.c.
|
1999-07-23 05:34:49 +02:00
|
|
|
*/
|
2005-04-12 01:06:57 +02:00
|
|
|
bool
|
2003-05-28 18:04:02 +02:00
|
|
|
match_index_to_operand(Node *operand,
|
|
|
|
|
int indexcol,
|
2000-01-09 01:26:47 +01:00
|
|
|
IndexOptInfo *index)
|
1996-07-09 08:22:35 +02:00
|
|
|
{
|
2003-05-28 18:04:02 +02:00
|
|
|
int indkey;
|
|
|
|
|
|
2000-08-13 04:50:35 +02:00
|
|
|
/*
|
2003-05-28 18:04:02 +02:00
|
|
|
* Ignore any RelabelType node above the operand. This is needed to be
|
2000-08-13 04:50:35 +02:00
|
|
|
* able to apply indexscanning in binary-compatible-operator cases. Note:
|
|
|
|
|
* we can assume there is at most one RelabelType node;
|
|
|
|
|
* eval_const_expressions() will have simplified if more than one.
|
|
|
|
|
*/
|
|
|
|
|
if (operand && IsA(operand, RelabelType))
|
2003-01-15 20:35:48 +01:00
|
|
|
operand = (Node *) ((RelabelType *) operand)->arg;
|
2000-08-13 04:50:35 +02:00
|
|
|
|
2003-05-28 18:04:02 +02:00
|
|
|
indkey = index->indexkeys[indexcol];
|
|
|
|
|
if (indkey != 0)
|
1999-07-23 05:34:49 +02:00
|
|
|
{
|
|
|
|
|
/*
|
2003-05-28 18:04:02 +02:00
|
|
|
* Simple index column; operand must be a matching Var.
|
1999-07-23 05:34:49 +02:00
|
|
|
*/
|
2000-08-13 04:50:35 +02:00
|
|
|
if (operand && IsA(operand, Var) &&
|
2005-03-27 08:29:49 +02:00
|
|
|
index->rel->relid == ((Var *) operand)->varno &&
|
Make Vars be outer-join-aware.
Traditionally we used the same Var struct to represent the value
of a table column everywhere in parse and plan trees. This choice
predates our support for SQL outer joins, and it's really a pretty
bad idea with outer joins, because the Var's value can depend on
where it is in the tree: it might go to NULL above an outer join.
So expression nodes that are equal() per equalfuncs.c might not
represent the same value, which is a huge correctness hazard for
the planner.
To improve this, decorate Var nodes with a bitmapset showing
which outer joins (identified by RTE indexes) may have nulled
them at the point in the parse tree where the Var appears.
This allows us to trust that equal() Vars represent the same value.
A certain amount of klugery is still needed to cope with cases
where we re-order two outer joins, but it's possible to make it
work without sacrificing that core principle. PlaceHolderVars
receive similar decoration for the same reason.
In the planner, we include these outer join bitmapsets into the relids
that an expression is considered to depend on, and in consequence also
add outer-join relids to the relids of join RelOptInfos. This allows
us to correctly perceive whether an expression can be calculated above
or below a particular outer join.
This change affects FDWs that want to plan foreign joins. They *must*
follow suit when labeling foreign joins in order to match with the
core planner, but for many purposes (if postgres_fdw is any guide)
they'd prefer to consider only base relations within the join.
To support both requirements, redefine ForeignScan.fs_relids as
base+OJ relids, and add a new field fs_base_relids that's set up by
the core planner.
Large though it is, this commit just does the minimum necessary to
install the new mechanisms and get check-world passing again.
Follow-up patches will perform some cleanup. (The README additions
and comments mention some stuff that will appear in the follow-up.)
Patch by me; thanks to Richard Guo for review.
Discussion: https://postgr.es/m/830269.1656693747@sss.pgh.pa.us
2023-01-30 19:16:20 +01:00
|
|
|
indkey == ((Var *) operand)->varattno &&
|
|
|
|
|
((Var *) operand)->varnullingrels == NULL)
|
1999-08-16 04:17:58 +02:00
|
|
|
return true;
|
1999-07-23 05:34:49 +02:00
|
|
|
}
|
2003-05-28 18:04:02 +02:00
|
|
|
else
|
1997-09-07 07:04:48 +02:00
|
|
|
{
|
2003-05-28 18:04:02 +02:00
|
|
|
/*
|
|
|
|
|
* Index expression; find the correct expression. (This search could
|
|
|
|
|
* be avoided, at the cost of complicating all the callers of this
|
|
|
|
|
* routine; doesn't seem worth it.)
|
|
|
|
|
*/
|
2004-05-26 06:41:50 +02:00
|
|
|
ListCell *indexpr_item;
|
2003-05-28 18:04:02 +02:00
|
|
|
int i;
|
|
|
|
|
Node *indexkey;
|
1999-08-12 06:32:54 +02:00
|
|
|
|
2004-05-26 06:41:50 +02:00
|
|
|
indexpr_item = list_head(index->indexprs);
|
2003-05-28 18:04:02 +02:00
|
|
|
for (i = 0; i < indexcol; i++)
|
|
|
|
|
{
|
|
|
|
|
if (index->indexkeys[i] == 0)
|
|
|
|
|
{
|
2004-05-26 06:41:50 +02:00
|
|
|
if (indexpr_item == NULL)
|
2003-05-28 18:04:02 +02:00
|
|
|
elog(ERROR, "wrong number of index expressions");
|
Represent Lists as expansible arrays, not chains of cons-cells.
Originally, Postgres Lists were a more or less exact reimplementation of
Lisp lists, which consist of chains of separately-allocated cons cells,
each having a value and a next-cell link. We'd hacked that once before
(commit d0b4399d8) to add a separate List header, but the data was still
in cons cells. That makes some operations -- notably list_nth() -- O(N),
and it's bulky because of the next-cell pointers and per-cell palloc
overhead, and it's very cache-unfriendly if the cons cells end up
scattered around rather than being adjacent.
In this rewrite, we still have List headers, but the data is in a
resizable array of values, with no next-cell links. Now we need at
most two palloc's per List, and often only one, since we can allocate
some values in the same palloc call as the List header. (Of course,
extending an existing List may require repalloc's to enlarge the array.
But this involves just O(log N) allocations not O(N).)
Of course this is not without downsides. The key difficulty is that
addition or deletion of a list entry may now cause other entries to
move, which it did not before.
For example, that breaks foreach() and sister macros, which historically
used a pointer to the current cons-cell as loop state. We can repair
those macros transparently by making their actual loop state be an
integer list index; the exposed "ListCell *" pointer is no longer state
carried across loop iterations, but is just a derived value. (In
practice, modern compilers can optimize things back to having just one
loop state value, at least for simple cases with inline loop bodies.)
In principle, this is a semantics change for cases where the loop body
inserts or deletes list entries ahead of the current loop index; but
I found no such cases in the Postgres code.
The change is not at all transparent for code that doesn't use foreach()
but chases lists "by hand" using lnext(). The largest share of such
code in the backend is in loops that were maintaining "prev" and "next"
variables in addition to the current-cell pointer, in order to delete
list cells efficiently using list_delete_cell(). However, we no longer
need a previous-cell pointer to delete a list cell efficiently. Keeping
a next-cell pointer doesn't work, as explained above, but we can improve
matters by changing such code to use a regular foreach() loop and then
using the new macro foreach_delete_current() to delete the current cell.
(This macro knows how to update the associated foreach loop's state so
that no cells will be missed in the traversal.)
There remains a nontrivial risk of code assuming that a ListCell *
pointer will remain good over an operation that could now move the list
contents. To help catch such errors, list.c can be compiled with a new
define symbol DEBUG_LIST_MEMORY_USAGE that forcibly moves list contents
whenever that could possibly happen. This makes list operations
significantly more expensive so it's not normally turned on (though it
is on by default if USE_VALGRIND is on).
There are two notable API differences from the previous code:
* lnext() now requires the List's header pointer in addition to the
current cell's address.
* list_delete_cell() no longer requires a previous-cell argument.
These changes are somewhat unfortunate, but on the other hand code using
either function needs inspection to see if it is assuming anything
it shouldn't, so it's not all bad.
Programmers should be aware of these significant performance changes:
* list_nth() and related functions are now O(1); so there's no
major access-speed difference between a list and an array.
* Inserting or deleting a list element now takes time proportional to
the distance to the end of the list, due to moving the array elements.
(However, it typically *doesn't* require palloc or pfree, so except in
long lists it's probably still faster than before.) Notably, lcons()
used to be about the same cost as lappend(), but that's no longer true
if the list is long. Code that uses lcons() and list_delete_first()
to maintain a stack might usefully be rewritten to push and pop at the
end of the list rather than the beginning.
* There are now list_insert_nth...() and list_delete_nth...() functions
that add or remove a list cell identified by index. These have the
data-movement penalty explained above, but there's no search penalty.
* list_concat() and variants now copy the second list's data into
storage belonging to the first list, so there is no longer any
sharing of cells between the input lists. The second argument is
now declared "const List *" to reflect that it isn't changed.
This patch just does the minimum needed to get the new implementation
in place and fix bugs exposed by the regression tests. As suggested
by the foregoing, there's a fair amount of followup work remaining to
do.
Also, the ENABLE_LIST_COMPAT macros are finally removed in this
commit. Code using those should have been gone a dozen years ago.
Patch by me; thanks to David Rowley, Jesper Pedersen, and others
for review.
Discussion: https://postgr.es/m/11587.1550975080@sss.pgh.pa.us
2019-07-15 19:41:58 +02:00
|
|
|
indexpr_item = lnext(index->indexprs, indexpr_item);
|
2003-05-28 18:04:02 +02:00
|
|
|
}
|
|
|
|
|
}
|
2004-05-26 06:41:50 +02:00
|
|
|
if (indexpr_item == NULL)
|
2003-05-28 18:04:02 +02:00
|
|
|
elog(ERROR, "wrong number of index expressions");
|
2004-05-26 06:41:50 +02:00
|
|
|
indexkey = (Node *) lfirst(indexpr_item);
|
2003-08-04 02:43:34 +02:00
|
|
|
|
2003-05-28 18:04:02 +02:00
|
|
|
/*
|
|
|
|
|
* Does it match the operand? Again, strip any relabeling.
|
|
|
|
|
*/
|
|
|
|
|
if (indexkey && IsA(indexkey, RelabelType))
|
|
|
|
|
indexkey = (Node *) ((RelabelType *) indexkey)->arg;
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2003-05-28 18:04:02 +02:00
|
|
|
if (equal(indexkey, operand))
|
|
|
|
|
return true;
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
1997-09-07 07:04:48 +02:00
|
|
|
|
2003-05-28 18:04:02 +02:00
|
|
|
return false;
|
1996-07-09 08:22:35 +02:00
|
|
|
}
|
1999-07-27 05:51:11 +02:00
|
|
|
|
|
|
|
|
/*
|
2019-02-12 03:26:08 +01:00
|
|
|
* is_pseudo_constant_for_index()
|
|
|
|
|
* Test whether the given expression can be used as an indexscan
|
|
|
|
|
* comparison value.
|
|
|
|
|
*
|
|
|
|
|
* An indexscan comparison value must not contain any volatile functions,
|
|
|
|
|
* and it can't contain any Vars of the index's own table. Vars of
|
|
|
|
|
* other tables are okay, though; in that case we'd be producing an
|
|
|
|
|
* indexqual usable in a parameterized indexscan. This is, therefore,
|
|
|
|
|
* a weaker condition than is_pseudo_constant_clause().
|
|
|
|
|
*
|
|
|
|
|
* This function is exported for use by planner support functions,
|
|
|
|
|
* which will have available the IndexOptInfo, but not any RestrictInfo
|
|
|
|
|
* infrastructure. It is making the same test made by functions above
|
|
|
|
|
* such as match_opclause_to_indexcol(), but those rely where possible
|
|
|
|
|
* on RestrictInfo information about variable membership.
|
|
|
|
|
*
|
|
|
|
|
* expr: the nodetree to be checked
|
|
|
|
|
* index: the index of interest
|
1999-07-27 05:51:11 +02:00
|
|
|
*/
|
2019-02-12 03:26:08 +01:00
|
|
|
bool
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
is_pseudo_constant_for_index(PlannerInfo *root, Node *expr, IndexOptInfo *index)
|
1999-07-27 05:51:11 +02:00
|
|
|
{
|
2019-02-12 03:26:08 +01:00
|
|
|
/* pull_varnos is cheaper than volatility check, so do that first */
|
Fix pull_varnos' miscomputation of relids set for a PlaceHolderVar.
Previously, pull_varnos() took the relids of a PlaceHolderVar as being
equal to the relids in its contents, but that fails to account for the
possibility that we have to postpone evaluation of the PHV due to outer
joins. This could result in a malformed plan. The known cases end up
triggering the "failed to assign all NestLoopParams to plan nodes"
sanity check in createplan.c, but other symptoms may be possible.
The right value to use is the join level we actually intend to evaluate
the PHV at. We can get that from the ph_eval_at field of the associated
PlaceHolderInfo. However, there are some places that call pull_varnos()
before the PlaceHolderInfos have been created; in that case, fall back
to the conservative assumption that the PHV will be evaluated at its
syntactic level. (In principle this might result in missing some legal
optimization, but I'm not aware of any cases where it's an issue in
practice.) Things are also a bit ticklish for calls occurring during
deconstruct_jointree(), but AFAICS the ph_eval_at fields should have
reached their final values by the time we need them.
The main problem in making this work is that pull_varnos() has no
way to get at the PlaceHolderInfos. We can fix that easily, if a
bit tediously, in HEAD by passing it the planner "root" pointer.
In the back branches that'd cause an unacceptable API/ABI break for
extensions, so leave the existing entry points alone and add new ones
with the additional parameter. (If an old entry point is called and
encounters a PHV, it'll fall back to using the syntactic level,
again possibly missing some valid optimization.)
Back-patch to v12. The computation is surely also wrong before that,
but it appears that we cannot reach a bad plan thanks to join order
restrictions imposed on the subquery that the PlaceHolderVar came from.
The error only became reachable when commit 4be058fe9 allowed trivial
subqueries to be collapsed out completely, eliminating their join order
restrictions.
Per report from Stephan Springl.
Discussion: https://postgr.es/m/171041.1610849523@sss.pgh.pa.us
2021-01-21 21:37:23 +01:00
|
|
|
if (bms_is_member(index->rel->relid, pull_varnos(root, expr)))
|
2019-02-12 03:26:08 +01:00
|
|
|
return false; /* no good, contains Var of table */
|
|
|
|
|
if (contain_volatile_functions(expr))
|
|
|
|
|
return false; /* no good, volatile comparison value */
|
|
|
|
|
return true;
|
1999-12-31 06:38:25 +01:00
|
|
|
}
|
