rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/optimizer/path/indxpath.c

3982 lines
124 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* indxpath.c
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
* Routines to determine which indexes are usable for scanning a
* given relation, and create Paths accordingly.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyright for 2014 Update all files in head, and files COPYRIGHT and legal.sgml in all back branches.
2014-01-07 22:05:30 +01:00
* Portions Copyright (c) 1996-2014, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/optimizer/path/indxpath.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
#include <math.h>
Separate predicate-testing code out of indxpath.c, making it a module in its own right. As proposed by Simon Riggs, but with some editorializing of my own.
2005-06-11 00:25:37 +02:00
#include "access/skey.h"
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
#include "access/sysattr.h"
More include file adjustments.
2006-07-13 19:47:02 +02:00
#include "catalog/pg_am.h"
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
#include "catalog/pg_collation.h"
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
#include "catalog/pg_operator.h"
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
#include "catalog/pg_opfamily.h"
Change pg_amop's index on (amopclaid,amopopr) to index (amopopr,amopclaid). This makes no difference for existing uses, but allows SelectSortFunction() and pred_test_simple_clause() to use indexscans instead of seqscans to locate entries for a particular operator in pg_amop. Better yet, they can use the SearchSysCacheList() API to cache the search results.
2003-05-13 06:38:58 +02:00
#include "catalog/pg_type.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include "nodes/makefuncs.h"
#include "optimizer/clauses.h"
#include "optimizer/cost.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "optimizer/pathnode.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "optimizer/paths.h"
Separate predicate-testing code out of indxpath.c, making it a module in its own right. As proposed by Simon Riggs, but with some editorializing of my own.
2005-06-11 00:25:37 +02:00
#include "optimizer/predtest.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "optimizer/restrictinfo.h"
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
#include "optimizer/var.h"
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
#include "utils/builtins.h"
Support hex-string input and output for type BYTEA. Both hex format and the traditional "escape" format are automatically handled on input. The output format is selected by the new GUC variable bytea_output. As committed, bytea_output defaults to HEX, which is an *incompatible change*. We will keep it this way for awhile for testing purposes, but should consider whether to switch to the more backwards-compatible default of ESCAPE before 8.5 is released. Peter Eisentraut
2009-08-04 18:08:37 +02:00
#include "utils/bytea.h"
This patch... 1. Removes the unnecessary "#define AbcRegProcedure 123"'s from pg_proc.h. 2. Changes those #defines to use the names already defined in fmgr.h. 3. Forces the make of fmgr.h in backend/Makefile instead of having it made as a dependency in access/common/Makefile *hack*hack*hack* 4. Rearranged the #includes to a less helter-skelter arrangement, also changing <file.h> to "file.h" to signify a non-system header. 5. Removed "pg_proc.h" from files where its only purpose was for the #defines removed in item #1. 6. Added "fmgr.h" to each file changed for completeness sake. Turns out that #6 was not necessary for some files because fmgr.h was being included in a roundabout way SIX levels deep by the first include. "access/genam.h" ->"access/relscan.h" ->"utils/rel.h" ->"access/strat.h" ->"access/skey.h" ->"fmgr.h" So adding fmgr.h really didn't add anything to the compile, hopefully just made it clearer to the programmer. S Darren.
1998-04-27 06:08:07 +02:00
#include "utils/lsyscache.h"
Add missing #include.
2003-05-15 21:34:46 +02:00
#include "utils/pg_locale.h"
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
#include "utils/selfuncs.h"
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Another round of planner/optimizer work. This is just restructuring and code cleanup; no major improvements yet. However, EXPLAIN does produce more intuitive outputs for nested loops with indexscans now...
2000-01-09 01:26:47 +01:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
#define IsBooleanOpfamily(opfamily) \
((opfamily) == BOOL_BTREE_FAM_OID || (opfamily) == BOOL_HASH_FAM_OID)
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
#define IndexCollMatchesExprColl(idxcollation, exprcollation) \
((idxcollation) == InvalidOid || (idxcollation) == (exprcollation))
Reimplement planner's handling of MIN/MAX aggregate optimization (again). Instead of playing cute games with pathkeys, just build a direct representation of the intended sub-select, and feed it through query_planner to get a Path for the index access. This is a bit slower than 9.1's previous method, since we'll duplicate most of the overhead of query_planner; but since the whole optimization only applies to rather simple single-table queries, that probably won't be much of a problem in practice. The advantage is that we get to do the right thing when there's a partial index that needs the implicit IS NOT NULL clause to be usable. Also, although this makes planagg.c be a bit more closely tied to the ordering of operations in grouping_planner, we can get rid of some coupling to lower-level parts of the planner. Per complaint from Marti Raudsepp.
2011-03-22 05:34:31 +01:00
/* Whether to use ScalarArrayOpExpr to build index qualifications */
typedef enum
{
Teach btree to handle ScalarArrayOpExpr quals natively. This allows "indexedcol op ANY(ARRAY[...])" conditions to be used in plain indexscans, and particularly in index-only scans.
2011-10-16 21:39:24 +02:00
SAOP_PER_AM, /* Use ScalarArrayOpExpr if amsearcharray */
SAOP_ALLOW, /* Use ScalarArrayOpExpr for all indexes */
SAOP_REQUIRE /* Require ScalarArrayOpExpr to be used */
Reimplement planner's handling of MIN/MAX aggregate optimization (again). Instead of playing cute games with pathkeys, just build a direct representation of the intended sub-select, and feed it through query_planner to get a Path for the index access. This is a bit slower than 9.1's previous method, since we'll duplicate most of the overhead of query_planner; but since the whole optimization only applies to rather simple single-table queries, that probably won't be much of a problem in practice. The advantage is that we get to do the right thing when there's a partial index that needs the implicit IS NOT NULL clause to be usable. Also, although this makes planagg.c be a bit more closely tied to the ordering of operations in grouping_planner, we can get rid of some coupling to lower-level parts of the planner. Per complaint from Marti Raudsepp.
2011-03-22 05:34:31 +01:00
} SaOpControl;
When dealing with OR-of-ANDs quals, extract multiple subclauses of an AND to use with a multiple-key index. Formerly we would only extract clauses that had to do with the first key of the index, which was correct but didn't exploit the index fully.
2000-07-27 01:46:22 +02:00
Teach the planner to support index access methods that only implement amgettuple or only implement amgetbitmap, instead of the former assumption that every AM supports both APIs. Extracted with minor editorialization from Teodor's fast-GIN-insert patch; whatever becomes of that, this seems like a simple and reasonable generalization of the index AM interface spec.
2009-03-06 00:06:45 +01:00
/* Whether we are looking for plain indexscan, bitmap scan, or either */
typedef enum
{
ST_INDEXSCAN, /* must support amgettuple */
ST_BITMAPSCAN, /* must support amgetbitmap */
ST_ANYSCAN /* either is okay */
} ScanTypeControl;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Data structure for collecting qual clauses that match an index */
typedef struct
{
bool nonempty; /* True if lists are not all empty */
/* Lists of RestrictInfos, one per index column */
List *indexclauses[INDEX_MAX_KEYS];
} IndexClauseSet;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* Per-path data used within choose_bitmap_and() */
typedef struct
{
Path *path; /* IndexPath, BitmapAndPath, or BitmapOrPath */
List *quals; /* the WHERE clauses it uses */
List *preds; /* predicates of its partial index(es) */
Bitmapset *clauseids; /* quals+preds represented as a bitmapset */
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
2007-11-15 23:25:18 +01:00
} PathClauseUsage;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
/* Callback argument for ec_member_matches_indexcol */
typedef struct
{
IndexOptInfo *index; /* index we're considering */
int indexcol; /* index column we want to match to */
} ec_member_matches_arg;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
static void consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
static void consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths,
List *indexjoinclauses,
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
int considered_clauses,
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
List **considered_relids);
static void get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
List **considered_relids);
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
static bool eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
static bool bms_equal_any(Relids relids, List *relids_list);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
static void get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
List **bitindexpaths);
static List *build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
SaOpControl saop_control, ScanTypeControl scantype);
static List *build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
static List *generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses);
In the recent changes to make the planner account better for cache effects in a nestloop inner indexscan, I had only dealt with plain index scans and the index portion of bitmap scans. But there will be cache benefits for the heap accesses of bitmap scans too, so fix cost_bitmap_heap_scan() to account for that.
2006-07-22 17:41:56 +02:00
static Path *choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
List *paths);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
static int path_usage_comparator(const void *a, const void *b);
static Cost bitmap_scan_cost_est(PlannerInfo *root, RelOptInfo *rel,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Path *ipath);
In the recent changes to make the planner account better for cache effects in a nestloop inner indexscan, I had only dealt with plain index scans and the index portion of bitmap scans. But there will be cache benefits for the heap accesses of bitmap scans too, so fix cost_bitmap_heap_scan() to account for that.
2006-07-22 17:41:56 +02:00
static Cost bitmap_and_cost_est(PlannerInfo *root, RelOptInfo *rel,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
List *paths);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
static PathClauseUsage *classify_index_clause_usage(Path *path,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
List **clauselist);
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
static Relids get_bitmap_tree_required_outer(Path *bitmapqual);
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
static void find_indexpath_quals(Path *bitmapqual, List **quals, List **preds);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
static int find_list_position(Node *node, List **nodelist);
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
static bool check_index_only(RelOptInfo *rel, IndexOptInfo *index);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
static double get_loop_count(PlannerInfo *root, Relids outer_relids);
static void match_restriction_clauses_to_index(RelOptInfo *rel,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
IndexOptInfo *index,
IndexClauseSet *clauseset);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
static void match_join_clauses_to_index(PlannerInfo *root,
RelOptInfo *rel, IndexOptInfo *index,
IndexClauseSet *clauseset,
List **joinorclauses);
static void match_eclass_clauses_to_index(PlannerInfo *root,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
IndexOptInfo *index,
IndexClauseSet *clauseset);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
static void match_clauses_to_index(IndexOptInfo *index,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
List *clauses,
IndexClauseSet *clauseset);
static void match_clause_to_index(IndexOptInfo *index,
RestrictInfo *rinfo,
IndexClauseSet *clauseset);
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
static bool match_clause_to_indexcol(IndexOptInfo *index,
Minor cleanup of indxpath.c. Eliminate some superfluous notational complexity around match_clause_to_indexcol(), and rip out the DoneMatchingIndexKeys crock.
2010-11-20 19:57:27 +01:00
int indexcol,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RestrictInfo *rinfo);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
static bool is_indexable_operator(Oid expr_op, Oid opfamily,
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
bool indexkey_on_left);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
static bool match_rowcompare_to_indexcol(IndexOptInfo *index,
int indexcol,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid opfamily,
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid idxcollation,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RowCompareExpr *clause);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
static void match_pathkeys_to_index(IndexOptInfo *index, List *pathkeys,
List **orderby_clauses_p,
List **clause_columns_p);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
static Expr *match_clause_to_ordering_op(IndexOptInfo *index,
int indexcol, Expr *clause, Oid pk_opfamily);
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
static bool ec_member_matches_indexcol(PlannerInfo *root, RelOptInfo *rel,
EquivalenceClass *ec, EquivalenceMember *em,
void *arg);
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
static bool match_boolean_index_clause(Node *clause, int indexcol,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
IndexOptInfo *index);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
static bool match_special_index_operator(Expr *clause,
Oid opfamily, Oid idxcollation,
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
bool indexkey_on_left);
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
static Expr *expand_boolean_index_clause(Node *clause, int indexcol,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
IndexOptInfo *index);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
static List *expand_indexqual_opclause(RestrictInfo *rinfo,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
Oid opfamily, Oid idxcollation);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
static RestrictInfo *expand_indexqual_rowcompare(RestrictInfo *rinfo,
IndexOptInfo *index,
int indexcol);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
static List *prefix_quals(Node *leftop, Oid opfamily, Oid collation,
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *prefix, Pattern_Prefix_Status pstatus);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
static List *network_prefix_quals(Node *leftop, Oid expr_op, Oid opfamily,
pgindent run.
2003-08-04 02:43:34 +02:00
Datum rightop);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
static Datum string_to_datum(const char *str, Oid datatype);
static Const *string_to_const(const char *str, Oid datatype);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
/*
* create_index_paths()
* Generate all interesting index paths for the given relation.
New cost model for planning, incorporating a penalty for random page accesses versus sequential accesses, a (very crude) estimate of the effects of caching on random page accesses, and cost to evaluate WHERE- clause expressions. Export critical parameters for this model as SET variables. Also, create SET variables for the planner's enable flags (enable_seqscan, enable_indexscan, etc) so that these can be controlled more conveniently than via PGOPTIONS. Planner now estimates both startup cost (cost before retrieving first tuple) and total cost of each path, so it can optimize queries with LIMIT on a reasonable basis by interpolating between these costs. Same facility is a win for EXISTS(...) subqueries and some other cases. Redesign pathkey representation to achieve a major speedup in planning (I saw as much as 5X on a 10-way join); also minor changes in planner to reduce memory consumption by recycling discarded Path nodes and not constructing unnecessary lists. Minor cleanups to display more-plausible costs in some cases in EXPLAIN output. Initdb forced by change in interface to index cost estimation functions.
2000-02-15 21:49:31 +01:00
* Candidate paths are added to the rel's pathlist (using add_path).
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
* To be considered for an index scan, an index must match one or more
Major revision of sort-node handling: push knowledge of query sort order down into planner, instead of handling it only at the very top level of the planner. This fixes many things. An explicit sort is now avoided if there is a cheaper alternative (typically an indexscan) not only for ORDER BY, but also for the internal sort of GROUP BY. It works even when there is no other reason (such as a WHERE condition) to consider the indexscan. It works for indexes on functions. It works for indexes on functions, backwards. It's just so cool... CAUTION: I have changed the representation of SortClause nodes, therefore THIS UPDATE BREAKS STORED RULES. You will need to initdb.
1999-08-21 05:49:17 +02:00
* restriction clauses or join clauses from the query's qual condition,
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
* or match the query's ORDER BY condition, or have a predicate that
* matches the query's qual condition.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
* There are two basic kinds of index scans. A "plain" index scan uses
* only restriction clauses (possibly none at all) in its indexqual,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* so it can be applied in any context. A "parameterized" index scan uses
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
* join clauses (plus restriction clauses, if available) in its indexqual.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* When joining such a scan to one of the relations supplying the other
* variables used in its indexqual, the parameterized scan must appear as
* the inner relation of a nestloop join; it can't be used on the outer side,
* nor in a merge or hash join. In that context, values for the other rels'
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
* attributes are available and fixed during any one scan of the indexpath.
*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* An IndexPath is generated and submitted to add_path() for each plain or
* parameterized index scan this routine deems potentially interesting for
* the current query.
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
*
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
* 'rel' is the relation for which we want to generate index paths
Add the ability to extract OR indexscan conditions from OR-of-AND join conditions in which each OR subclause includes a constraint on the same relation. This implements the other useful side-effect of conversion to CNF format, without its unpleasant side-effects. As per pghackers discussion of a few weeks ago.
2004-01-05 06:07:36 +01:00
*
Improve my initial, rather hacky implementation of joins to append relations: fix the executor so that we can have an Append plan on the inside of a nestloop and still pass down outer index keys to index scans within the Append, then generate such plans as if they were regular inner indexscans. This avoids the need to evaluate the outer relation multiple times.
2006-02-05 03:59:17 +01:00
* Note: check_partial_indexes() must have been run previously for this rel.
Fix up planner infrastructure to support LATERAL properly. This patch takes care of a number of problems having to do with failure to choose valid join orders and incorrect handling of lateral references pulled up from subqueries. Notable changes: * Add a LateralJoinInfo data structure similar to SpecialJoinInfo, to represent join ordering constraints created by lateral references. (I first considered extending the SpecialJoinInfo structure, but the semantics are different enough that a separate data structure seems better.) Extend join_is_legal() and related functions to prevent trying to form unworkable joins, and to ensure that we will consider joins that satisfy lateral references even if the joins would be clauseless. * Fill in the infrastructure needed for the last few types of relation scan paths to support parameterization. We'd have wanted this eventually anyway, but it is necessary now because a relation that gets pulled up out of a UNION ALL subquery may acquire a reltargetlist containing lateral references, meaning that its paths *have* to be parameterized whether or not we have any code that can push join quals down into the scan. * Compute data about lateral references early in query_planner(), and save in RelOptInfo nodes, to avoid repetitive calculations later. * Assorted corner-case bug fixes. There's probably still some bugs left, but this is a lot closer to being real than it was before.
2012-08-27 04:48:55 +02:00
*
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
* Note: in cases involving LATERAL references in the relation's tlist, it's
* possible that rel->lateral_relids is nonempty. Currently, we include
* lateral_relids into the parameterization reported for each path, but don't
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* take it into account otherwise. The fact that any such rels *must* be
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
* available as parameter sources perhaps should influence our choices of
* index quals ... but for now, it doesn't seem worth troubling over.
* In particular, comments below about "unparameterized" paths should be read
* as meaning "unparameterized so far as the indexquals are concerned".
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
New cost model for planning, incorporating a penalty for random page accesses versus sequential accesses, a (very crude) estimate of the effects of caching on random page accesses, and cost to evaluate WHERE- clause expressions. Export critical parameters for this model as SET variables. Also, create SET variables for the planner's enable flags (enable_seqscan, enable_indexscan, etc) so that these can be controlled more conveniently than via PGOPTIONS. Planner now estimates both startup cost (cost before retrieving first tuple) and total cost of each path, so it can optimize queries with LIMIT on a reasonable basis by interpolating between these costs. Same facility is a win for EXISTS(...) subqueries and some other cases. Redesign pathkey representation to achieve a major speedup in planning (I saw as much as 5X on a 10-way join); also minor changes in planner to reduce memory consumption by recycling discarded Path nodes and not constructing unnecessary lists. Minor cleanups to display more-plausible costs in some cases in EXPLAIN output. Initdb forced by change in interface to index cost estimation functions.
2000-02-15 21:49:31 +01:00
void
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
create_index_paths(PlannerInfo *root, RelOptInfo *rel)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
List *indexpaths;
List *bitindexpaths;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
List *bitjoinpaths;
List *joinorclauses;
IndexClauseSet rclauseset;
IndexClauseSet jclauseset;
IndexClauseSet eclauseset;
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
ListCell *lc;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/* Skip the whole mess if no indexes */
if (rel->indexlist == NIL)
return;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Bitmap paths are collected and then dealt with at the end */
bitindexpaths = bitjoinpaths = joinorclauses = NIL;
/* Examine each index in turn */
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
foreach(lc, rel->indexlist)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
{
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Protect limited-size array in IndexClauseSets */
Assert(index->ncolumns <= INDEX_MAX_KEYS);
/*
* Ignore partial indexes that do not match the query.
* (generate_bitmap_or_paths() might be able to do something with
* them, but that's of no concern here.)
*/
if (index->indpred != NIL && !index->predOK)
continue;
/*
* Identify the restriction clauses that can match the index.
*/
MemSet(&rclauseset, 0, sizeof(rclauseset));
match_restriction_clauses_to_index(rel, index, &rclauseset);
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Build index paths from the restriction clauses. These will be
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* non-parameterized paths. Plain paths go directly to add_path(),
* bitmap paths are added to bitindexpaths to be handled below.
*/
get_index_paths(root, rel, index, &rclauseset,
&bitindexpaths);
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Identify the join clauses that can match the index. For the moment
* we keep them separate from the restriction clauses. Note that this
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* step finds only "loose" join clauses that have not been merged into
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* EquivalenceClasses. Also, collect join OR clauses for later.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
MemSet(&jclauseset, 0, sizeof(jclauseset));
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
match_join_clauses_to_index(root, rel, index,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
&jclauseset, &joinorclauses);
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Look for EquivalenceClasses that can generate joinclauses matching
* the index.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
MemSet(&eclauseset, 0, sizeof(eclauseset));
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
match_eclass_clauses_to_index(root, index,
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
&eclauseset);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* If we found any plain or eclass join clauses, build parameterized
* index paths using them.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
if (jclauseset.nonempty || eclauseset.nonempty)
consider_index_join_clauses(root, rel, index,
&rclauseset,
&jclauseset,
&eclauseset,
&bitjoinpaths);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Generate BitmapOrPaths for any suitable OR-clauses present in the
* restriction list. Add these to bitindexpaths.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
indexpaths = generate_bitmap_or_paths(root, rel,
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
rel->baserestrictinfo, NIL);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bitindexpaths = list_concat(bitindexpaths, indexpaths);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Likewise, generate BitmapOrPaths for any suitable OR-clauses present in
* the joinclause list. Add these to bitjoinpaths.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
indexpaths = generate_bitmap_or_paths(root, rel,
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
joinorclauses, rel->baserestrictinfo);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bitjoinpaths = list_concat(bitjoinpaths, indexpaths);
/*
* If we found anything usable, generate a BitmapHeapPath for the most
* promising combination of restriction bitmap index paths. Note there
* will be only one such path no matter how many indexes exist. This
* should be sufficient since there's basically only one figure of merit
* (total cost) for such a path.
*/
if (bitindexpaths != NIL)
{
Path *bitmapqual;
BitmapHeapPath *bpath;
bitmapqual = choose_bitmap_and(root, rel, bitindexpaths);
Fix up planner infrastructure to support LATERAL properly. This patch takes care of a number of problems having to do with failure to choose valid join orders and incorrect handling of lateral references pulled up from subqueries. Notable changes: * Add a LateralJoinInfo data structure similar to SpecialJoinInfo, to represent join ordering constraints created by lateral references. (I first considered extending the SpecialJoinInfo structure, but the semantics are different enough that a separate data structure seems better.) Extend join_is_legal() and related functions to prevent trying to form unworkable joins, and to ensure that we will consider joins that satisfy lateral references even if the joins would be clauseless. * Fill in the infrastructure needed for the last few types of relation scan paths to support parameterization. We'd have wanted this eventually anyway, but it is necessary now because a relation that gets pulled up out of a UNION ALL subquery may acquire a reltargetlist containing lateral references, meaning that its paths *have* to be parameterized whether or not we have any code that can push join quals down into the scan. * Compute data about lateral references early in query_planner(), and save in RelOptInfo nodes, to avoid repetitive calculations later. * Assorted corner-case bug fixes. There's probably still some bugs left, but this is a lot closer to being real than it was before.
2012-08-27 04:48:55 +02:00
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
rel->lateral_relids, 1.0);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
add_path(rel, (Path *) bpath);
}
/*
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
* Likewise, if we found anything usable, generate BitmapHeapPaths for the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* most promising combinations of join bitmap index paths. Our strategy
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
* is to generate one such path for each distinct parameterization seen
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* among the available bitmap index paths. This may look pretty
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
* expensive, but usually there won't be very many distinct
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* parameterizations. (This logic is quite similar to that in
* consider_index_join_clauses, but we're working with whole paths not
* individual clauses.)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
if (bitjoinpaths != NIL)
{
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
List *path_outer;
List *all_path_outers;
ListCell *lc;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
/*
* path_outer holds the parameterization of each path in bitjoinpaths
* (to save recalculating that several times), while all_path_outers
* holds all distinct parameterization sets.
*/
path_outer = all_path_outers = NIL;
foreach(lc, bitjoinpaths)
{
Path *path = (Path *) lfirst(lc);
Relids required_outer;
required_outer = get_bitmap_tree_required_outer(path);
path_outer = lappend(path_outer, required_outer);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
if (!bms_equal_any(required_outer, all_path_outers))
Allow create_index_paths() to consider multiple join bitmapscan paths. In the initial cut at the "parameterized paths" feature, I'd simplified create_index_paths() to the point where it would only generate a single parameterized bitmap path per relation. Experimentation with an example supplied by Josh Berkus convinces me that that's not good enough: we really need to consider a bitmap path for each possible outer relation. Otherwise we have regressions relative to pre-9.2 versions, in which the planner picks a plain indexscan where it should have used a bitmap scan in queries involving three or more tables. Indeed, after fixing this, several queries in the regression tests show improved plans as a result of using bitmap not plain indexscans.
2012-08-16 19:03:54 +02:00
all_path_outers = lappend(all_path_outers, required_outer);
}
/* Now, for each distinct parameterization set ... */
foreach(lc, all_path_outers)
{
Relids max_outers = (Relids) lfirst(lc);
List *this_path_set;
Path *bitmapqual;
Relids required_outer;
double loop_count;
BitmapHeapPath *bpath;
ListCell *lcp;
ListCell *lco;
/* Identify all the bitmap join paths needing no more than that */
this_path_set = NIL;
forboth(lcp, bitjoinpaths, lco, path_outer)
{
Path *path = (Path *) lfirst(lcp);
Relids p_outers = (Relids) lfirst(lco);
if (bms_is_subset(p_outers, max_outers))
this_path_set = lappend(this_path_set, path);
}
/*
* Add in restriction bitmap paths, since they can be used
* together with any join paths.
*/
this_path_set = list_concat(this_path_set, bitindexpaths);
/* Select best AND combination for this parameterization */
bitmapqual = choose_bitmap_and(root, rel, this_path_set);
/* And push that path into the mix */
required_outer = get_bitmap_tree_required_outer(bitmapqual);
loop_count = get_loop_count(root, required_outer);
bpath = create_bitmap_heap_path(root, rel, bitmapqual,
required_outer, loop_count);
add_path(rel, (Path *) bpath);
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
}
/*
* consider_index_join_clauses
* Given sets of join clauses for an index, decide which parameterized
* index paths to build.
*
* Plain indexpaths are sent directly to add_path, while potential
* bitmap indexpaths are added to *bitindexpaths for later processing.
*
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'rclauseset' is the collection of indexable restriction clauses
* 'jclauseset' is the collection of indexable simple join clauses
* 'eclauseset' is the collection of indexable clauses from EquivalenceClasses
* '*bitindexpaths' is the list to add bitmap paths to
*/
static void
consider_index_join_clauses(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths)
{
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
int considered_clauses = 0;
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
List *considered_relids = NIL;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
int indexcol;
/*
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* The strategy here is to identify every potentially useful set of outer
* rels that can provide indexable join clauses. For each such set,
* select all the join clauses available from those outer rels, add on all
* the indexable restriction clauses, and generate plain and/or bitmap
* index paths for that set of clauses. This is based on the assumption
* that it's always better to apply a clause as an indexqual than as a
* filter (qpqual); which is where an available clause would end up being
* applied if we omit it from the indexquals.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
* This looks expensive, but in most practical cases there won't be very
* many distinct sets of outer rels to consider. As a safety valve when
* that's not true, we use a heuristic: limit the number of outer rel sets
* considered to a multiple of the number of clauses considered. (We'll
* always consider using each individual join clause, though.)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* For simplicity in selecting relevant clauses, we represent each set of
* outer rels as a maximum set of clause_relids --- that is, the indexed
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* relation itself is also included in the relids set. considered_relids
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* lists all relids sets we've already tried.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
{
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* Consider each applicable simple join clause */
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
considered_clauses += list_length(jclauseset->indexclauses[indexcol]);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
consider_index_join_outer_rels(root, rel, index,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
jclauseset->indexclauses[indexcol],
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
considered_clauses,
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
&considered_relids);
/* Consider each applicable eclass join clause */
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
considered_clauses += list_length(eclauseset->indexclauses[indexcol]);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
consider_index_join_outer_rels(root, rel, index,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
eclauseset->indexclauses[indexcol],
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
considered_clauses,
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
&considered_relids);
}
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/*
* consider_index_join_outer_rels
* Generate parameterized paths based on clause relids in the clause list.
*
* Workhorse for consider_index_join_clauses; see notes therein for rationale.
*
* 'rel', 'index', 'rclauseset', 'jclauseset', 'eclauseset', and
* 'bitindexpaths' as above
* 'indexjoinclauses' is a list of RestrictInfos for join clauses
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
* 'considered_clauses' is the total number of clauses considered (so far)
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* '*considered_relids' is a list of all relids sets already considered
*/
static void
consider_index_join_outer_rels(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths,
List *indexjoinclauses,
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
int considered_clauses,
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
List **considered_relids)
{
ListCell *lc;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* Examine relids of each joinclause in the given list */
foreach(lc, indexjoinclauses)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Relids clause_relids = rinfo->clause_relids;
ListCell *lc2;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* If we already tried its relids set, no need to do so again */
if (bms_equal_any(clause_relids, *considered_relids))
continue;
/*
* Generate the union of this clause's relids set with each
* previously-tried set. This ensures we try this clause along with
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
* every interesting subset of previous clauses. However, to avoid
* exponential growth of planning time when there are many clauses,
* limit the number of relid sets accepted to 10 * considered_clauses.
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
*
* Note: get_join_index_paths adds entries to *considered_relids, but
* it prepends them to the list, so that we won't visit new entries
* during the inner foreach loop. No real harm would be done if we
* did, since the subset check would reject them; but it would waste
* some cycles.
*/
foreach(lc2, *considered_relids)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
{
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
Relids oldrelids = (Relids) lfirst(lc2);
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/*
* If either is a subset of the other, no new set is possible.
* This isn't a complete test for redundancy, but it's easy and
* cheap. get_join_index_paths will check more carefully if we
* already generated the same relids set.
*/
if (bms_subset_compare(clause_relids, oldrelids) != BMS_DIFFERENT)
continue;
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
/*
* If this clause was derived from an equivalence class, the
* clause list may contain other clauses derived from the same
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* eclass. We should not consider that combining this clause with
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
* one of those clauses generates a usefully different
* parameterization; so skip if any clause derived from the same
* eclass would already have been included when using oldrelids.
*/
if (rinfo->parent_ec &&
eclass_already_used(rinfo->parent_ec, oldrelids,
indexjoinclauses))
continue;
/*
* If the number of relid sets considered exceeds our heuristic
* limit, stop considering combinations of clauses. We'll still
* consider the current clause alone, though (below this loop).
*/
if (list_length(*considered_relids) >= 10 * considered_clauses)
break;
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* OK, try the union set */
get_join_index_paths(root, rel, index,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
bms_union(clause_relids, oldrelids),
considered_relids);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* Also try this set of relids by itself */
get_join_index_paths(root, rel, index,
rclauseset, jclauseset, eclauseset,
bitindexpaths,
clause_relids,
considered_relids);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
}
/*
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* get_join_index_paths
* Generate index paths using clauses from the specified outer relations.
* In addition to generating paths, relids is added to *considered_relids
* if not already present.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*
* Workhorse for consider_index_join_clauses; see notes therein for rationale.
*
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* 'rel', 'index', 'rclauseset', 'jclauseset', 'eclauseset',
* 'bitindexpaths', 'considered_relids' as above
* 'relids' is the current set of relids to consider (the target rel plus
* one or more outer rels)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
static void
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
get_join_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index,
IndexClauseSet *rclauseset,
IndexClauseSet *jclauseset,
IndexClauseSet *eclauseset,
List **bitindexpaths,
Relids relids,
List **considered_relids)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
{
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
IndexClauseSet clauseset;
int indexcol;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* If we already considered this relids set, don't repeat the work */
if (bms_equal_any(relids, *considered_relids))
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
return;
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* Identify indexclauses usable with this relids set */
MemSet(&clauseset, 0, sizeof(clauseset));
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
{
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
ListCell *lc;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* First find applicable simple join clauses */
foreach(lc, jclauseset->indexclauses[indexcol])
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
if (bms_is_subset(rinfo->clause_relids, relids))
clauseset.indexclauses[indexcol] =
lappend(clauseset.indexclauses[indexcol], rinfo);
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Add applicable eclass join clauses. The clauses generated for each
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
* column are redundant (cf generate_implied_equalities_for_column),
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* so we need at most one. This is the only exception to the general
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
* rule of using all available index clauses.
*/
foreach(lc, eclauseset->indexclauses[indexcol])
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (bms_is_subset(rinfo->clause_relids, relids))
{
clauseset.indexclauses[indexcol] =
lappend(clauseset.indexclauses[indexcol], rinfo);
break;
}
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* Add restriction clauses (this is nondestructive to rclauseset) */
clauseset.indexclauses[indexcol] =
list_concat(clauseset.indexclauses[indexcol],
rclauseset->indexclauses[indexcol]);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
if (clauseset.indexclauses[indexcol] != NIL)
clauseset.nonempty = true;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/* We should have found something, else caller passed silly relids */
Assert(clauseset.nonempty);
/* Build index path(s) using the collected set of clauses */
get_index_paths(root, rel, index, &clauseset, bitindexpaths);
/*
* Remember we considered paths for this set of relids. We use lcons not
* lappend to avoid confusing the loop in consider_index_join_outer_rels.
*/
*considered_relids = lcons(relids, *considered_relids);
}
Limit the number of rel sets considered in consider_index_join_outer_rels. In bug #7626, Brian Dunavant exposes a performance problem created by commit 3b8968f25232ad09001bf35ab4cc59f5a501193e: that commit attempted to consider *all* possible combinations of indexable join clauses, but if said clauses join to enough different relations, there's an exponential increase in the number of outer-relation sets considered. In Brian's example, all the clauses come from the same equivalence class, which means it's redundant to use more than one of them in an indexscan anyway. So we can prevent the problem in this class of cases (which is probably the majority of real examples) by rejecting combinations that would only serve to add a known-redundant clause. But that still leaves us exposed to exponential growth of planning time when the query has a lot of non-equivalence join clauses that are usable with the same index. I chose to prevent such cases by setting an upper limit on the number of relation sets considered, equal to ten times the number of index clauses considered so far. (This sliding limit still allows new relsets to be added on as we move to additional index columns, which is probably more important than considering even more combinations of clauses for the previous column.) This should keep the amount of work done roughly linear rather than exponential in the apparent query complexity. This part of the fix is pretty ad-hoc; but without a clearer idea of real-world cases for which this would result in markedly inferior plans, it's hard to see how to do better.
2012-11-01 19:08:42 +01:00
/*
* eclass_already_used
* True if any join clause usable with oldrelids was generated from
* the specified equivalence class.
*/
static bool
eclass_already_used(EquivalenceClass *parent_ec, Relids oldrelids,
List *indexjoinclauses)
{
ListCell *lc;
foreach(lc, indexjoinclauses)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (rinfo->parent_ec == parent_ec &&
bms_is_subset(rinfo->clause_relids, oldrelids))
return true;
}
return false;
}
Rethink heuristics for choosing index quals for parameterized paths. Some experimentation with examples similar to bug #7539 has convinced me that indxpath.c's original implementation of parameterized-path generation was several bricks shy of a load. In general, if we are relying on a particular outer rel or set of outer rels for a parameterized path, the path should use every indexable join clause that's available from that rel or rels. Any join clauses that get left out of the indexqual will end up getting applied as plain filter quals (qpquals), and that's generally a significant loser compared to having the index AM enforce them. (This is particularly true with btree, which can skip the index scan entirely if it can see that the given indexquals are mutually contradictory.) The original heuristics failed to ensure this, though, and were overly complicated anyway. Rewrite to make the code explicitly identify each useful set of outer rels and then select all applicable join clauses for each one. The one plan that changes in the regression tests is in fact for the better according to the planner's cost estimates. (Note: this is not a correctness issue but just a matter of plan quality. I don't yet know what is going on in bug #7539, but I don't expect this change to fix that.)
2012-09-16 23:57:18 +02:00
/*
* bms_equal_any
* True if relids is bms_equal to any member of relids_list
*
* Perhaps this should be in bitmapset.c someday.
*/
static bool
bms_equal_any(Relids relids, List *relids_list)
{
ListCell *lc;
foreach(lc, relids_list)
{
if (bms_equal(relids, (Relids) lfirst(lc)))
return true;
}
return false;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
/*
* get_index_paths
* Given an index and a set of index clauses for it, construct IndexPaths.
*
* Plain indexpaths are sent directly to add_path, while potential
* bitmap indexpaths are added to *bitindexpaths for later processing.
*
* This is a fairly simple frontend to build_index_paths(). Its reason for
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* existence is mainly to handle ScalarArrayOpExpr quals properly. If the
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* index AM supports them natively, we should just include them in simple
* index paths. If not, we should exclude them while building simple index
* paths, and then make a separate attempt to include them in bitmap paths.
*/
static void
get_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
List **bitindexpaths)
{
List *indexpaths;
ListCell *lc;
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Build simple index paths using the clauses. Allow ScalarArrayOpExpr
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* clauses only if the index AM supports them natively.
*/
indexpaths = build_index_paths(root, rel,
index, clauses,
index->predOK,
SAOP_PER_AM, ST_ANYSCAN);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Submit all the ones that can form plain IndexScan plans to add_path. (A
* plain IndexPath can represent either a plain IndexScan or an
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* IndexOnlyScan, but for our purposes here that distinction does not
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* matter. However, some of the indexes might support only bitmap scans,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* and those we mustn't submit to add_path here.)
*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Also, pick out the ones that are usable as bitmap scans. For that, we
* must discard indexes that don't support bitmap scans, and we also are
* only interested in paths that have some selectivity; we should discard
* anything that was generated solely for ordering purposes.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, indexpaths)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
IndexPath *ipath = (IndexPath *) lfirst(lc);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (index->amhasgettuple)
Teach the planner to support index access methods that only implement amgettuple or only implement amgetbitmap, instead of the former assumption that every AM supports both APIs. Extracted with minor editorialization from Teodor's fast-GIN-insert patch; whatever becomes of that, this seems like a simple and reasonable generalization of the index AM interface spec.
2009-03-06 00:06:45 +01:00
add_path(rel, (Path *) ipath);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (index->amhasgetbitmap &&
Tweak create_index_paths()'s test for whether to consider a bitmap scan. Per my note of a couple days ago, create_index_paths would refuse to consider any path at all for GIN indexes if the selectivity estimate came out as 1.0; not even if you tried to force it with enable_seqscan. While this isn't really a bad outcome in practice, it could be annoying for testing purposes. Adjust the test for "is this path only useful for sorting" so that it doesn't fire on paths with nil pathkeys, which will include all GIN paths.
2011-01-11 18:12:04 +01:00
(ipath->path.pathkeys == NIL ||
ipath->indexselectivity < 1.0))
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*bitindexpaths = lappend(*bitindexpaths, ipath);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* If the index doesn't handle ScalarArrayOpExpr clauses natively, check
* to see if there are any such clauses, and if so generate bitmap scan
* paths relying on executor-managed ScalarArrayOpExpr.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (!index->amsearcharray)
{
indexpaths = build_index_paths(root, rel,
index, clauses,
false,
SAOP_REQUIRE, ST_BITMAPSCAN);
*bitindexpaths = list_concat(*bitindexpaths, indexpaths);
}
}
/*
* build_index_paths
* Given an index and a set of index clauses for it, construct zero
* or more IndexPaths.
*
* We return a list of paths because (1) this routine checks some cases
* that should cause us to not generate any IndexPath, and (2) in some
* cases we want to consider both a forward and a backward scan, so as
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* to obtain both sort orders. Note that the paths are just returned
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* to the caller and not immediately fed to add_path().
*
* At top level, useful_predicate should be exactly the index's predOK flag
* (ie, true if it has a predicate that was proven from the restriction
* clauses). When working on an arm of an OR clause, useful_predicate
* should be true if the predicate required the current OR list to be proven.
* Note that this routine should never be called at all if the index has an
* unprovable predicate.
*
* saop_control indicates whether ScalarArrayOpExpr clauses can be used.
* When it's SAOP_REQUIRE, index paths are created only if we found at least
* one ScalarArrayOpExpr clause.
*
* scantype indicates whether we want to create plain indexscans, bitmap
* indexscans, or both. When it's ST_BITMAPSCAN, we will not consider
* index ordering while deciding if a Path is worth generating.
*
* 'rel' is the index's heap relation
* 'index' is the index for which we want to generate paths
* 'clauses' is the collection of indexable clauses (RestrictInfo nodes)
* 'useful_predicate' indicates whether the index has a useful predicate
* 'saop_control' indicates whether ScalarArrayOpExpr clauses can be used
* 'scantype' indicates whether we need plain or bitmap scan support
*/
static List *
build_index_paths(PlannerInfo *root, RelOptInfo *rel,
IndexOptInfo *index, IndexClauseSet *clauses,
bool useful_predicate,
SaOpControl saop_control, ScanTypeControl scantype)
{
List *result = NIL;
IndexPath *ipath;
List *index_clauses;
List *clause_columns;
Relids outer_relids;
double loop_count;
List *orderbyclauses;
List *orderbyclausecols;
List *index_pathkeys;
List *useful_pathkeys;
bool found_clause;
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
bool found_lower_saop_clause;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bool pathkeys_possibly_useful;
bool index_is_ordered;
bool index_only_scan;
int indexcol;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable condition: when there are multiple possible index paths involving ScalarArrayOpExprs, they are logically to be ANDed together not ORed. This thinko was a direct consequence of trying to put the processing inside generate_bitmap_or_paths(), which I now see was a bit too cute. So pull it out and make the callers do it separately (there are only two that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden. There are some additional infelicities exposed by his example, but they are also in 8.1.x, while this mistake is not.
2006-05-18 19:12:10 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Check that index supports the desired scan type(s)
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable condition: when there are multiple possible index paths involving ScalarArrayOpExprs, they are logically to be ANDed together not ORed. This thinko was a direct consequence of trying to put the processing inside generate_bitmap_or_paths(), which I now see was a bit too cute. So pull it out and make the callers do it separately (there are only two that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden. There are some additional infelicities exposed by his example, but they are also in 8.1.x, while this mistake is not.
2006-05-18 19:12:10 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
switch (scantype)
{
case ST_INDEXSCAN:
if (!index->amhasgettuple)
return NIL;
break;
case ST_BITMAPSCAN:
if (!index->amhasgetbitmap)
return NIL;
break;
case ST_ANYSCAN:
/* either or both are OK */
break;
}
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable condition: when there are multiple possible index paths involving ScalarArrayOpExprs, they are logically to be ANDed together not ORed. This thinko was a direct consequence of trying to put the processing inside generate_bitmap_or_paths(), which I now see was a bit too cute. So pull it out and make the callers do it separately (there are only two that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden. There are some additional infelicities exposed by his example, but they are also in 8.1.x, while this mistake is not.
2006-05-18 19:12:10 +02:00
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* 1. Collect the index clauses into a single list.
*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* We build a list of RestrictInfo nodes for clauses to be used with this
* index, along with an integer list of the index column numbers (zero
* based) that each clause should be used with. The clauses are ordered
* by index key, so that the column numbers form a nondecreasing sequence.
* (This order is depended on by btree and possibly other places.) The
* lists can be empty, if the index AM allows that.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* found_clause is set true only if there's at least one index clause; and
* if saop_control is SAOP_REQUIRE, it has to be a ScalarArrayOpExpr
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* clause.
*
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
* found_lower_saop_clause is set true if there's a ScalarArrayOpExpr
* index clause for a non-first index column. This prevents us from
* assuming that the scan result is ordered. (Actually, the result is
* still ordered if there are equality constraints for all earlier
* columns, but it seems too expensive and non-modular for this code to be
* aware of that refinement.)
*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* We also build a Relids set showing which outer rels are required by the
Fix up planner infrastructure to support LATERAL properly. This patch takes care of a number of problems having to do with failure to choose valid join orders and incorrect handling of lateral references pulled up from subqueries. Notable changes: * Add a LateralJoinInfo data structure similar to SpecialJoinInfo, to represent join ordering constraints created by lateral references. (I first considered extending the SpecialJoinInfo structure, but the semantics are different enough that a separate data structure seems better.) Extend join_is_legal() and related functions to prevent trying to form unworkable joins, and to ensure that we will consider joins that satisfy lateral references even if the joins would be clauseless. * Fill in the infrastructure needed for the last few types of relation scan paths to support parameterization. We'd have wanted this eventually anyway, but it is necessary now because a relation that gets pulled up out of a UNION ALL subquery may acquire a reltargetlist containing lateral references, meaning that its paths *have* to be parameterized whether or not we have any code that can push join quals down into the scan. * Compute data about lateral references early in query_planner(), and save in RelOptInfo nodes, to avoid repetitive calculations later. * Assorted corner-case bug fixes. There's probably still some bugs left, but this is a lot closer to being real than it was before.
2012-08-27 04:48:55 +02:00
* selected clauses. Any lateral_relids are included in that, but not
* otherwise accounted for.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
index_clauses = NIL;
clause_columns = NIL;
found_clause = false;
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
found_lower_saop_clause = false;
Fix up planner infrastructure to support LATERAL properly. This patch takes care of a number of problems having to do with failure to choose valid join orders and incorrect handling of lateral references pulled up from subqueries. Notable changes: * Add a LateralJoinInfo data structure similar to SpecialJoinInfo, to represent join ordering constraints created by lateral references. (I first considered extending the SpecialJoinInfo structure, but the semantics are different enough that a separate data structure seems better.) Extend join_is_legal() and related functions to prevent trying to form unworkable joins, and to ensure that we will consider joins that satisfy lateral references even if the joins would be clauseless. * Fill in the infrastructure needed for the last few types of relation scan paths to support parameterization. We'd have wanted this eventually anyway, but it is necessary now because a relation that gets pulled up out of a UNION ALL subquery may acquire a reltargetlist containing lateral references, meaning that its paths *have* to be parameterized whether or not we have any code that can push join quals down into the scan. * Compute data about lateral references early in query_planner(), and save in RelOptInfo nodes, to avoid repetitive calculations later. * Assorted corner-case bug fixes. There's probably still some bugs left, but this is a lot closer to being real than it was before.
2012-08-27 04:48:55 +02:00
outer_relids = bms_copy(rel->lateral_relids);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
ListCell *lc;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, clauses->indexclauses[indexcol])
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
if (IsA(rinfo->clause, ScalarArrayOpExpr))
{
/* Ignore if not supported by index */
if (saop_control == SAOP_PER_AM && !index->amsearcharray)
continue;
found_clause = true;
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
if (indexcol > 0)
found_lower_saop_clause = true;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
else
{
if (saop_control != SAOP_REQUIRE)
found_clause = true;
}
index_clauses = lappend(index_clauses, rinfo);
clause_columns = lappend_int(clause_columns, indexcol);
outer_relids = bms_add_members(outer_relids,
rinfo->clause_relids);
}
/*
* If no clauses match the first index column, check for amoptionalkey
* restriction. We can't generate a scan over an index with
* amoptionalkey = false unless there's at least one index clause.
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* (When working on columns after the first, this test cannot fail. It
* is always okay for columns after the first to not have any
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* clauses.)
*/
if (index_clauses == NIL && !index->amoptionalkey)
return NIL;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* We do not want the index's rel itself listed in outer_relids */
outer_relids = bms_del_member(outer_relids, rel->relid);
/* Enforce convention that outer_relids is exactly NULL if empty */
if (bms_is_empty(outer_relids))
outer_relids = NULL;
/* Compute loop_count for cost estimation purposes */
loop_count = get_loop_count(root, outer_relids);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
* 2. Compute pathkeys describing index's ordering, if any, then see how
* many of them are actually useful for this query. This is not relevant
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
* if we are only trying to build bitmap indexscans, nor if we have to
* assume the scan is unordered.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
pathkeys_possibly_useful = (scantype != ST_BITMAPSCAN &&
Fix planning of btree index scans using ScalarArrayOpExpr quals. In commit 9e8da0f75731aaa7605cf4656c21ea09e84d2eb1, I improved btree to handle ScalarArrayOpExpr quals natively, so that constructs like "indexedcol IN (list)" could be supported by index-only scans. Using such a qual results in multiple scans of the index, under-the-hood. I went to some lengths to ensure that this still produces rows in index order ... but I failed to recognize that if a higher-order index column is lacking an equality constraint, rescans can produce out-of-order data from that column. Tweak the planner to not expect sorted output in that case. Per trouble report from Robert McGehee.
2012-09-18 18:20:34 +02:00
!found_lower_saop_clause &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
has_useful_pathkeys(root, rel));
index_is_ordered = (index->sortopfamily != NULL);
if (index_is_ordered && pathkeys_possibly_useful)
{
index_pathkeys = build_index_pathkeys(root, index,
ForwardScanDirection);
useful_pathkeys = truncate_useless_pathkeys(root, rel,
index_pathkeys);
orderbyclauses = NIL;
orderbyclausecols = NIL;
}
else if (index->amcanorderbyop && pathkeys_possibly_useful)
{
/* see if we can generate ordering operators for query_pathkeys */
match_pathkeys_to_index(index, root->query_pathkeys,
&orderbyclauses,
&orderbyclausecols);
if (orderbyclauses)
useful_pathkeys = root->query_pathkeys;
else
useful_pathkeys = NIL;
}
else
{
useful_pathkeys = NIL;
orderbyclauses = NIL;
orderbyclausecols = NIL;
}
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* 3. Check if an index-only scan is possible. If we're not building
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* plain indexscans, this isn't relevant since bitmap scans don't support
* index data retrieval anyway.
*/
index_only_scan = (scantype != ST_BITMAPSCAN &&
check_index_only(rel, index));
/*
* 4. Generate an indexscan path if there are relevant restriction clauses
* in the current clauses, OR the index ordering is potentially useful for
* later merging or final output ordering, OR the index has a useful
* predicate, OR an index-only scan is possible.
*/
if (found_clause || useful_pathkeys != NIL || useful_predicate ||
index_only_scan)
{
ipath = create_index_path(root, index,
index_clauses,
clause_columns,
orderbyclauses,
orderbyclausecols,
useful_pathkeys,
index_is_ordered ?
ForwardScanDirection :
NoMovementScanDirection,
index_only_scan,
outer_relids,
loop_count);
result = lappend(result, ipath);
}
/*
* 5. If the index is ordered, a backwards scan might be interesting.
*/
if (index_is_ordered && pathkeys_possibly_useful)
{
index_pathkeys = build_index_pathkeys(root, index,
BackwardScanDirection);
useful_pathkeys = truncate_useless_pathkeys(root, rel,
index_pathkeys);
if (useful_pathkeys != NIL)
{
ipath = create_index_path(root, index,
index_clauses,
clause_columns,
NIL,
NIL,
useful_pathkeys,
BackwardScanDirection,
index_only_scan,
outer_relids,
loop_count);
result = lappend(result, ipath);
}
}
return result;
}
/*
* build_paths_for_OR
* Given a list of restriction clauses from one arm of an OR clause,
* construct all matching IndexPaths for the relation.
*
* Here we must scan all indexes of the relation, since a bitmap OR tree
* can use multiple indexes.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*
* The caller actually supplies two lists of restriction clauses: some
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* "current" ones and some "other" ones. Both lists can be used freely
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
* to match keys of the index, but an index must use at least one of the
* "current" clauses to be considered usable. The motivation for this is
* examples like
* WHERE (x = 42) AND (... OR (y = 52 AND z = 77) OR ....)
* While we are considering the y/z subclause of the OR, we can use "x = 42"
* as one of the available index conditions; but we shouldn't match the
* subclause to any index on x alone, because such a Path would already have
* been generated at the upper level. So we could use an index on x,y,z
* or an index on x,y for the OR subclause, but not an index on just x.
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
* When dealing with a partial index, a match of the index predicate to
* one of the "current" clauses also makes the index usable.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*
* 'rel' is the relation for which we want to generate index paths
* 'clauses' is the current list of clauses (RestrictInfo nodes)
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* 'other_clauses' is the list of additional upper-level clauses
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
static List *
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
build_paths_for_OR(PlannerInfo *root, RelOptInfo *rel,
List *clauses, List *other_clauses)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
List *result = NIL;
List *all_clauses = NIL; /* not computed till needed */
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
ListCell *lc;
Allow binary-compatible indices to be considered when checking for valid indices for restriction clauses containing a constant. Note that if an index does not match directly (usually because the types on both side of the clause don't match), and if a binary-compatible index is identified, then the operator function will be replaced by a new one. Should not be a problem, but be sure that if types are listed as being binary compatible (in parse_coerce.h) then the comparison functions are also binary-compatible, giving equivalent results.
1998-08-14 18:13:07 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, rel->indexlist)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
IndexClauseSet clauseset;
List *indexpaths;
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
bool useful_predicate;
Allow binary-compatible indices to be considered when checking for valid indices for restriction clauses containing a constant. Note that if an index does not match directly (usually because the types on both side of the clause don't match), and if a binary-compatible index is identified, then the operator function will be replaced by a new one. Should not be a problem, but be sure that if types are listed as being binary compatible (in parse_coerce.h) then the comparison functions are also binary-compatible, giving equivalent results.
1998-08-14 18:13:07 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Ignore index if it doesn't support bitmap scans */
if (!index->amhasgetbitmap)
Teach btree to handle ScalarArrayOpExpr quals natively. This allows "indexedcol op ANY(ARRAY[...])" conditions to be used in plain indexscans, and particularly in index-only scans.
2011-10-16 21:39:24 +02:00
continue;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Ignore partial indexes that do not match the query. If a partial
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* index is marked predOK then we know it's OK. Otherwise, we have to
* test whether the added clauses are sufficient to imply the
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* predicate. If so, we can use the index in the current context.
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* We set useful_predicate to true iff the predicate was proven using
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* the current set of clauses. This is needed to prevent matching a
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* predOK index to an arm of an OR, which would be a legal but
* pointlessly inefficient plan. (A better plan will be generated by
* just scanning the predOK index alone, no OR.)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
useful_predicate = false;
if (index->indpred != NIL)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
if (index->predOK)
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Usable, but don't set useful_predicate */
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
}
else
{
/* Form all_clauses if not done already */
if (all_clauses == NIL)
all_clauses = list_concat(list_copy(clauses),
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
other_clauses);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
if (!predicate_implied_by(index->indpred, all_clauses))
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
continue; /* can't use it at all */
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (!predicate_implied_by(index->indpred, other_clauses))
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
useful_predicate = true;
}
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
Allow binary-compatible indices to be considered when checking for valid indices for restriction clauses containing a constant. Note that if an index does not match directly (usually because the types on both side of the clause don't match), and if a binary-compatible index is identified, then the operator function will be replaced by a new one. Should not be a problem, but be sure that if types are listed as being binary compatible (in parse_coerce.h) then the comparison functions are also binary-compatible, giving equivalent results.
1998-08-14 18:13:07 +02:00
Allow index use with OR clauses.
1998-08-02 00:12:13 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Identify the restriction clauses that can match the index.
Allow index use with OR clauses.
1998-08-02 00:12:13 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
MemSet(&clauseset, 0, sizeof(clauseset));
match_clauses_to_index(index, clauses, &clauseset);
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* If no matches so far, and the index predicate isn't useful, we
* don't want it.
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (!clauseset.nonempty && !useful_predicate)
Fix a bunch of bad interactions between partial indexes and the new planning logic for bitmap indexscans. Partial indexes create corner cases in which a scan might be done with no explicit index qual conditions, and the code wasn't handling those cases nicely. Also be a little tenser about eliminating redundant clauses in the generated plan. Per report from Dmitry Karasik.
2005-07-28 22:26:22 +02:00
continue;
Further cleanups of indexqual processing: simplify control logic in indxpath.c, avoid generation of redundant indexscan paths for the same relation and index.
1999-07-30 06:07:25 +02:00
Allow index use with OR clauses.
1998-08-02 00:12:13 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Add "other" restriction clauses to the clauseset.
Major planner/optimizer revision: get rid of PathOrder node type, store all ordering information in pathkeys lists (which are now lists of lists of PathKeyItem nodes, not just lists of lists of vars). This was a big win --- the code is smaller and IMHO more understandable than it was, even though it handles more cases. I believe the node changes will not force an initdb for anyone; planner nodes don't show up in stored rules.
1999-08-16 04:17:58 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
match_clauses_to_index(index, other_clauses, &clauseset);
Planner speedup hacking. Avoid saving useless pathkeys, so that path comparison does not consider paths different when they differ only in uninteresting aspects of sort order. (We had a special case of this consideration for indexscans already, but generalize it to apply to ordered join paths too.) Be stricter about what is a canonical pathkey to allow faster pathkey comparison. Cache canonical pathkeys and dispersion stats for left and right sides of a RestrictInfo's clause, to avoid repeated computation. Total speedup will depend on number of tables in a query, but I see about 4x speedup of planning phase for a sample seven-table query.
2000-12-14 23:30:45 +01:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Construct paths if possible.
Planner speedup hacking. Avoid saving useless pathkeys, so that path comparison does not consider paths different when they differ only in uninteresting aspects of sort order. (We had a special case of this consideration for indexscans already, but generalize it to apply to ordered join paths too.) Be stricter about what is a canonical pathkey to allow faster pathkey comparison. Cache canonical pathkeys and dispersion stats for left and right sides of a RestrictInfo's clause, to avoid repeated computation. Total speedup will depend on number of tables in a query, but I see about 4x speedup of planning phase for a sample seven-table query.
2000-12-14 23:30:45 +01:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
indexpaths = build_index_paths(root, rel,
index, &clauseset,
useful_predicate,
SAOP_ALLOW, ST_BITMAPSCAN);
result = list_concat(result, indexpaths);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
return result;
}
/*
* generate_bitmap_or_paths
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable condition: when there are multiple possible index paths involving ScalarArrayOpExprs, they are logically to be ANDed together not ORed. This thinko was a direct consequence of trying to put the processing inside generate_bitmap_or_paths(), which I now see was a bit too cute. So pull it out and make the callers do it separately (there are only two that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden. There are some additional infelicities exposed by his example, but they are also in 8.1.x, while this mistake is not.
2006-05-18 19:12:10 +02:00
* Look through the list of clauses to find OR clauses, and generate
* a BitmapOrPath for each one we can handle that way. Return a list
* of the generated BitmapOrPaths.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* other_clauses is a list of additional clauses that can be assumed true
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
* for the purpose of generating indexquals, but are not to be searched for
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* ORs. (See build_paths_for_OR() for motivation.)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
static List *
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
generate_bitmap_or_paths(PlannerInfo *root, RelOptInfo *rel,
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
List *clauses, List *other_clauses)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
List *result = NIL;
List *all_clauses;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
ListCell *lc;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* We can use both the current and other clauses as context for
* build_paths_for_OR; no need to remove ORs from the lists.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
all_clauses = list_concat(list_copy(clauses), other_clauses);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, clauses)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
List *pathlist;
Path *bitmapqual;
ListCell *j;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Assert(IsA(rinfo, RestrictInfo));
Fix thinko in recent changes to handle ScalarArrayOpExpr as an indexable condition: when there are multiple possible index paths involving ScalarArrayOpExprs, they are logically to be ANDed together not ORed. This thinko was a direct consequence of trying to put the processing inside generate_bitmap_or_paths(), which I now see was a bit too cute. So pull it out and make the callers do it separately (there are only two that need it anyway). Partially responds to bug #2441 from Arjen van der Meijden. There are some additional infelicities exposed by his example, but they are also in 8.1.x, while this mistake is not.
2006-05-18 19:12:10 +02:00
/* Ignore RestrictInfos that aren't ORs */
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
if (!restriction_is_or_clause(rinfo))
continue;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We must be able to match at least one index to each of the arms of
* the OR, else we can't use it.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
pathlist = NIL;
foreach(j, ((BoolExpr *) rinfo->orclause)->args)
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Node *orarg = (Node *) lfirst(j);
List *indlist;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/* OR arguments should be ANDs or sub-RestrictInfos */
if (and_clause(orarg))
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
List *andargs = ((BoolExpr *) orarg)->args;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
indlist = build_paths_for_OR(root, rel,
andargs,
all_clauses);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/* Recurse in case there are sub-ORs */
indlist = list_concat(indlist,
generate_bitmap_or_paths(root, rel,
andargs,
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
all_clauses));
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
else
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
List *orargs;
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
Assert(IsA(orarg, RestrictInfo));
Assert(!restriction_is_or_clause((RestrictInfo *) orarg));
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
orargs = list_make1(orarg);
indlist = build_paths_for_OR(root, rel,
orargs,
all_clauses);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
}
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If nothing matched this arm, we can't do anything with this OR
* clause.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
if (indlist == NIL)
{
pathlist = NIL;
break;
}
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* OK, pick the most promising AND combination, and add it to
* pathlist.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bitmapqual = choose_bitmap_and(root, rel, indlist);
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
pathlist = lappend(pathlist, bitmapqual);
}
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Major planner/optimizer revision: get rid of PathOrder node type, store all ordering information in pathkeys lists (which are now lists of lists of PathKeyItem nodes, not just lists of lists of vars). This was a big win --- the code is smaller and IMHO more understandable than it was, even though it handles more cases. I believe the node changes will not force an initdb for anyone; planner nodes don't show up in stored rules.
1999-08-16 04:17:58 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we have a match for every arm, then turn them into a
* BitmapOrPath, and add to result list.
Allow index use with OR clauses.
1998-08-02 00:12:13 +02:00
*/
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
if (pathlist != NIL)
{
bitmapqual = (Path *) create_bitmap_or_path(root, rel, pathlist);
result = lappend(result, bitmapqual);
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
return result;
}
/*
* choose_bitmap_and
* Given a nonempty list of bitmap paths, AND them into one path.
*
* This is a nontrivial decision since we can legally use any subset of the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* given path set. We want to choose a good tradeoff between selectivity
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
* and cost of computing the bitmap.
*
* The result is either a single one of the inputs, or a BitmapAndPath
* combining multiple inputs.
*/
static Path *
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
choose_bitmap_and(PlannerInfo *root, RelOptInfo *rel, List *paths)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
{
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
int npaths = list_length(paths);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
PathClauseUsage **pathinfoarray;
PathClauseUsage *pathinfo;
List *clauselist;
List *bestpaths = NIL;
Cost bestcost = 0;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
int i,
j;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
ListCell *l;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Assert(npaths > 0); /* else caller error */
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
if (npaths == 1)
return (Path *) linitial(paths); /* easy case */
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
/*
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
* In theory we should consider every nonempty subset of the given paths.
* In practice that seems like overkill, given the crude nature of the
* estimates, not to mention the possible effects of higher-level AND and
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* OR clauses. Moreover, it's completely impractical if there are a large
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* number of paths, since the work would grow as O(2^N).
While determining the filter clauses for an index scan (either plain or bitmap), use pred_test to be a little smarter about cases where a filter clause is logically unnecessary. This may be overkill for the plain indexscan case, but it's definitely useful for OR'd bitmap scans.
2005-04-25 05:58:30 +02:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* As a heuristic, we first check for paths using exactly the same sets of
* WHERE clauses + index predicate conditions, and reject all but the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* cheapest-to-scan in any such group. This primarily gets rid of indexes
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* that include the interesting columns but also irrelevant columns. (In
* situations where the DBA has gone overboard on creating variant
* indexes, this can make for a very large reduction in the number of
* paths considered further.)
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* We then sort the surviving paths with the cheapest-to-scan first, and
* for each path, consider using that path alone as the basis for a bitmap
* scan. Then we consider bitmap AND scans formed from that path plus
* each subsequent (higher-cost) path, adding on a subsequent path if it
* results in a reduction in the estimated total scan cost. This means we
* consider about O(N^2) rather than O(2^N) path combinations, which is
* quite tolerable, especially given than N is usually reasonably small
* because of the prefiltering step. The cheapest of these is returned.
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* We will only consider AND combinations in which no two indexes use the
* same WHERE clause. This is a bit of a kluge: it's needed because
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* costsize.c and clausesel.c aren't very smart about redundant clauses.
* They will usually double-count the redundant clauses, producing a
* too-small selectivity that makes a redundant AND step look like it
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* reduces the total cost. Perhaps someday that code will be smarter and
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* we can remove this limitation. (But note that this also defends
* against flat-out duplicate input paths, which can happen because
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* match_join_clauses_to_index will find the same OR join clauses that
Extract restriction OR clauses whether or not they are indexable. It's possible to extract a restriction OR clause from a join clause that has the form of an OR-of-ANDs, if each sub-AND includes a clause that mentions only one specific relation. While PG has been aware of that idea for many years, the code previously only did it if it could extract an indexable OR clause. On reflection, though, that seems a silly limitation: adding a restriction clause can be a win by reducing the number of rows that have to be filtered at the join step, even if we have to test the clause as a plain filter clause during the scan. This should be especially useful for foreign tables, where the change can cut the number of rows that have to be retrieved from the foreign server; but testing shows it can win even on local tables. Per a suggestion from Robert Haas. As a heuristic, I made the code accept an extracted restriction clause if its estimated selectivity is less than 0.9, which will probably result in accepting extracted clauses just about always. We might need to tweak that later based on experience. Since the code no longer has even a weak connection to Path creation, remove orindxpath.c and create a new file optimizer/util/orclauses.c. There's some additional janitorial cleanup of now-dead code that needs to happen, but it seems like that's a fit subject for a separate commit.
2013-12-30 18:24:37 +01:00
* extract_restriction_or_clauses has pulled OR restriction clauses out
* of.)
Tweak nodeBitmapAnd to stop evaluating sub-plan scans if it finds it's got an empty bitmap after any step; the remaining subplans can no longer affect the result. Per a suggestion from Ilia Kantor.
2005-08-29 00:47:20 +02:00
*
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* For the same reason, we reject AND combinations in which an index
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* predicate clause duplicates another clause. Here we find it necessary
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* to be even stricter: we'll reject a partial index if any of its
* predicate clauses are implied by the set of WHERE clauses and predicate
* clauses used so far. This covers cases such as a condition "x = 42"
* used with a plain index, followed by a clauseless scan of a partial
* index "WHERE x >= 40 AND x < 50". The partial index has been accepted
* only because "x = 42" was present, and so allowing it would partially
* double-count selectivity. (We could use predicate_implied_by on
* regular qual clauses too, to have a more intelligent, but much more
* expensive, check for redundancy --- but in most cases simple equality
* seems to suffice.)
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*/
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Extract clause usage info and detect any paths that use exactly the
* same set of clauses; keep only the cheapest-to-scan of any such groups.
* The surviving paths are put into an array for qsort'ing.
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
*/
pathinfoarray = (PathClauseUsage **)
palloc(npaths * sizeof(PathClauseUsage *));
clauselist = NIL;
npaths = 0;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
foreach(l, paths)
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Path *ipath = (Path *) lfirst(l);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
pathinfo = classify_index_clause_usage(ipath, &clauselist);
for (i = 0; i < npaths; i++)
{
if (bms_equal(pathinfo->clauseids, pathinfoarray[i]->clauseids))
break;
}
if (i < npaths)
{
/* duplicate clauseids, keep the cheaper one */
Cost ncost;
Cost ocost;
Selectivity nselec;
Selectivity oselec;
cost_bitmap_tree_node(pathinfo->path, &ncost, &nselec);
cost_bitmap_tree_node(pathinfoarray[i]->path, &ocost, &oselec);
if (ncost < ocost)
pathinfoarray[i] = pathinfo;
}
else
{
/* not duplicate clauseids, add to array */
pathinfoarray[npaths++] = pathinfo;
}
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* If only one surviving path, we're done */
if (npaths == 1)
return pathinfoarray[0]->path;
/* Sort the surviving paths by index access cost */
qsort(pathinfoarray, npaths, sizeof(PathClauseUsage *),
path_usage_comparator);
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* For each surviving index, consider it as an "AND group leader", and see
* whether adding on any of the later indexes results in an AND path with
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* cheaper total cost than before. Then take the cheapest AND group.
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
*/
for (i = 0; i < npaths; i++)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Cost costsofar;
List *qualsofar;
Bitmapset *clauseidsofar;
ListCell *lastcell;
pathinfo = pathinfoarray[i];
paths = list_make1(pathinfo->path);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
costsofar = bitmap_scan_cost_est(root, rel, pathinfo->path);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
qualsofar = list_concat(list_copy(pathinfo->quals),
list_copy(pathinfo->preds));
clauseidsofar = bms_copy(pathinfo->clauseids);
lastcell = list_head(paths); /* for quick deletions */
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
for (j = i + 1; j < npaths; j++)
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Cost newcost;
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
pathinfo = pathinfoarray[j];
/* Check for redundancy */
if (bms_overlap(pathinfo->clauseids, clauseidsofar))
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
continue; /* consider it redundant */
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (pathinfo->preds)
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
bool redundant = false;
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* we check each predicate clause separately */
foreach(l, pathinfo->preds)
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Node *np = (Node *) lfirst(l);
if (predicate_implied_by(list_make1(np), qualsofar))
{
redundant = true;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
break; /* out of inner foreach loop */
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
}
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (redundant)
continue;
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* tentatively add new path to paths, so we can estimate cost */
paths = lappend(paths, pathinfo->path);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
newcost = bitmap_and_cost_est(root, rel, paths);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (newcost < costsofar)
{
/* keep new path in paths, update subsidiary variables */
costsofar = newcost;
qualsofar = list_concat(qualsofar,
list_copy(pathinfo->quals));
qualsofar = list_concat(qualsofar,
list_copy(pathinfo->preds));
clauseidsofar = bms_add_members(clauseidsofar,
pathinfo->clauseids);
lastcell = lnext(lastcell);
}
else
{
/* reject new path, remove it from paths list */
paths = list_delete_cell(paths, lnext(lastcell), lastcell);
}
Assert(lnext(lastcell) == NULL);
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* Keep the cheapest AND-group (or singleton) */
if (i == 0 || costsofar < bestcost)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
bestpaths = paths;
bestcost = costsofar;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* some easy cleanup (we don't try real hard though) */
list_free(qualsofar);
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (list_length(bestpaths) == 1)
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
return (Path *) linitial(bestpaths); /* no need for AND */
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
return (Path *) create_bitmap_and_path(root, rel, bestpaths);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/* qsort comparator to sort in increasing index access cost order */
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
static int
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
path_usage_comparator(const void *a, const void *b)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
{
Re-run pgindent with updated list of typedefs. (Updated README should avoid this problem in the future.)
2007-11-15 23:25:18 +01:00
PathClauseUsage *pa = *(PathClauseUsage *const *) a;
PathClauseUsage *pb = *(PathClauseUsage *const *) b;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Cost acost;
Cost bcost;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Selectivity aselec;
Selectivity bselec;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
cost_bitmap_tree_node(pa->path, &acost, &aselec);
cost_bitmap_tree_node(pb->path, &bcost, &bselec);
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
/*
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* If costs are the same, sort by selectivity.
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
*/
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (acost < bcost)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
return -1;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (acost > bcost)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
return 1;
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (aselec < bselec)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
return -1;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (aselec > bselec)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
return 1;
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
return 0;
}
/*
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* Estimate the cost of actually executing a bitmap scan with a single
Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered.
2012-04-26 20:17:13 +02:00
* index path (no BitmapAnd, at least not at this level; but it could be
* a BitmapOr).
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
*/
static Cost
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bitmap_scan_cost_est(PlannerInfo *root, RelOptInfo *rel, Path *ipath)
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
BitmapHeapPath bpath;
Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered.
2012-04-26 20:17:13 +02:00
Relids required_outer;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered.
2012-04-26 20:17:13 +02:00
/* Identify required outer rels, in case it's a parameterized scan */
required_outer = get_bitmap_tree_required_outer(ipath);
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Set up a dummy BitmapHeapPath */
bpath.path.type = T_BitmapHeapPath;
bpath.path.pathtype = T_BitmapHeapScan;
bpath.path.parent = rel;
Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered.
2012-04-26 20:17:13 +02:00
bpath.path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bpath.path.pathkeys = NIL;
bpath.bitmapqual = ipath;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
cost_bitmap_heap_scan(&bpath.path, root, rel,
bpath.path.param_info,
ipath,
Fix oversight in recent parameterized-path patch. bitmap_scan_cost_est() has to be able to cope with a BitmapOrPath, but I'd taken a shortcut that didn't work for that case. Noted by Heikki. Add some regression tests since this area is evidently under-covered.
2012-04-26 20:17:13 +02:00
get_loop_count(root, required_outer));
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
return bpath.path.total_cost;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
}
/*
* Estimate the cost of actually executing a BitmapAnd scan with the given
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
* inputs.
*/
static Cost
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bitmap_and_cost_est(PlannerInfo *root, RelOptInfo *rel, List *paths)
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
{
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
BitmapAndPath apath;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
BitmapHeapPath bpath;
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
Relids required_outer;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
/* Set up a dummy BitmapAndPath */
apath.path.type = T_BitmapAndPath;
apath.path.pathtype = T_BitmapAnd;
apath.path.parent = rel;
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
apath.path.param_info = NULL; /* not used in bitmap trees */
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
apath.path.pathkeys = NIL;
apath.bitmapquals = paths;
cost_bitmap_and_node(&apath, root);
/* Identify required outer rels, in case it's a parameterized scan */
required_outer = get_bitmap_tree_required_outer((Path *) &apath);
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Set up a dummy BitmapHeapPath */
bpath.path.type = T_BitmapHeapPath;
bpath.path.pathtype = T_BitmapHeapScan;
bpath.path.parent = rel;
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
bpath.path.param_info = get_baserel_parampathinfo(root, rel,
required_outer);
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
bpath.path.pathkeys = NIL;
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
bpath.bitmapqual = (Path *) &apath;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
/* Now we can do cost_bitmap_heap_scan */
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
cost_bitmap_heap_scan(&bpath.path, root, rel,
bpath.path.param_info,
(Path *) &apath,
get_loop_count(root, required_outer));
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
return bpath.path.total_cost;
Teach choose_bitmap_and() to actually be choosy --- that is, try to make some estimate of which available indexes to AND together, rather than blindly taking 'em all. This could probably stand further improvement, but it seems to do OK in simple tests.
2005-04-23 03:57:34 +02:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
/*
* classify_index_clause_usage
* Construct a PathClauseUsage struct describing the WHERE clauses and
* index predicate clauses used by the given indexscan path.
* We consider two clauses the same if they are equal().
*
* At some point we might want to migrate this info into the Path data
* structure proper, but for the moment it's only needed within
* choose_bitmap_and().
*
* *clauselist is used and expanded as needed to identify all the distinct
* clauses seen across successive calls. Caller must initialize it to NIL
* before first call of a set.
*/
static PathClauseUsage *
classify_index_clause_usage(Path *path, List **clauselist)
{
PathClauseUsage *result;
Bitmapset *clauseids;
ListCell *lc;
result = (PathClauseUsage *) palloc(sizeof(PathClauseUsage));
result->path = path;
/* Recursively find the quals and preds used by the path */
result->quals = NIL;
result->preds = NIL;
find_indexpath_quals(path, &result->quals, &result->preds);
/* Build up a bitmapset representing the quals and preds */
clauseids = NULL;
foreach(lc, result->quals)
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Node *node = (Node *) lfirst(lc);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
clauseids = bms_add_member(clauseids,
find_list_position(node, clauselist));
}
foreach(lc, result->preds)
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Node *node = (Node *) lfirst(lc);
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
clauseids = bms_add_member(clauseids,
find_list_position(node, clauselist));
}
result->clauseids = clauseids;
return result;
}
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
/*
* get_bitmap_tree_required_outer
* Find the required outer rels for a bitmap tree (index/and/or)
*
* We don't associate any particular parameterization with a BitmapAnd or
* BitmapOr node; however, the IndexPaths have parameterization info, in
* their capacity as standalone access paths. The parameterization required
* for the bitmap heap scan node is the union of rels referenced in the
* child IndexPaths.
*/
static Relids
get_bitmap_tree_required_outer(Path *bitmapqual)
{
Relids result = NULL;
ListCell *lc;
if (IsA(bitmapqual, IndexPath))
{
return bms_copy(PATH_REQ_OUTER(bitmapqual));
}
else if (IsA(bitmapqual, BitmapAndPath))
{
foreach(lc, ((BitmapAndPath *) bitmapqual)->bitmapquals)
{
result = bms_join(result,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
get_bitmap_tree_required_outer((Path *) lfirst(lc)));
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
}
}
else if (IsA(bitmapqual, BitmapOrPath))
{
foreach(lc, ((BitmapOrPath *) bitmapqual)->bitmapquals)
{
result = bms_join(result,
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
get_bitmap_tree_required_outer((Path *) lfirst(lc)));
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
}
}
else
elog(ERROR, "unrecognized node type: %d", nodeTag(bitmapqual));
return result;
}
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
/*
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* find_indexpath_quals
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
*
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* Given the Path structure for a plain or bitmap indexscan, extract lists
Fix choose_bitmap_and() so that partial index predicates are considered when deciding whether a potential additional indexscan is redundant or not. As now coded, any use of a partial index that was already used in a previous AND arm will be rejected as redundant. This might be overly restrictive, but not considering the point at all is definitely bad, as per example in bug #2441 from Arjen van der Meijden. In particular, a clauseless scan of a partial index was *never* considered redundant by the previous coding, and that's surely wrong. Being more flexible would also require some consideration of how not to double-count the index predicate's selectivity.
2006-05-18 21:56:46 +02:00
* of all the indexquals and index predicate conditions used in the Path.
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* These are appended to the initial contents of *quals and *preds (hence
* caller should initialize those to NIL).
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
*
Remove dead code now that orindxpath.c is history. We don't need make_restrictinfo_from_bitmapqual() anymore at all. generate_bitmap_or_paths() doesn't need to be exported, and we can drop its rather klugy restriction_only flag.
2013-12-30 18:50:31 +01:00
* Note we are not trying to produce an accurate representation of the AND/OR
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
* semantics of the Path, but just find out all the base conditions used.
*
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* The result lists contain pointers to the expressions used in the Path,
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
* but all the list cells are freshly built, so it's safe to destructively
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* modify the lists (eg, by concat'ing with other lists).
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
*/
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
static void
find_indexpath_quals(Path *bitmapqual, List **quals, List **preds)
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
{
if (IsA(bitmapqual, BitmapAndPath))
{
BitmapAndPath *apath = (BitmapAndPath *) bitmapqual;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
ListCell *l;
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
foreach(l, apath->bitmapquals)
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
find_indexpath_quals((Path *) lfirst(l), quals, preds);
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
}
}
else if (IsA(bitmapqual, BitmapOrPath))
{
BitmapOrPath *opath = (BitmapOrPath *) bitmapqual;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
ListCell *l;
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
foreach(l, opath->bitmapquals)
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
find_indexpath_quals((Path *) lfirst(l), quals, preds);
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
}
}
else if (IsA(bitmapqual, IndexPath))
{
IndexPath *ipath = (IndexPath *) bitmapqual;
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
*quals = list_concat(*quals, get_actual_clauses(ipath->indexclauses));
*preds = list_concat(*preds, list_copy(ipath->indexinfo->indpred));
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
}
else
elog(ERROR, "unrecognized node type: %d", nodeTag(bitmapqual));
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
/*
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* find_list_position
* Return the given node's position (counting from 0) in the given
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* list of nodes. If it's not equal() to any existing list member,
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
* add it at the end, and return that position.
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
*/
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
static int
find_list_position(Node *node, List **nodelist)
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
{
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
int i;
ListCell *lc;
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
i = 0;
foreach(lc, *nodelist)
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Node *oldnode = (Node *) lfirst(lc);
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
if (equal(node, oldnode))
return i;
i++;
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
}
Rewrite choose_bitmap_and() to make it more robust in the presence of competing alternatives for indexes to use in a bitmap scan. The former coding took estimated selectivity as an overriding factor, causing it to sometimes choose indexes that were much slower to scan than ones with a slightly worse selectivity. It was also too narrow-minded about which combinations of indexes to consider ANDing. The rewrite makes it pay more attention to index scan cost than selectivity; this seems sane since it's impossible to have very bad selectivity with low cost, whereas the reverse isn't true. Also, we now consider each index alone, as well as adding each index to an AND-group led by each prior index, for a total of about O(N^2) rather than O(N) combinations considered. This makes the results much less dependent on the exact order in which the indexes are considered. It's still a lot cheaper than an O(2^N) exhaustive search. A prefilter step eliminates all but the cheapest of those indexes using the same set of WHERE conditions, to keep the effective value of N down in scenarios where the DBA has created lots of partially-redundant indexes.
2007-04-17 22:03:03 +02:00
*nodelist = lappend(*nodelist, node);
return i;
Tweak choose_bitmap_and() heuristics in the light of example provided in bug #2075: consider an index redundant if any of its index conditions were already used, rather than if all of them were. Also, make the selectivity comparison a bit fuzzy, so that very small differences in estimated selectivities don't skew the results.
2005-11-30 18:10:19 +01:00
}
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
/*
* check_index_only
* Determine whether an index-only scan is possible for this index.
*/
static bool
check_index_only(RelOptInfo *rel, IndexOptInfo *index)
{
bool result;
Bitmapset *attrs_used = NULL;
Bitmapset *index_attrs = NULL;
ListCell *lc;
int i;
Replace simple constant pg_am.amcanreturn with an AM support function. The need for this was debated when we put in the index-only-scan feature, but at the time we had no near-term expectation of having AMs that could support such scans for only some indexes; so we kept it simple. However, the SP-GiST AM forces the issue, so let's fix it. This patch only installs the new API; no behavior actually changes.
2011-12-18 21:49:00 +01:00
/* Index-only scans must be enabled, and index must be capable of them */
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
if (!enable_indexonlyscan)
return false;
Replace simple constant pg_am.amcanreturn with an AM support function. The need for this was debated when we put in the index-only-scan feature, but at the time we had no near-term expectation of having AMs that could support such scans for only some indexes; so we kept it simple. However, the SP-GiST AM forces the issue, so let's fix it. This patch only installs the new API; no behavior actually changes.
2011-12-18 21:49:00 +01:00
if (!index->canreturn)
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
return false;
/*
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* Check that all needed attributes of the relation are available from the
* index.
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
*
* XXX this is overly conservative for partial indexes, since we will
* consider attributes involved in the index predicate as required even
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* though the predicate won't need to be checked at runtime. (The same is
* true for attributes used only in index quals, if we are certain that
* the index is not lossy.) However, it would be quite expensive to
* determine that accurately at this point, so for now we take the easy
* way out.
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
*/
/*
* Add all the attributes needed for joins or final output. Note: we must
* look at reltargetlist, not the attr_needed data, because attr_needed
* isn't computed for inheritance child rels.
*/
pull_varattnos((Node *) rel->reltargetlist, rel->relid, &attrs_used);
/* Add all the attributes used by restriction clauses. */
foreach(lc, rel->baserestrictinfo)
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
pull_varattnos((Node *) rinfo->clause, rel->relid, &attrs_used);
}
/* Construct a bitmapset of columns stored in the index. */
for (i = 0; i < index->ncolumns; i++)
{
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
int attno = index->indexkeys[i];
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
/*
* For the moment, we just ignore index expressions. It might be nice
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
2011-10-11 20:20:06 +02:00
* to do something with them, later.
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
*/
Rearrange the implementation of index-only scans. This commit changes index-only scans so that data is read directly from the index tuple without first generating a faux heap tuple. The only immediate benefit is that indexes on system columns (such as OID) can be used in index-only scans, but this is necessary infrastructure if we are ever to support index-only scans on expression indexes. The executor is now ready for that, though the planner still needs substantial work to recognize the possibility. To do this, Vars in index-only plan nodes have to refer to index columns not heap columns. I introduced a new special varno, INDEX_VAR, to mark such Vars to avoid confusion. (In passing, this commit renames the two existing special varnos to OUTER_VAR and INNER_VAR.) This allows ruleutils.c to handle them with logic similar to what we use for subplan reference Vars. Since index-only scans are now fundamentally different from regular indexscans so far as their expression subtrees are concerned, I also chose to change them to have their own plan node type (and hence, their own executor source file).
2011-10-11 20:20:06 +02:00
if (attno == 0)
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
continue;
index_attrs =
bms_add_member(index_attrs,
attno - FirstLowInvalidHeapAttributeNumber);
}
/* Do we have all the necessary attributes? */
result = bms_is_subset(attrs_used, index_attrs);
bms_free(attrs_used);
bms_free(index_attrs);
return result;
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
* get_loop_count
* Choose the loop count estimate to use for costing a parameterized path
* with the given set of outer relids.
*
* Since we produce parameterized paths before we've begun to generate join
* relations, it's impossible to predict exactly how many times a parameterized
* path will be iterated; we don't know the size of the relation that will be
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* on the outside of the nestloop. However, we should try to account for
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* multiple iterations somehow in costing the path. The heuristic embodied
* here is to use the rowcount of the smallest other base relation needed in
* the join clauses used by the path. (We could alternatively consider the
* largest one, but that seems too optimistic.) This is of course the right
* answer for single-other-relation cases, and it seems like a reasonable
* zero-order approximation for multiway-join cases.
*
* Note: for this to work, allpaths.c must establish all baserel size
* estimates before it begins to compute paths, or at least before it
* calls create_index_paths().
*/
static double
get_loop_count(PlannerInfo *root, Relids outer_relids)
{
double result = 1.0;
/* For a non-parameterized path, just return 1.0 quickly */
if (outer_relids != NULL)
{
int relid;
/* Need a working copy since bms_first_member is destructive */
outer_relids = bms_copy(outer_relids);
while ((relid = bms_first_member(outer_relids)) >= 0)
{
RelOptInfo *outer_rel;
/* Paranoia: ignore bogus relid indexes */
if (relid >= root->simple_rel_array_size)
continue;
outer_rel = root->simple_rel_array[relid];
if (outer_rel == NULL)
continue;
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
Assert(outer_rel->relid == relid); /* sanity check on array */
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Other relation could be proven empty, if so ignore */
if (IS_DUMMY_REL(outer_rel))
continue;
/* Otherwise, rel's rows estimate should be valid by now */
Assert(outer_rel->rows > 0);
/* Remember smallest row count estimate among the outer rels */
if (result == 1.0 || result > outer_rel->rows)
result = outer_rel->rows;
}
bms_free(outer_relids);
}
return result;
}
/****************************************************************************
* ---- ROUTINES TO CHECK QUERY CLAUSES ----
****************************************************************************/
/*
* match_restriction_clauses_to_index
* Identify restriction clauses for the rel that match the index.
* Matching clauses are added to *clauseset.
*/
static void
match_restriction_clauses_to_index(RelOptInfo *rel, IndexOptInfo *index,
IndexClauseSet *clauseset)
{
match_clauses_to_index(index, rel->baserestrictinfo, clauseset);
}
/*
* match_join_clauses_to_index
* Identify join clauses for the rel that match the index.
* Matching clauses are added to *clauseset.
* Also, add any potentially usable join OR clauses to *joinorclauses.
*/
static void
match_join_clauses_to_index(PlannerInfo *root,
RelOptInfo *rel, IndexOptInfo *index,
IndexClauseSet *clauseset,
List **joinorclauses)
{
ListCell *lc;
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
/* Scan the rel's join clauses */
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Revise parameterized-path mechanism to fix assorted issues. This patch adjusts the treatment of parameterized paths so that all paths with the same parameterization (same set of required outer rels) for the same relation will have the same rowcount estimate. We cache the rowcount estimates to ensure that property, and hopefully save a few cycles too. Doing this makes it practical for add_path_precheck to operate without a rowcount estimate: it need only assume that paths with different parameterizations never dominate each other, which is close enough to true anyway for coarse filtering, because normally a more-parameterized path should yield fewer rows thanks to having more join clauses to apply. In add_path, we do the full nine yards of comparing rowcount estimates along with everything else, so that we can discard parameterized paths that don't actually have an advantage. This fixes some issues I'd found with add_path rejecting parameterized paths on the grounds that they were more expensive than not-parameterized ones, even though they yielded many fewer rows and hence would be cheaper once subsequent joining was considered. To make the same-rowcounts assumption valid, we have to require that any parameterized path enforce *all* join clauses that could be obtained from the particular set of outer rels, even if not all of them are useful for indexing. This is required at both base scans and joins. It's a good thing anyway since the net impact is that join quals are checked at the lowest practical level in the join tree. Hence, discard the original rather ad-hoc mechanism for choosing parameterization joinquals, and build a better one that has a more principled rule for when clauses can be moved. The original rule was actually buggy anyway for lack of knowledge about which relations are part of an outer join's outer side; getting this right requires adding an outer_relids field to RestrictInfo.
2012-04-19 21:52:46 +02:00
/* Check if clause can be moved to this rel */
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
if (!join_clause_is_movable_to(rinfo, rel))
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
continue;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* Potentially usable, so see if it matches the index or is an OR */
if (restriction_is_or_clause(rinfo))
*joinorclauses = lappend(*joinorclauses, rinfo);
else
match_clause_to_index(index, rinfo, clauseset);
}
}
/*
* match_eclass_clauses_to_index
* Identify EquivalenceClass join clauses for the rel that match the index.
* Matching clauses are added to *clauseset.
*/
static void
match_eclass_clauses_to_index(PlannerInfo *root, IndexOptInfo *index,
IndexClauseSet *clauseset)
{
int indexcol;
/* No work if rel is not in any such ECs */
if (!index->rel->has_eclass_joins)
return;
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
{
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
ec_member_matches_arg arg;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
List *clauses;
Support index-only scans using the visibility map to avoid heap fetches. When a btree index contains all columns required by the query, and the visibility map shows that all tuples on a target heap page are visible-to-all, we don't need to fetch that heap page. This patch depends on the previous patches that made the visibility map reliable. There's a fair amount left to do here, notably trying to figure out a less chintzy way of estimating the cost of an index-only scan, but the core functionality seems ready to commit. Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.
2011-10-08 02:13:02 +02:00
Suppress creation of backwardly-indexed paths for LATERAL join clauses. Given a query such as SELECT * FROM foo JOIN LATERAL (SELECT foo.var1) ss(x) ON ss.x = foo.var2 the existence of the join clause "ss.x = foo.var2" encourages indxpath.c to build a parameterized path for foo using any index available for foo.var2. This is completely useless activity, though, since foo has got to be on the outside not the inside of any nestloop join with ss. It's reasonably inexpensive to add tests that prevent creation of such paths, so let's do that.
2012-08-30 20:32:22 +02:00
/* Generate clauses, skipping any that join to lateral_referencers */
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
arg.index = index;
arg.indexcol = indexcol;
clauses = generate_implied_equalities_for_column(root,
index->rel,
ec_member_matches_indexcol,
(void *) &arg,
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
index->rel->lateral_referencers);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
* We have to check whether the results actually do match the index,
* since for non-btree indexes the EC's equality operators might not
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
* be in the index opclass (cf ec_member_matches_indexcol).
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*/
match_clauses_to_index(index, clauses, clauseset);
}
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* match_clauses_to_index
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Perform match_clause_to_index() for each clause in a list.
* Matching clauses are added to *clauseset.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
static void
match_clauses_to_index(IndexOptInfo *index,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
List *clauses,
IndexClauseSet *clauseset)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
ListCell *lc;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
foreach(lc, clauses)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Assert(IsA(rinfo, RestrictInfo));
match_clause_to_index(index, rinfo, clauseset);
}
}
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
* match_clause_to_index
* Test whether a qual clause can be used with an index.
*
* If the clause is usable, add it to the appropriate list in *clauseset.
* *clauseset must be initialized to zeroes before first call.
*
* Note: in some circumstances we may find the same RestrictInfos coming from
* multiple places. Defend against redundant outputs by refusing to add a
* clause twice (pointer equality should be a good enough check for this).
*
* Note: it's possible that a badly-defined index could have multiple matching
* columns. We always select the first match if so; this avoids scenarios
* wherein we get an inflated idea of the index's selectivity by using the
* same clause multiple times with different index columns.
*/
static void
match_clause_to_index(IndexOptInfo *index,
RestrictInfo *rinfo,
IndexClauseSet *clauseset)
{
int indexcol;
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
{
if (match_clause_to_indexcol(index,
indexcol,
rinfo))
Be a little smarter in group_clauses_by_indexkey_for_join: detect cases where a joinclause is redundant with a restriction clause. Original coding believed this was impossible and didn't need to be checked for, but that was a thinko ...
2003-12-18 01:22:12 +01:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
clauseset->indexclauses[indexcol] =
list_append_unique_ptr(clauseset->indexclauses[indexcol],
rinfo);
clauseset->nonempty = true;
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
return;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
}
Further cleanup of indxpath logic related to IndexOptInfo.opfamily array. We no longer need the terminating zero entry in opfamily[], so get rid of it. Also replace assorted ad-hoc looping logic with simple for and foreach constructs. This code is now noticeably more readable than it was an hour ago; credit to Robert for seeing that it could be simplified.
2010-11-20 21:07:16 +01:00
}
Patches for Vadim's multikey indexing...
1997-03-18 19:41:37 +01:00
}
1. INDEXSCAN_PATCH changes: (op Param Var) should work too 2. IndexScanableOperand now uses match_indexkey_operand instead of equal_indexkey_var (if we have some index on attribute X then we shouldn't use it for 'where some_func(X) OP CONST').
1997-01-22 07:25:42 +01:00
/*
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
* match_clause_to_indexcol()
* Determines whether a restriction clause matches a column of an index.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* To match an index normally, the clause:
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
*
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
* (1) must be in the form (indexkey op const) or (const op indexkey);
* and
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* (2) must contain an operator which is in the same family as the index
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
* operator for this column, or is a "special" operator as recognized
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
* by match_special_index_operator();
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* and
* (3) must match the collation of the index, if collation is relevant.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Our definition of "const" is exceedingly liberal: we allow anything that
* doesn't involve a volatile function or a Var of the index's relation.
* In particular, Vars belonging to other relations of the query are
* accepted here, since a clause of that form can be used in a
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* parameterized indexscan. It's the responsibility of higher code levels
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* to manage restriction and join clauses appropriately.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Note: we do need to check for Vars of the index's relation on the
* "const" side of the clause, since clauses like (a.f1 OP (b.f2 OP a.f3))
* are not processable by a parameterized indexscan on a.f1, whereas
* something like (a.f1 OP (b.f2 OP c.f3)) is.
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
*
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
* Presently, the executor can only deal with indexquals that have the
* indexkey on the left, so we can only use clauses that have the indexkey
* on the right if we can commute the clause to put the key on the left.
* We do not actually do the commuting here, but we check whether a
* suitable commutator operator is available.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
* If the index has a collation, the clause must have the same collation.
* For collation-less indexes, we assume it doesn't matter; this is
* necessary for cases like "hstore ? text", wherein hstore's operators
* don't care about collation but the clause will get marked with a
* collation anyway because of the text argument. (This logic is
* embodied in the macro IndexCollMatchesExprColl.)
*
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* It is also possible to match RowCompareExpr clauses to indexes (but
* currently, only btree indexes handle this). In this routine we will
* report a match if the first column of the row comparison matches the
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* target index column. This is sufficient to guarantee that some index
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* condition can be constructed from the RowCompareExpr --- whether the
* remaining columns match the index too is considered in
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
* adjust_rowcompare_for_index().
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* It is also possible to match ScalarArrayOpExpr clauses to indexes, when
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* the clause is of the form "indexkey op ANY (arrayconst)".
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
* For boolean indexes, it is also possible to match the clause directly
* to the indexkey; or perhaps the clause is (NOT indexkey).
*
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
* 'index' is the index of interest.
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
* 'indexcol' is a column number of 'index' (counting from 0).
Rewrite OR indexscan processing to be more flexible. We can now for the first time generate an OR indexscan for a two-column index when the WHERE condition is like 'col1 = foo AND (col2 = bar OR col2 = baz)' --- before, the OR had to be on the first column of the index or we'd not notice the possibility of using it. Some progress towards extracting OR indexscans from subclauses of an OR that references multiple relations, too, although this code is #ifdef'd out because it needs more work.
2004-01-04 01:07:32 +01:00
* 'rinfo' is the clause to be tested (as a RestrictInfo node).
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
* Returns true if the clause can be used with this index key.
*
Further cleanup for OR-of-AND WHERE-clauses. orindxpath can now handle extracting from an AND subclause just those opclauses that are relevant for a particular index. For example, we can now consider using an index on x to process WHERE (x = 1 AND y = 2) OR (x = 2 AND y = 4) OR ...
2000-02-05 19:26:09 +01:00
* NOTE: returns false if clause is an OR or AND clause; it is the
* responsibility of higher-level routines to cope with those.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
static bool
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
match_clause_to_indexcol(IndexOptInfo *index,
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
int indexcol,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RestrictInfo *rinfo)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rewrite OR indexscan processing to be more flexible. We can now for the first time generate an OR indexscan for a two-column index when the WHERE condition is like 'col1 = foo AND (col2 = bar OR col2 = baz)' --- before, the OR had to be on the first column of the index or we'd not notice the possibility of using it. Some progress towards extracting OR indexscans from subclauses of an OR that references multiple relations, too, although this code is #ifdef'd out because it needs more work.
2004-01-04 01:07:32 +01:00
Expr *clause = rinfo->clause;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Index index_relid = index->rel->relid;
Further cleanup of indxpath logic related to IndexOptInfo.opfamily array. We no longer need the terminating zero entry in opfamily[], so get rid of it. Also replace assorted ad-hoc looping logic with simple for and foreach constructs. This code is now noticeably more readable than it was an hour ago; credit to Robert for seeing that it could be simplified.
2010-11-20 21:07:16 +01:00
Oid opfamily = index->opfamily[indexcol];
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid idxcollation = index->indexcollations[indexcol];
Allow merge and hash joins to occur on arbitrary expressions (anything not containing a volatile function), rather than only on 'Var = Var' clauses as before. This makes it practical to do flatten_join_alias_vars at the start of planning, which in turn eliminates a bunch of klugery inside the planner to deal with alias vars. As a free side effect, we now detect implied equality of non-Var expressions; for example in SELECT ... WHERE a.x = b.y and b.y = 42 we will deduce a.x = 42 and use that as a restriction qual on a. Also, we can remove the restriction introduced 12/5/02 to prevent pullup of subqueries whose targetlists contain sublinks. Still TODO: make statistical estimation routines in selfuncs.c and costsize.c smarter about expressions that are more complex than plain Vars. The need for this is considerably greater now that we have to be able to estimate the suitability of merge and hash join techniques on such expressions.
2003-01-15 20:35:48 +01:00
Node *leftop,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*rightop;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
Relids left_relids;
Relids right_relids;
Oid expr_op;
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
Oid expr_coll;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
bool plain_op;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Revise the planner's handling of "pseudoconstant" WHERE clauses, that is clauses containing no variables and no volatile functions. Such a clause can be used as a one-time qual in a gating Result plan node, to suppress plan execution entirely when it is false. Even when the clause is true, putting it in a gating node wins by avoiding repeated evaluation of the clause. In previous PG releases, query_planner() would do this for pseudoconstant clauses appearing at the top level of the jointree, but there was no ability to generate a gating Result deeper in the plan tree. To fix it, get rid of the special case in query_planner(), and instead process pseudoconstant clauses through the normal RestrictInfo qual distribution mechanism. When a pseudoconstant clause is found attached to a path node in create_plan(), pull it out and generate a gating Result at that point. This requires special-casing pseudoconstants in selectivity estimation and cost_qual_eval, but on the whole it's pretty clean. It probably even makes the planner a bit faster than before for the normal case of no pseudoconstants, since removing pull_constant_clauses saves one useless traversal of the qual tree. Per gripe from Phil Frost.
2006-07-01 20:38:33 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Never match pseudoconstants to indexes. (Normally this could not
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* happen anyway, since a pseudoconstant clause couldn't contain a Var,
* but what if someone builds an expression index on a constant? It's not
* totally unreasonable to do so with a partial index, either.)
Revise the planner's handling of "pseudoconstant" WHERE clauses, that is clauses containing no variables and no volatile functions. Such a clause can be used as a one-time qual in a gating Result plan node, to suppress plan execution entirely when it is false. Even when the clause is true, putting it in a gating node wins by avoiding repeated evaluation of the clause. In previous PG releases, query_planner() would do this for pseudoconstant clauses appearing at the top level of the jointree, but there was no ability to generate a gating Result deeper in the plan tree. To fix it, get rid of the special case in query_planner(), and instead process pseudoconstant clauses through the normal RestrictInfo qual distribution mechanism. When a pseudoconstant clause is found attached to a path node in create_plan(), pull it out and generate a gating Result at that point. This requires special-casing pseudoconstants in selectivity estimation and cost_qual_eval, but on the whole it's pretty clean. It probably even makes the planner a bit faster than before for the normal case of no pseudoconstants, since removing pull_constant_clauses saves one useless traversal of the qual tree. Per gripe from Phil Frost.
2006-07-01 20:38:33 +02:00
*/
if (rinfo->pseudoconstant)
return false;
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/* First check for boolean-index cases. */
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
if (IsBooleanOpfamily(opfamily))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
{
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
if (match_boolean_index_clause((Node *) clause, indexcol, index))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
return true;
}
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
* Clause must be a binary opclause, or possibly a ScalarArrayOpExpr
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* (which is always binary, by definition). Or it could be a
* RowCompareExpr, which we pass off to match_rowcompare_to_indexcol().
Support "x IS NOT NULL" clauses as indexscan conditions. This turns out to be just a minor extension of the previous patch that made "x IS NULL" indexable, because we can treat the IS NOT NULL condition as if it were "x < NULL" or "x > NULL" (depending on the index's NULLS FIRST/LAST option), just like IS NULL is treated like "x = NULL". Aside from any possible usefulness in its own right, this is an important improvement for index-optimized MAX/MIN aggregates: it is now reliably possible to get a column's min or max value cheaply, even when there are a lot of nulls cluttering the interesting end of the index.
2010-01-01 22:53:49 +01:00
* Or, if the index supports it, we can handle IS NULL/NOT NULL clauses.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*/
if (is_opclause(clause))
{
leftop = get_leftop(clause);
rightop = get_rightop(clause);
if (!leftop || !rightop)
return false;
left_relids = rinfo->left_relids;
right_relids = rinfo->right_relids;
expr_op = ((OpExpr *) clause)->opno;
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
expr_coll = ((OpExpr *) clause)->inputcollid;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
plain_op = true;
}
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
else if (clause && IsA(clause, ScalarArrayOpExpr))
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
{
ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) clause;
/* We only accept ANY clauses, not ALL */
if (!saop->useOr)
return false;
leftop = (Node *) linitial(saop->args);
rightop = (Node *) lsecond(saop->args);
left_relids = NULL; /* not actually needed */
right_relids = pull_varnos(rightop);
expr_op = saop->opno;
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
expr_coll = saop->inputcollid;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
plain_op = false;
}
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
else if (clause && IsA(clause, RowCompareExpr))
{
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return match_rowcompare_to_indexcol(index, indexcol,
opfamily, idxcollation,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
(RowCompareExpr *) clause);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
else if (index->amsearchnulls && IsA(clause, NullTest))
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
NullTest *nt = (NullTest *) clause;
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
Add an "argisrow" field to NullTest nodes, following a plan made way back in 8.2beta but never carried out. This avoids repetitive tests of whether the argument is of scalar or composite type. Also, be a bit more paranoid about composite arguments in some places where we previously weren't checking.
2010-01-02 00:03:10 +01:00
if (!nt->argisrow &&
match_index_to_operand((Node *) nt->arg, indexcol, index))
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
return true;
return false;
}
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
else
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
return false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
/*
pgindent run.
2003-08-04 02:43:34 +02:00
* Check for clauses of the form: (indexkey operator constant) or
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
* (constant operator indexkey). See above notes about const-ness.
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
*/
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
if (match_index_to_operand(leftop, indexcol, index) &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
!bms_is_member(index_relid, right_relids) &&
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
!contain_volatile_functions(rightop))
Patches for Vadim's multikey indexing...
1997-03-18 19:41:37 +01:00
{
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
if (IndexCollMatchesExprColl(idxcollation, expr_coll) &&
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
is_indexable_operator(expr_op, opfamily, true))
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
return true;
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
/*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* If we didn't find a member of the index's opfamily, see whether it
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* is a "special" indexable operator.
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (plain_op &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
match_special_index_operator(clause, opfamily,
idxcollation, true))
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
return true;
return false;
}
pgindent run over code.
1999-05-25 18:15:34 +02:00
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (plain_op &&
match_index_to_operand(rightop, indexcol, index) &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
!bms_is_member(index_relid, left_relids) &&
First cut at planner support for bitmap index scans. Lots to do yet, but the code is basically working. Along the way, rewrite the entire approach to processing OR index conditions, and make it work in join cases for the first time ever. orindxpath.c is now basically obsolete, but I left it in for the time being to allow easy comparison testing against the old implementation.
2005-04-22 23:58:32 +02:00
!contain_volatile_functions(leftop))
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
{
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
if (IndexCollMatchesExprColl(idxcollation, expr_coll) &&
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
is_indexable_operator(expr_op, opfamily, false))
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
return true;
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
/*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* If we didn't find a member of the index's opfamily, see whether it
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* is a "special" indexable operator.
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (match_special_index_operator(clause, opfamily,
idxcollation, false))
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
return true;
return false;
}
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
return false;
}
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
/*
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* is_indexable_operator
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* Does the operator match the specified index opfamily?
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
*
Another round of planner/optimizer work. This is just restructuring and code cleanup; no major improvements yet. However, EXPLAIN does produce more intuitive outputs for nested loops with indexscans now...
2000-01-09 01:26:47 +01:00
* If the indexkey is on the right, what we actually want to know
* is whether the operator has a commutator operator that matches
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* the opfamily.
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
static bool
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
is_indexable_operator(Oid expr_op, Oid opfamily, bool indexkey_on_left)
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
{
/* Get the commuted operator if necessary */
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (!indexkey_on_left)
{
expr_op = get_commutator(expr_op);
if (expr_op == InvalidOid)
return false;
}
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
/* OK if the (commuted) operator is a member of the index's opfamily */
return op_in_opfamily(expr_op, opfamily);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/*
* match_rowcompare_to_indexcol()
* Handles the RowCompareExpr case for match_clause_to_indexcol(),
* which see for comments.
*/
static bool
match_rowcompare_to_indexcol(IndexOptInfo *index,
int indexcol,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid opfamily,
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid idxcollation,
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
RowCompareExpr *clause)
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
{
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Index index_relid = index->rel->relid;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
Node *leftop,
*rightop;
Oid expr_op;
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid expr_coll;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/* Forget it if we're not dealing with a btree index */
if (index->relam != BTREE_AM_OID)
return false;
/*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* We could do the matching on the basis of insisting that the opfamily
* shown in the RowCompareExpr be the same as the index column's opfamily,
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* but that could fail in the presence of reverse-sort opfamilies: it'd be
* a matter of chance whether RowCompareExpr had picked the forward or
* reverse-sort family. So look only at the operator, and match if it is
* a member of the index's opfamily (after commutation, if the indexkey is
* on the right). We'll worry later about whether any additional
* operators are matchable to the index.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*/
leftop = (Node *) linitial(clause->largs);
rightop = (Node *) linitial(clause->rargs);
expr_op = linitial_oid(clause->opnos);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
expr_coll = linitial_oid(clause->inputcollids);
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
/* Collations must match, if relevant */
if (!IndexCollMatchesExprColl(idxcollation, expr_coll))
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return false;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/*
* These syntactic tests are the same as in match_clause_to_indexcol()
*/
if (match_index_to_operand(leftop, indexcol, index) &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
!bms_is_member(index_relid, pull_varnos(rightop)) &&
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
!contain_volatile_functions(rightop))
{
/* OK, indexkey is on left */
}
else if (match_index_to_operand(rightop, indexcol, index) &&
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
!bms_is_member(index_relid, pull_varnos(leftop)) &&
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
!contain_volatile_functions(leftop))
{
/* indexkey is on right, so commute the operator */
expr_op = get_commutator(expr_op);
if (expr_op == InvalidOid)
return false;
}
else
return false;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
/* We're good if the operator is the right type of opfamily member */
switch (get_op_opfamily_strategy(expr_op, opfamily))
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
{
case BTLessStrategyNumber:
case BTLessEqualStrategyNumber:
case BTGreaterEqualStrategyNumber:
case BTGreaterStrategyNumber:
return true;
}
return false;
}
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
/****************************************************************************
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* ---- ROUTINES TO CHECK ORDERING OPERATORS ----
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
****************************************************************************/
/*
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* match_pathkeys_to_index
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
* Test whether an index can produce output ordered according to the
* given pathkeys using "ordering operators".
*
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* If it can, return a list of suitable ORDER BY expressions, each of the form
* "indexedcol operator pseudoconstant", along with an integer list of the
* index column numbers (zero based) that each clause would be used with.
* NIL lists are returned if the ordering is not achievable this way.
*
* On success, the result list is ordered by pathkeys, and in fact is
* one-to-one with the requested pathkeys.
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
*/
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
static void
match_pathkeys_to_index(IndexOptInfo *index, List *pathkeys,
List **orderby_clauses_p,
List **clause_columns_p)
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
List *orderby_clauses = NIL;
List *clause_columns = NIL;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
ListCell *lc1;
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
*orderby_clauses_p = NIL; /* set default results */
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
*clause_columns_p = NIL;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
/* Only indexes with the amcanorderbyop property are interesting here */
if (!index->amcanorderbyop)
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
return;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
foreach(lc1, pathkeys)
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
PathKey *pathkey = (PathKey *) lfirst(lc1);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
bool found = false;
ListCell *lc2;
/*
* Note: for any failure to match, we just return NIL immediately.
* There is no value in matching just some of the pathkeys.
*/
/* Pathkey must request default sort order for the target opfamily */
if (pathkey->pk_strategy != BTLessStrategyNumber ||
pathkey->pk_nulls_first)
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
return;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
/* If eclass is volatile, no hope of using an indexscan */
if (pathkey->pk_eclass->ec_has_volatile)
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
return;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
Revisit handling of UNION ALL subqueries with non-Var output columns. In commit 57664ed25e5dea117158a2e663c29e60b3546e1c I tried to fix a bug reported by Teodor Sigaev by making non-simple-Var output columns distinct (by wrapping their expressions with dummy PlaceHolderVar nodes). This did not work too well. Commit b28ffd0fcc583c1811e5295279e7d4366c3cae6c fixed some ensuing problems with matching to child indexes, but per a recent report from Claus Stadler, constraint exclusion of UNION ALL subqueries was still broken, because constant-simplification didn't handle the injected PlaceHolderVars well either. On reflection, the original patch was quite misguided: there is no reason to expect that EquivalenceClass child members will be distinct. So instead of trying to make them so, we should ensure that we can cope with the situation when they're not. Accordingly, this patch reverts the code changes in the above-mentioned commits (though the regression test cases they added stay). Instead, I've added assorted defenses to make sure that duplicate EC child members don't cause any problems. Teodor's original problem ("MergeAppend child's targetlist doesn't match MergeAppend") is addressed more directly by revising prepare_sort_from_pathkeys to let the parent MergeAppend's sort list guide creation of each child's sort list. In passing, get rid of add_sort_column; as far as I can tell, testing for duplicate sort keys at this stage is dead code. Certainly it doesn't trigger often enough to be worth expending cycles on in ordinary queries. And keeping the test would've greatly complicated the new logic in prepare_sort_from_pathkeys, because comparing pathkey list entries against a previous output array requires that we not skip any entries in the list. Back-patch to 9.1, like the previous patches. The only known issue in this area that wasn't caused by the ill-advised previous patches was the MergeAppend planning failure, which of course is not relevant before 9.1. It's possible that we need some of the new defenses against duplicate child EC entries in older branches, but until there's some clear evidence of that I'm going to refrain from back-patching further.
2012-03-16 18:11:12 +01:00
/*
* Try to match eclass member expression(s) to index. Note that child
* EC members are considered, but only when they belong to the target
* relation. (Unlike regular members, the same expression could be a
* child member of more than one EC. Therefore, the same index could
* be considered to match more than one pathkey list, which is OK
* here. See also get_eclass_for_sort_expr.)
*/
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
foreach(lc2, pathkey->pk_eclass->ec_members)
{
EquivalenceMember *member = (EquivalenceMember *) lfirst(lc2);
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
int indexcol;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
/* No possibility of match if it references other relations */
if (!bms_equal(member->em_relids, index->rel->relids))
continue;
Improve commentary in match_pathkeys_to_index(). For a little while there I thought match_pathkeys_to_index() was broken because it wasn't trying to match index columns to pathkeys in order. Actually that's correct, because GiST can support ordering operators on any random collection of index columns, but it sure needs a comment.
2012-03-16 19:07:21 +01:00
/*
* We allow any column of the index to match each pathkey; they
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* don't have to match left-to-right as you might expect. This is
* correct for GiST, which is the sole existing AM supporting
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* amcanorderbyop. We might need different logic in future for
Improve commentary in match_pathkeys_to_index(). For a little while there I thought match_pathkeys_to_index() was broken because it wasn't trying to match index columns to pathkeys in order. Actually that's correct, because GiST can support ordering operators on any random collection of index columns, but it sure needs a comment.
2012-03-16 19:07:21 +01:00
* other implementations.
*/
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
for (indexcol = 0; indexcol < index->ncolumns; indexcol++)
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
Expr *expr;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
expr = match_clause_to_ordering_op(index,
indexcol,
member->em_expr,
pathkey->pk_opfamily);
if (expr)
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
orderby_clauses = lappend(orderby_clauses, expr);
clause_columns = lappend_int(clause_columns, indexcol);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
found = true;
break;
}
}
if (found) /* don't want to look at remaining members */
break;
}
if (!found) /* fail if no match for this pathkey */
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
return;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
}
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
*orderby_clauses_p = orderby_clauses; /* success! */
*clause_columns_p = clause_columns;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
}
/*
* match_clause_to_ordering_op
* Determines whether an ordering operator expression matches an
* index column.
*
* This is similar to, but simpler than, match_clause_to_indexcol.
* We only care about simple OpExpr cases. The input is a bare
* expression that is being ordered by, which must be of the form
* (indexkey op const) or (const op indexkey) where op is an ordering
* operator for the column's opfamily.
*
* 'index' is the index of interest.
* 'indexcol' is a column number of 'index' (counting from 0).
* 'clause' is the ordering expression to be tested.
* 'pk_opfamily' is the btree opfamily describing the required sort order.
*
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
* Note that we currently do not consider the collation of the ordering
* operator's result. In practical cases the result type will be numeric
* and thus have no collation, and it's not very clear what to match to
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* if it did have a collation. The index's collation should match the
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
* ordering operator's input collation, not its result.
*
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
* If successful, return 'clause' as-is if the indexkey is on the left,
* otherwise a commuted copy of 'clause'. If no match, return NULL.
*/
static Expr *
match_clause_to_ordering_op(IndexOptInfo *index,
int indexcol,
Expr *clause,
Oid pk_opfamily)
{
Oid opfamily = index->opfamily[indexcol];
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid idxcollation = index->indexcollations[indexcol];
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
Node *leftop,
*rightop;
Oid expr_op;
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
Oid expr_coll;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
Oid sortfamily;
bool commuted;
/*
* Clause must be a binary opclause.
*/
if (!is_opclause(clause))
return NULL;
leftop = get_leftop(clause);
rightop = get_rightop(clause);
if (!leftop || !rightop)
return NULL;
expr_op = ((OpExpr *) clause)->opno;
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
expr_coll = ((OpExpr *) clause)->inputcollid;
/*
* We can forget the whole thing right away if wrong collation.
*/
Fix index matching for operators with mixed collatable/noncollatable inputs. If an indexable operator for a non-collatable indexed datatype has a collatable right-hand input type, any OpExpr for it will be marked with a nonzero inputcollid (since having one collatable input is sufficient to make that happen). However, an index on a non-collatable column certainly doesn't have any collation. This caused us to fail to match such operators to their indexes, because indxpath.c required an exact match of index collation and clause collation. It seems correct to allow a match when the index is collation-less regardless of the clause's inputcollid: an operator with both noncollatable and collatable inputs could perhaps depend on the collation of the collatable input, but it could hardly expect the index for the noncollatable input to have that same collation. Per bug #6232 from Pierre Ducroquet. His example is specifically about "hstore ? text" but the problem seems quite generic.
2011-09-29 06:43:42 +02:00
if (!IndexCollMatchesExprColl(idxcollation, expr_coll))
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return NULL;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
/*
* Check for clauses of the form: (indexkey operator constant) or
* (constant operator indexkey).
*/
if (match_index_to_operand(leftop, indexcol, index) &&
!contain_var_clause(rightop) &&
!contain_volatile_functions(rightop))
{
commuted = false;
}
else if (match_index_to_operand(rightop, indexcol, index) &&
!contain_var_clause(leftop) &&
!contain_volatile_functions(leftop))
{
/* Might match, but we need a commuted operator */
expr_op = get_commutator(expr_op);
if (expr_op == InvalidOid)
return NULL;
commuted = true;
}
else
return NULL;
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Is the (commuted) operator an ordering operator for the opfamily? And
* if so, does it yield the right sorting semantics?
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
*/
sortfamily = get_op_opfamily_sortfamily(expr_op, opfamily);
if (sortfamily != pk_opfamily)
return NULL;
/* We have a match. Return clause or a commuted version thereof. */
if (commuted)
{
OpExpr *newclause = makeNode(OpExpr);
/* flat-copy all the fields of clause */
memcpy(newclause, clause, sizeof(OpExpr));
/* commute it */
newclause->opno = expr_op;
newclause->opfuncid = InvalidOid;
newclause->args = list_make2(rightop, leftop);
clause = (Expr *) newclause;
}
return clause;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/****************************************************************************
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* ---- ROUTINES TO DO PARTIAL INDEX PREDICATE TESTS ----
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
****************************************************************************/
Add the ability to extract OR indexscan conditions from OR-of-AND join conditions in which each OR subclause includes a constraint on the same relation. This implements the other useful side-effect of conversion to CNF format, without its unpleasant side-effects. As per pghackers discussion of a few weeks ago.
2004-01-05 06:07:36 +01:00
/*
* check_partial_indexes
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
* Check each partial index of the relation, and mark it predOK if
* the index's predicate is satisfied for this query.
*
* Note: it is possible for this to get re-run after adding more restrictions
* to the rel; so we might be able to prove more indexes OK. We assume that
* adding more restrictions can't make an index not OK.
Add the ability to extract OR indexscan conditions from OR-of-AND join conditions in which each OR subclause includes a constraint on the same relation. This implements the other useful side-effect of conversion to CNF format, without its unpleasant side-effects. As per pghackers discussion of a few weeks ago.
2004-01-05 06:07:36 +01:00
*/
void
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
check_partial_indexes(PlannerInfo *root, RelOptInfo *rel)
Add the ability to extract OR indexscan conditions from OR-of-AND join conditions in which each OR subclause includes a constraint on the same relation. This implements the other useful side-effect of conversion to CNF format, without its unpleasant side-effects. As per pghackers discussion of a few weeks ago.
2004-01-05 06:07:36 +01:00
{
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
List *clauselist;
bool have_partial;
Relids otherrels;
ListCell *lc;
Add the ability to extract OR indexscan conditions from OR-of-AND join conditions in which each OR subclause includes a constraint on the same relation. This implements the other useful side-effect of conversion to CNF format, without its unpleasant side-effects. As per pghackers discussion of a few weeks ago.
2004-01-05 06:07:36 +01:00
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
/*
* Frequently, there will be no partial indexes, so first check to make
* sure there's something useful to do here.
*/
have_partial = false;
foreach(lc, rel->indexlist)
Restructure pg_opclass, pg_amop, and pg_amproc per previous discussions in pgsql-hackers. pg_opclass now has a row for each opclass supported by each index AM, not a row for each opclass name. This allows pg_opclass to show directly whether an AM supports an opclass, and furthermore makes it possible to store additional information about an opclass that might be AM-dependent. pg_opclass and pg_amop now store "lossy" and "haskeytype" information that we previously expected the user to remember to provide in CREATE INDEX commands. Lossiness is no longer an index-level property, but is associated with the use of a particular operator in a particular index opclass. Along the way, IndexSupportInitialize now uses the syscaches to retrieve pg_amop and pg_amproc entries. I find this reduces backend launch time by about ten percent, at the cost of a couple more special cases in catcache.c's IndexScanOK. Initial work by Oleg Bartunov and Teodor Sigaev, further hacking by Tom Lane. initdb forced.
2001-08-21 18:36:06 +02:00
{
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
if (index->indpred == NIL)
continue; /* ignore non-partial indexes */
if (index->predOK)
continue; /* don't repeat work if already proven OK */
have_partial = true;
break;
}
if (!have_partial)
return;
/*
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* Construct a list of clauses that we can assume true for the purpose of
* proving the index(es) usable. Restriction clauses for the rel are
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
* always usable, and so are any join clauses that are "movable to" this
* rel. Also, we can consider any EC-derivable join clauses (which must
* be "movable to" this rel, by definition).
*/
clauselist = list_copy(rel->baserestrictinfo);
/* Scan the rel's join clauses */
foreach(lc, rel->joininfo)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
/* Check if clause can be moved to this rel */
Fix planner problems with LATERAL references in PlaceHolderVars. The planner largely failed to consider the possibility that a PlaceHolderVar's expression might contain a lateral reference to a Var coming from somewhere outside the PHV's syntactic scope. We had a previous report of a problem in this area, which I tried to fix in a quick-hack way in commit 4da6439bd8553059766011e2a42c6e39df08717f, but Antonin Houska pointed out that there were still some problems, and investigation turned up other issues. This patch largely reverts that commit in favor of a more thoroughly thought-through solution. The new theory is that a PHV's ph_eval_at level cannot be higher than its original syntactic level. If it contains lateral references, those don't change the ph_eval_at level, but rather they create a lateral-reference requirement for the ph_eval_at join relation. The code in joinpath.c needs to handle that. Another issue is that createplan.c wasn't handling nested PlaceHolderVars properly. In passing, push knowledge of lateral-reference checks for join clauses into join_clause_is_movable_to. This is mainly so that FDWs don't need to deal with it. This patch doesn't fix the original join-qual-placement problem reported by Jeremy Evans (and indeed, one of the new regression test cases shows the wrong answer because of that). But the PlaceHolderVar problems need to be fixed before that issue can be addressed, so committing this separately seems reasonable.
2013-08-18 02:22:37 +02:00
if (!join_clause_is_movable_to(rinfo, rel))
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
continue;
clauselist = lappend(clauselist, rinfo);
}
/*
* Add on any equivalence-derivable join clauses. Computing the correct
* relid sets for generate_join_implied_equalities is slightly tricky
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
* because the rel could be a child rel rather than a true baserel, and in
Fix some more problems with nested append relations. As of commit a87c72915 (which later got backpatched as far as 9.1), we're explicitly supporting the notion that append relations can be nested; this can occur when UNION ALL constructs are nested, or when a UNION ALL contains a table with inheritance children. Bug #11457 from Nelson Page, as well as an earlier report from Elvis Pranskevichus, showed that there were still nasty bugs associated with such cases: in particular the EquivalenceClass mechanism could try to generate "join" clauses connecting an appendrel child to some grandparent appendrel, which would result in assertion failures or bogus plans. Upon investigation I concluded that all current callers of find_childrel_appendrelinfo() need to be fixed to explicitly consider multiple levels of parent appendrels. The most complex fix was in processing of "broken" EquivalenceClasses, which are ECs for which we have been unable to generate all the derived equality clauses we would like to because of missing cross-type equality operators in the underlying btree operator family. That code path is more or less entirely untested by the regression tests to date, because no standard opfamilies have such holes in them. So I wrote a new regression test script to try to exercise it a bit, which turned out to be quite a worthwhile activity as it exposed existing bugs in all supported branches. The present patch is essentially the same as far back as 9.2, which is where parameterized paths were introduced. In 9.0 and 9.1, we only need to back-patch a small fragment of commit 5b7b5518d, which fixes failure to propagate out the original WHERE clauses when a broken EC contains constant members. (The regression test case results show that these older branches are noticeably stupider than 9.2+ in terms of the quality of the plans generated; but we don't really care about plan quality in such cases, only that the plan not be outright wrong. A more invasive fix in the older branches would not be a good idea anyway from a plan-stability standpoint.)
2014-10-02 01:30:24 +02:00
* that case we must remove its parents' relid(s) from all_baserels.
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
*/
if (rel->reloptkind == RELOPT_OTHER_MEMBER_REL)
otherrels = bms_difference(root->all_baserels,
Fix some more problems with nested append relations. As of commit a87c72915 (which later got backpatched as far as 9.1), we're explicitly supporting the notion that append relations can be nested; this can occur when UNION ALL constructs are nested, or when a UNION ALL contains a table with inheritance children. Bug #11457 from Nelson Page, as well as an earlier report from Elvis Pranskevichus, showed that there were still nasty bugs associated with such cases: in particular the EquivalenceClass mechanism could try to generate "join" clauses connecting an appendrel child to some grandparent appendrel, which would result in assertion failures or bogus plans. Upon investigation I concluded that all current callers of find_childrel_appendrelinfo() need to be fixed to explicitly consider multiple levels of parent appendrels. The most complex fix was in processing of "broken" EquivalenceClasses, which are ECs for which we have been unable to generate all the derived equality clauses we would like to because of missing cross-type equality operators in the underlying btree operator family. That code path is more or less entirely untested by the regression tests to date, because no standard opfamilies have such holes in them. So I wrote a new regression test script to try to exercise it a bit, which turned out to be quite a worthwhile activity as it exposed existing bugs in all supported branches. The present patch is essentially the same as far back as 9.2, which is where parameterized paths were introduced. In 9.0 and 9.1, we only need to back-patch a small fragment of commit 5b7b5518d, which fixes failure to propagate out the original WHERE clauses when a broken EC contains constant members. (The regression test case results show that these older branches are noticeably stupider than 9.2+ in terms of the quality of the plans generated; but we don't really care about plan quality in such cases, only that the plan not be outright wrong. A more invasive fix in the older branches would not be a good idea anyway from a plan-stability standpoint.)
2014-10-02 01:30:24 +02:00
find_childrel_parents(root, rel));
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
else
otherrels = bms_difference(root->all_baserels, rel->relids);
if (!bms_is_empty(otherrels))
clauselist =
list_concat(clauselist,
generate_join_implied_equalities(root,
pgindent run for release 9.3 This is the first run of the Perl-based pgindent script. Also update pgindent instructions.
2013-05-29 22:58:43 +02:00
bms_union(rel->relids,
otherrels),
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
otherrels,
rel));
/* Now try to prove each index predicate true */
foreach(lc, rel->indexlist)
{
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
Revise executor APIs so that all per-query state structure is built in a per-query memory context created by CreateExecutorState --- and destroyed by FreeExecutorState. This provides a final solution to the longstanding problem of memory leaked by various ExecEndNode calls.
2002-12-15 17:17:59 +01:00
Separate predicate-testing code out of indxpath.c, making it a module in its own right. As proposed by Simon Riggs, but with some editorializing of my own.
2005-06-11 00:25:37 +02:00
if (index->indpred == NIL)
continue; /* ignore non-partial indexes */
SQL-language functions are now callable in ordinary fmgr contexts ... for example, an SQL function can be used in a functional index. (I make no promises about speed, but it'll work ;-).) Clean up and simplify handling of functions returning sets.
2000-08-24 05:29:15 +02:00
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
if (index->predOK)
continue; /* don't repeat work if already proven OK */
Improve check_partial_indexes() to consider join clauses in proof attempts. Traditionally check_partial_indexes() has only looked at restriction clauses while trying to prove partial indexes usable in queries. However, join clauses can also be used in some cases; mainly, that a strict operator on "x" proves an "x IS NOT NULL" index predicate, even if the operator is in a join clause rather than a restriction clause. Adding this code fixes a regression in 9.2, because previously we would take join clauses into account when considering whether a partial index could be used in a nestloop inner indexscan path. 9.2 doesn't handle nestloop inner indexscans in the same way, and this consideration was overlooked in the rewrite. Moving the work to check_partial_indexes() is a better solution anyway, since the proof applies whether or not we actually use the index in that particular way, and we don't have to do it over again for each possible outer relation. Per report from Dave Cramer.
2012-11-16 01:29:05 +01:00
index->predOK = predicate_implied_by(index->indpred, clauselist);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/****************************************************************************
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* ---- ROUTINES TO CHECK EXTERNALLY-VISIBLE CONDITIONS ----
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
****************************************************************************/
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
* ec_member_matches_indexcol
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* Test whether an EquivalenceClass member matches an index column.
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
*
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
* This is a callback for use by generate_implied_equalities_for_column.
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
*/
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
static bool
ec_member_matches_indexcol(PlannerInfo *root, RelOptInfo *rel,
EquivalenceClass *ec, EquivalenceMember *em,
void *arg)
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
{
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
IndexOptInfo *index = ((ec_member_matches_arg *) arg)->index;
int indexcol = ((ec_member_matches_arg *) arg)->indexcol;
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
Oid curFamily = index->opfamily[indexcol];
Oid curCollation = index->indexcollations[indexcol];
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
/*
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* If it's a btree index, we can reject it if its opfamily isn't
* compatible with the EC, since no clause generated from the EC could be
* used with the index. For non-btree indexes, we can't easily tell
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* whether clauses generated from the EC could be used with the index, so
* don't check the opfamily. This might mean we return "true" for a
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* useless EC, so we have to recheck the results of
Redo postgres_fdw's planner code so it can handle parameterized paths. I wasn't going to ship this without having at least some example of how to do that. This version isn't terribly bright; in particular it won't consider any combinations of multiple join clauses. Given the cost of executing a remote EXPLAIN, I'm not sure we want to be very aggressive about doing that, anyway. In support of this, refactor generate_implied_equalities_for_indexcol so that it can be used to extract equivalence clauses that aren't necessarily tied to an index.
2013-03-22 00:43:59 +01:00
* generate_implied_equalities_for_column; see
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
* match_eclass_clauses_to_index.
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
*/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
if (index->relam == BTREE_AM_OID &&
!list_member_oid(ec->ec_opfamilies, curFamily))
return false;
Restructure planning of nestloop inner indexscans so that the set of usable joinclauses is determined accurately for each join. Formerly, the code only considered joinclauses that used all of the rels from the outer side of the join; thus for example FROM (a CROSS JOIN b) JOIN c ON (c.f1 = a.x AND c.f2 = b.y) could not exploit a two-column index on c(f1,f2), since neither of the qual clauses would be in the joininfo list it looked in. The new code does this correctly, and also is able to eliminate redundant clauses, thus fixing the problem noted 24-Oct-02 by Hans-Jürgen Schönig.
2002-11-24 22:52:15 +01:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/* We insist on collation match for all index types, though */
if (!IndexCollMatchesExprColl(curCollation, ec->ec_collation))
return false;
Revert my best_inner_indexscan patch of yesterday, which turns out to have had a bad side-effect: it stopped finding plans that involved BitmapAnd combinations of indexscans using both join and non-join conditions. Instead, make choose_bitmap_and more aggressive about detecting redundancies between BitmapOr subplans.
2006-04-09 20:18:41 +02:00
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
return match_index_to_operand((Node *) em->em_expr, indexcol, index);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
/*
* relation_has_unique_index_for
* Determine whether the relation provably has at most one row satisfying
* a set of equality conditions, because the conditions constrain all
* columns of some unique index.
*
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
* The conditions can be represented in either or both of two ways:
* 1. A list of RestrictInfo nodes, where the caller has already determined
* that each condition is a mergejoinable equality with an expression in
* this relation on one side, and an expression not involving this relation
* on the other. The transient outer_is_left flag is used to identify which
* side we should look at: left side if outer_is_left is false, right side
* if it is true.
* 2. A list of expressions in this relation, and a corresponding list of
* equality operators. The caller must have already checked that the operators
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* represent equality. (Note: the operators could be cross-type; the
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
* expressions should correspond to their RHS inputs.)
*
* The caller need only supply equality conditions arising from joins;
* this routine automatically adds in any usable baserestrictinfo clauses.
* (Note that the passed-in restrictlist will be destructively modified!)
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
*/
bool
relation_has_unique_index_for(PlannerInfo *root, RelOptInfo *rel,
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
List *restrictlist,
List *exprlist, List *oprlist)
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
{
ListCell *ic;
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
Assert(list_length(exprlist) == list_length(oprlist));
/* Short-circuit if no indexes... */
if (rel->indexlist == NIL)
return false;
/*
* Examine the rel's restriction clauses for usable var = const clauses
* that we can add to the restrictlist.
*/
foreach(ic, rel->baserestrictinfo)
{
RestrictInfo *restrictinfo = (RestrictInfo *) lfirst(ic);
/*
* Note: can_join won't be set for a restriction clause, but
* mergeopfamilies will be if it has a mergejoinable operator and
* doesn't contain volatile functions.
*/
if (restrictinfo->mergeopfamilies == NIL)
continue; /* not mergejoinable */
/*
* The clause certainly doesn't refer to anything but the given rel.
* If either side is pseudoconstant then we can use it.
*/
if (bms_is_empty(restrictinfo->left_relids))
{
/* righthand side is inner */
restrictinfo->outer_is_left = true;
}
else if (bms_is_empty(restrictinfo->right_relids))
{
/* lefthand side is inner */
restrictinfo->outer_is_left = false;
}
else
continue;
/* OK, add to list */
restrictlist = lappend(restrictlist, restrictinfo);
}
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
/* Short-circuit the easy case */
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
if (restrictlist == NIL && exprlist == NIL)
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
return false;
/* Examine each index of the relation ... */
foreach(ic, rel->indexlist)
{
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
IndexOptInfo *ind = (IndexOptInfo *) lfirst(ic);
int c;
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
/*
Don't trust deferred-unique indexes for join removal. The uniqueness condition might fail to hold intra-transaction, and assuming it does can give incorrect query results. Per report from Marti Raudsepp, though this is not his proposed patch. Back-patch to 9.0, where both these features were introduced. In the released branches, add the new IndexOptInfo field to the end of the struct, to try to minimize ABI breakage for third-party code that may be examining that struct.
2011-10-23 06:43:39 +02:00
* If the index is not unique, or not immediately enforced, or if it's
* a partial index that doesn't match the query, it's useless here.
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
*/
Don't trust deferred-unique indexes for join removal. The uniqueness condition might fail to hold intra-transaction, and assuming it does can give incorrect query results. Per report from Marti Raudsepp, though this is not his proposed patch. Back-patch to 9.0, where both these features were introduced. In the released branches, add the new IndexOptInfo field to the end of the struct, to try to minimize ABI breakage for third-party code that may be examining that struct.
2011-10-23 06:43:39 +02:00
if (!ind->unique || !ind->immediate ||
(ind->indpred != NIL && !ind->predOK))
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
continue;
/*
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
* Try to find each index column in the lists of conditions. This is
* O(N^2) or worse, but we expect all the lists to be short.
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
*/
for (c = 0; c < ind->ncolumns; c++)
{
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
bool matched = false;
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
ListCell *lc;
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
ListCell *lc2;
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
foreach(lc, restrictlist)
{
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lc);
Node *rexpr;
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
/*
* The condition's equality operator must be a member of the
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* index opfamily, else it is not asserting the right kind of
* equality behavior for this index. We check this first
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
* since it's probably cheaper than match_index_to_operand().
*/
if (!list_member_oid(rinfo->mergeopfamilies, ind->opfamily[c]))
continue;
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* XXX at some point we may need to check collations here too.
* For the moment we assume all collations reduce to the same
* notion of equality.
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
*/
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
/* OK, see if the condition operand matches the index key */
if (rinfo->outer_is_left)
rexpr = get_rightop(rinfo->clause);
else
rexpr = get_leftop(rinfo->clause);
if (match_index_to_operand(rexpr, c, ind))
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
{
matched = true; /* column is unique */
break;
}
}
if (matched)
continue;
forboth(lc, exprlist, lc2, oprlist)
{
Node *expr = (Node *) lfirst(lc);
Oid opr = lfirst_oid(lc2);
/* See if the expression matches the index key */
if (!match_index_to_operand(expr, c, ind))
continue;
/*
* The equality operator must be a member of the index
* opfamily, else it is not asserting the right kind of
* equality behavior for this index. We assume the caller
* determined it is an equality operator, so we don't need to
* check any more tightly than this.
*/
if (!op_in_opfamily(opr, ind->opfamily[c]))
continue;
/*
* XXX at some point we may need to check collations here too.
* For the moment we assume all collations reduce to the same
* notion of equality.
*/
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
matched = true; /* column is unique */
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
break;
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
}
Improve planner's ability to recognize cases where an IN's RHS is unique. If the right-hand side of a semijoin is unique, then we can treat it like a normal join (or another way to say that is: we don't need to explicitly unique-ify the data before doing it as a normal join). We were recognizing such cases when the RHS was a sub-query with appropriate DISTINCT or GROUP BY decoration, but there's another way: if the RHS is a plain relation with unique indexes, we can check if any of the indexes prove the output is unique. Most of the infrastructure for that was there already in the join removal code, though I had to rearrange it a bit. Per reflection about a recent example in pgsql-performance.
2011-10-26 23:52:02 +02:00
if (!matched)
Implement "join removal" for cases where the inner side of a left join is unique and is not referenced above the join. In this case the inner side doesn't affect the query result and can be thrown away entirely. Although perhaps nobody would ever write such a thing by hand, it's a reasonably common case in machine-generated SQL. The current implementation only recognizes the case where the inner side is a simple relation with a unique index matching the query conditions. This is enough for the use-cases that have been shown so far, but we might want to try to handle other cases later. Robert Haas, somewhat rewritten by Tom
2009-09-17 22:49:29 +02:00
break; /* no match; this index doesn't help us */
}
/* Matched all columns of this index? */
if (c == ind->ncolumns)
return true;
}
return false;
}
Teach planner to optionally ignore index columns that have an equality constraint while determining whether the index sort order matches the query's ORDER BY. This for example allows an index on (x,y) to match ... WHERE x = 42 ORDER BY y; It only works for btree indexes, but since those are the only ones we currently have that are ordered at all, that's good enough for now. Per popular demand.
2005-06-14 06:04:30 +02:00
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
/****************************************************************************
* ---- ROUTINES TO CHECK OPERANDS ----
****************************************************************************/
/*
* match_index_to_operand()
* Generalized test for a match between an index's key
* and the operand on one side of a restriction or join clause.
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
*
* operand: the nodetree to be compared to the index
* indexcol: the column number of the index (counting from 0)
* index: the index of interest
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
*
* Note that we aren't interested in collations here; the caller must check
* for a collation match, if it's dealing with an operator where that matters.
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
*
* This is exported for use in selfuncs.c.
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
*/
Create the planner mechanism for optimizing simple MIN and MAX queries into indexscans on matching indexes. For the moment, it only handles int4 and text datatypes; next step is to add a column to pg_aggregate so that all MIN/MAX aggregates can be handled. Per my recent proposal.
2005-04-12 01:06:57 +02:00
bool
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
match_index_to_operand(Node *operand,
int indexcol,
Another round of planner/optimizer work. This is just restructuring and code cleanup; no major improvements yet. However, EXPLAIN does produce more intuitive outputs for nested loops with indexscans now...
2000-01-09 01:26:47 +01:00
IndexOptInfo *index)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
int indkey;
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Ignore any RelabelType node above the operand. This is needed to be
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* able to apply indexscanning in binary-compatible-operator cases. Note:
* we can assume there is at most one RelabelType node;
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
* eval_const_expressions() will have simplified if more than one.
*/
if (operand && IsA(operand, RelabelType))
Allow merge and hash joins to occur on arbitrary expressions (anything not containing a volatile function), rather than only on 'Var = Var' clauses as before. This makes it practical to do flatten_join_alias_vars at the start of planning, which in turn eliminates a bunch of klugery inside the planner to deal with alias vars. As a free side effect, we now detect implied equality of non-Var expressions; for example in SELECT ... WHERE a.x = b.y and b.y = 42 we will deduce a.x = 42 and use that as a restriction qual on a. Also, we can remove the restriction introduced 12/5/02 to prevent pullup of subqueries whose targetlists contain sublinks. Still TODO: make statistical estimation routines in selfuncs.c and costsize.c smarter about expressions that are more complex than plain Vars. The need for this is considerably greater now that we have to be able to estimate the suitability of merge and hash join techniques on such expressions.
2003-01-15 20:35:48 +01:00
operand = (Node *) ((RelabelType *) operand)->arg;
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
indkey = index->indexkeys[indexcol];
if (indkey != 0)
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
{
/*
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
* Simple index column; operand must be a matching Var.
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
*/
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
if (operand && IsA(operand, Var) &&
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
index->rel->relid == ((Var *) operand)->varno &&
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
indkey == ((Var *) operand)->varattno)
Major planner/optimizer revision: get rid of PathOrder node type, store all ordering information in pathkeys lists (which are now lists of lists of PathKeyItem nodes, not just lists of lists of vars). This was a big win --- the code is smaller and IMHO more understandable than it was, even though it handles more cases. I believe the node changes will not force an initdb for anyone; planner nodes don't show up in stored rules.
1999-08-16 04:17:58 +02:00
return true;
Minor code beautification, extensive improvement of comments. This file was full of obsolete and just plain wrong commentary...
1999-07-23 05:34:49 +02:00
}
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
else
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Index expression; find the correct expression. (This search could
* be avoided, at the cost of complicating all the callers of this
* routine; doesn't seem worth it.)
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
*/
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *indexpr_item;
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
int i;
Node *indexkey;
Clean up optimizer's handling of indexscan quals that need to be commuted (ie, the index var appears on the right). These are now handled the same way as merge and hash join quals that need to be commuted: the actual reversing of the clause only happens if we actually choose the path and generate a plan from it. Furthermore, the clause is only reversed in the 'indexqual' field of the plan, not in the 'indxqualorig' field. This allows the clause to still be recognized and removed from qpquals of upper level join plans. Also, simplify and generalize match_clause_to_indexkey; now it recognizes binary-compatible indexes for join as well as restriction clauses.
1999-08-12 06:32:54 +02:00
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexpr_item = list_head(index->indexprs);
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
for (i = 0; i < indexcol; i++)
{
if (index->indexkeys[i] == 0)
{
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
if (indexpr_item == NULL)
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
elog(ERROR, "wrong number of index expressions");
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexpr_item = lnext(indexpr_item);
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
}
}
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
if (indexpr_item == NULL)
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
elog(ERROR, "wrong number of index expressions");
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexkey = (Node *) lfirst(indexpr_item);
pgindent run.
2003-08-04 02:43:34 +02:00
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
/*
* Does it match the operand? Again, strip any relabeling.
*/
if (indexkey && IsA(indexkey, RelabelType))
indexkey = (Node *) ((RelabelType *) indexkey)->arg;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
if (equal(indexkey, operand))
return true;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
return false;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
/****************************************************************************
* ---- ROUTINES FOR "SPECIAL" INDEXABLE OPERATORS ----
****************************************************************************/
Use parameterized paths to generate inner indexscans more flexibly. This patch fixes the planner so that it can generate nestloop-with- inner-indexscan plans even with one or more levels of joining between the indexscan and the nestloop join that is supplying the parameter. The executor was fixed to handle such cases some time ago, but the planner was not ready. This should improve our plans in many situations where join ordering restrictions formerly forced complete table scans. There is probably a fair amount of tuning work yet to be done, because of various heuristics that have been added to limit the number of parameterized paths considered. However, we are not going to find out what needs to be adjusted until the code gets some real-world use, so it's time to get it in there where it can be tested easily. Note API change for index AM amcostestimate functions. I'm not aware of any non-core index AMs, but if there are any, they will need minor adjustments.
2012-01-28 01:26:38 +01:00
/*
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
* These routines handle special optimization of operators that can be
* used with index scans even though they are not known to the executor's
* indexscan machinery. The key idea is that these operators allow us
* to derive approximate indexscan qual clauses, such that any tuples
* that pass the operator clause itself must also satisfy the simpler
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* indexscan condition(s). Then we can use the indexscan machinery
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
* to avoid scanning as much of the table as we'd otherwise have to,
* while applying the original operator as a qpqual condition to ensure
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* we deliver only the tuples we want. (In essence, we're using a regular
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
* index as if it were a lossy index.)
*
* An example of what we're doing is
* textfield LIKE 'abc%'
* from which we can generate the indexscanable conditions
* textfield >= 'abc' AND textfield < 'abd'
* which allow efficient scanning of an index on textfield.
* (In reality, character set and collation issues make the transformation
* from LIKE to indexscan limits rather harder than one might think ...
* but that's the basic idea.)
*
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
* Another thing that we do with this machinery is to provide special
* smarts for "boolean" indexes (that is, indexes on boolean columns
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* that support boolean equality). We can transform a plain reference
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
* to the indexkey into "indexkey = true", or "NOT indexkey" into
* "indexkey = false", so as to make the expression indexable using the
* regular index operators. (As of Postgres 8.1, we must do this here
* because constant simplification does the reverse transformation;
* without this code there'd be no way to use such an index at all.)
*
* Three routines are provided here:
*
* match_special_index_operator() is just an auxiliary function for
* match_clause_to_indexcol(); after the latter fails to recognize a
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* restriction opclause's operator as a member of an index's opfamily,
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
* it asks match_special_index_operator() whether the clause should be
* considered an indexqual anyway.
*
* match_boolean_index_clause() similarly detects clauses that can be
* converted into boolean equality operators.
*
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* expand_indexqual_conditions() converts a list of RestrictInfo nodes
* (with implicit AND semantics across list elements) into a list of clauses
* that the executor can actually handle. For operators that are members of
* the index's opfamily this transformation is a no-op, but clauses recognized
* by match_special_index_operator() or match_boolean_index_clause() must be
* converted into one or more "regular" indexqual conditions.
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
*/
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/*
* match_boolean_index_clause
* Recognize restriction clauses that can be matched to a boolean index.
*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* This should be called only when IsBooleanOpfamily() recognizes the
* index's operator family. We check to see if the clause matches the
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
* index's key.
*/
static bool
match_boolean_index_clause(Node *clause,
int indexcol,
IndexOptInfo *index)
{
/* Direct match? */
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
if (match_index_to_operand(clause, indexcol, index))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
return true;
/* NOT clause? */
if (not_clause(clause))
{
if (match_index_to_operand((Node *) get_notclausearg((Expr *) clause),
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
indexcol, index))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
return true;
}
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/*
* Since we only consider clauses at top level of WHERE, we can convert
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* indexkey IS TRUE and indexkey IS FALSE to index searches as well. The
* different meaning for NULL isn't important.
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
*/
else if (clause && IsA(clause, BooleanTest))
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
BooleanTest *btest = (BooleanTest *) clause;
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
if (btest->booltesttype == IS_TRUE ||
btest->booltesttype == IS_FALSE)
if (match_index_to_operand((Node *) btest->arg,
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
indexcol, index))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
return true;
}
return false;
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
/*
* match_special_index_operator
* Recognize restriction clauses that can be used to generate
* additional indexscanable qualifications.
*
* The given clause is already known to be a binary opclause having
Clean up handling of variable-free qual clauses. System now does the right thing with variable-free clauses that contain noncachable functions, such as 'WHERE random() < 0.5' --- these are evaluated once per potential output tuple. Expressions that contain only Params are now candidates to be indexscan quals --- for example, 'var = ($1 + 1)' can now be indexed. Cope with RelabelType nodes atop potential indexscan variables --- this oversight prevents 7.0.* from recognizing some potentially indexscanable situations.
2000-08-13 04:50:35 +02:00
* the form (indexkey OP pseudoconst) or (pseudoconst OP indexkey),
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* but the OP proved not to be one of the index's opfamily operators.
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
* Return 'true' if we can do something with it anyway.
*/
static bool
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
match_special_index_operator(Expr *clause, Oid opfamily, Oid idxcollation,
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
bool indexkey_on_left)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
bool isIndexable = false;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Node *rightop;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Oid expr_op;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
Oid expr_coll;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Const *patt;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *prefix = NULL;
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
Pattern_Prefix_Status pstatus = Pattern_Prefix_None;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
* Currently, all known special operators require the indexkey on the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* left, but this test could be pushed into the switch statement if some
* are added that do not...
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
*/
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
if (!indexkey_on_left)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
return false;
/* we know these will succeed */
rightop = get_rightop(clause);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr_op = ((OpExpr *) clause)->opno;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
expr_coll = ((OpExpr *) clause)->inputcollid;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
/* again, required for all current special ops: */
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
if (!IsA(rightop, Const) ||
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
((Const *) rightop)->constisnull)
return false;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
patt = (Const *) rightop;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
switch (expr_op)
{
case OID_TEXT_LIKE_OP:
case OID_BPCHAR_LIKE_OP:
case OID_NAME_LIKE_OP:
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Like, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
isIndexable = (pstatus != Pattern_Prefix_None);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
break;
case OID_BYTEA_LIKE_OP:
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Like, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
isIndexable = (pstatus != Pattern_Prefix_None);
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
break;
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
case OID_TEXT_ICLIKE_OP:
case OID_BPCHAR_ICLIKE_OP:
case OID_NAME_ICLIKE_OP:
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Like_IC, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
isIndexable = (pstatus != Pattern_Prefix_None);
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
break;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
case OID_TEXT_REGEXEQ_OP:
case OID_BPCHAR_REGEXEQ_OP:
case OID_NAME_REGEXEQ_OP:
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Regex, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
isIndexable = (pstatus != Pattern_Prefix_None);
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
break;
case OID_TEXT_ICREGEXEQ_OP:
case OID_BPCHAR_ICREGEXEQ_OP:
case OID_NAME_ICREGEXEQ_OP:
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Regex_IC, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
isIndexable = (pstatus != Pattern_Prefix_None);
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
break;
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
case OID_INET_SUB_OP:
case OID_INET_SUBEQ_OP:
isIndexable = true;
break;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (prefix)
{
pfree(DatumGetPointer(prefix->constvalue));
pfree(prefix);
}
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
/* done if the expression doesn't look indexable */
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
if (!isIndexable)
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
return false;
/*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* Must also check that index's opfamily supports the operators we will
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
* want to apply. (A hash index, for example, will not support ">=".)
Allow spgist's text_ops to handle pattern-matching operators. This was presumably intended to work this way all along, but a few key bits of indxpath.c didn't get the memo. Robert Haas and Tom Lane
2012-02-02 19:10:56 +01:00
* Currently, only btree and spgist support the operators we need.
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Note: actually, in the Pattern_Prefix_Exact case, we only need "=" so a
* hash index would work. Currently it doesn't seem worth checking for
* that, however.
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* We insist on the opfamily being the specific one we expect, else we'd
* do the wrong thing if someone were to make a reverse-sort opfamily with
* the same operators.
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
*
* The non-pattern opclasses will not sort the way we need in most non-C
* locales. We can use such an index anyway for an exact match (simple
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
* equality), but not for prefix-match cases. Note that here we are
* looking at the index's collation, not the expression's collation --
* this test is *not* dependent on the LIKE/regex operator's collation.
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
*/
switch (expr_op)
{
case OID_TEXT_LIKE_OP:
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
case OID_TEXT_ICLIKE_OP:
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
case OID_TEXT_REGEXEQ_OP:
case OID_TEXT_ICREGEXEQ_OP:
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
isIndexable =
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
(opfamily == TEXT_PATTERN_BTREE_FAM_OID) ||
Allow spgist's text_ops to handle pattern-matching operators. This was presumably intended to work this way all along, but a few key bits of indxpath.c didn't get the memo. Robert Haas and Tom Lane
2012-02-02 19:10:56 +01:00
(opfamily == TEXT_SPGIST_FAM_OID) ||
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
(opfamily == TEXT_BTREE_FAM_OID &&
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
(pstatus == Pattern_Prefix_Exact ||
lc_collate_is_c(idxcollation)));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
break;
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
case OID_BPCHAR_LIKE_OP:
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
case OID_BPCHAR_ICLIKE_OP:
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
case OID_BPCHAR_REGEXEQ_OP:
case OID_BPCHAR_ICREGEXEQ_OP:
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
isIndexable =
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
(opfamily == BPCHAR_PATTERN_BTREE_FAM_OID) ||
Improve match_special_index_operator() to recognize that LIKE with an exact-match pattern (no wildcard) can be index-optimized in some cases where a prefix-match pattern cannot; specifically, since the required index clause is simple equality, it works for regular text/varchar indexes even when the locale is not C. I'm not sure how often this case really comes up, but since it requires hardly any additional work to handle it, we might as well get it right. Motivated by a discussion on the JDBC list.
2009-03-11 04:32:22 +01:00
(opfamily == BPCHAR_BTREE_FAM_OID &&
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
(pstatus == Pattern_Prefix_Exact ||
lc_collate_is_c(idxcollation)));
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
break;
case OID_NAME_LIKE_OP:
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
case OID_NAME_ICLIKE_OP:
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
case OID_NAME_REGEXEQ_OP:
case OID_NAME_ICREGEXEQ_OP:
Alter the xxx_pattern_ops opclasses to use the regular equality operator of the associated datatype as their equality member. This means that these opclasses can now support plain equality comparisons along with LIKE tests, thus avoiding the need for an extra index in some applications. This optimization was not possible when the pattern opclasses were first introduced, because we didn't insist that text equality meant bitwise equality; but we do now, so there is no semantic difference between regular and pattern equality operators. I removed the name_pattern_ops opclass altogether, since it's really useless: name's regular comparisons are just strcmp() and are unlikely to become something different. Instead teach indxpath.c that btree name_ops can be used for LIKE whether or not the locale is C. This might lead to a useful speedup in LIKE queries on the system catalogs in non-C locales. The ~=~ and ~<>~ operators are gone altogether. (It would have been nice to keep them for backward compatibility's sake, but since the pg_amop structure doesn't allow multiple equality operators per opclass, there's no way.) A not-immediately-obvious incompatibility is that the sort order within bpchar_pattern_ops indexes changes --- it had been identical to plain strcmp, but is now trailing-blank-insensitive. This will impact in-place upgrades, if those ever happen. Per discussions a couple months ago.
2008-05-27 02:13:09 +02:00
/* name uses locale-insensitive sorting */
isIndexable = (opfamily == NAME_BTREE_FAM_OID);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case OID_BYTEA_LIKE_OP:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
isIndexable = (opfamily == BYTEA_BTREE_FAM_OID);
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
break;
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
case OID_INET_SUB_OP:
case OID_INET_SUBEQ_OP:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
isIndexable = (opfamily == NETWORK_BTREE_FAM_OID);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
break;
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
return isIndexable;
}
/*
* expand_indexqual_conditions
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* Given a list of RestrictInfo nodes, produce a list of directly usable
* index qual clauses.
*
* Standard qual clauses (those in the index's opfamily) are passed through
* unchanged. Boolean clauses and "special" index operators are expanded
* into clauses that the indexscan machinery will know what to do with.
* RowCompare clauses are simplified if necessary to create a clause that is
* fully checkable by the index.
*
* In addition to the expressions themselves, there are auxiliary lists
* of the index column numbers that the clauses are meant to be used with;
* we generate an updated column number list for the result. (This is not
* the identical list because one input clause sometimes produces more than
* one output clause.)
*
* The input clauses are sorted by column number, and so the output is too.
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
* (This is depended on in various places in both planner and executor.)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
*/
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
void
expand_indexqual_conditions(IndexOptInfo *index,
List *indexclauses, List *indexclausecols,
List **indexquals_p, List **indexqualcols_p)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
List *indexquals = NIL;
List *indexqualcols = NIL;
ListCell *lcc,
*lci;
Further cleanup of indxpath logic related to IndexOptInfo.opfamily array. We no longer need the terminating zero entry in opfamily[], so get rid of it. Also replace assorted ad-hoc looping logic with simple for and foreach constructs. This code is now noticeably more readable than it was an hour ago; credit to Robert for seeing that it could be simplified.
2010-11-20 21:07:16 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
forboth(lcc, indexclauses, lci, indexclausecols)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lcc);
int indexcol = lfirst_int(lci);
Expr *clause = rinfo->clause;
Minor cleanup of indxpath.c. Eliminate some superfluous notational complexity around match_clause_to_indexcol(), and rip out the DoneMatchingIndexKeys crock.
2010-11-20 19:57:27 +01:00
Oid curFamily = index->opfamily[indexcol];
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
Oid curCollation = index->indexcollations[indexcol];
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
/* First check for boolean cases */
if (IsBooleanOpfamily(curFamily))
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
Expr *boolqual;
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
boolqual = expand_boolean_index_clause((Node *) clause,
indexcol,
index);
if (boolqual)
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
indexquals = lappend(indexquals,
make_simple_restrictinfo(boolqual));
indexqualcols = lappend_int(indexqualcols, indexcol);
continue;
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
/*
* Else it must be an opclause (usual case), ScalarArrayOp,
* RowCompare, or NullTest
*/
if (is_opclause(clause))
{
indexquals = list_concat(indexquals,
expand_indexqual_opclause(rinfo,
curFamily,
curCollation));
/* expand_indexqual_opclause can produce multiple clauses */
while (list_length(indexqualcols) < list_length(indexquals))
indexqualcols = lappend_int(indexqualcols, indexcol);
}
else if (IsA(clause, ScalarArrayOpExpr))
{
/* no extra work at this time */
indexquals = lappend(indexquals, rinfo);
indexqualcols = lappend_int(indexqualcols, indexcol);
}
else if (IsA(clause, RowCompareExpr))
{
indexquals = lappend(indexquals,
expand_indexqual_rowcompare(rinfo,
index,
indexcol));
indexqualcols = lappend_int(indexqualcols, indexcol);
}
else if (IsA(clause, NullTest))
{
Assert(index->amsearchnulls);
indexquals = lappend(indexquals, rinfo);
indexqualcols = lappend_int(indexqualcols, indexcol);
}
else
elog(ERROR, "unsupported indexqual type: %d",
(int) nodeTag(clause));
Further cleanup of indxpath logic related to IndexOptInfo.opfamily array. We no longer need the terminating zero entry in opfamily[], so get rid of it. Also replace assorted ad-hoc looping logic with simple for and foreach constructs. This code is now noticeably more readable than it was an hour ago; credit to Robert for seeing that it could be simplified.
2010-11-20 21:07:16 +01:00
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
*indexquals_p = indexquals;
*indexqualcols_p = indexqualcols;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
}
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/*
* expand_boolean_index_clause
* Convert a clause recognized by match_boolean_index_clause into
* a boolean equality operator clause.
*
* Returns NULL if the clause isn't a boolean index qual.
*/
static Expr *
expand_boolean_index_clause(Node *clause,
int indexcol,
IndexOptInfo *index)
{
/* Direct match? */
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
if (match_index_to_operand(clause, indexcol, index))
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
{
/* convert to indexkey = TRUE */
return make_opclause(BooleanEqualOperator, BOOLOID, false,
(Expr *) clause,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) makeBoolConst(true, false),
InvalidOid, InvalidOid);
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
}
/* NOT clause? */
if (not_clause(clause))
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Node *arg = (Node *) get_notclausearg((Expr *) clause);
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/* It must have matched the indexkey */
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
Assert(match_index_to_operand(arg, indexcol, index));
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/* convert to indexkey = FALSE */
return make_opclause(BooleanEqualOperator, BOOLOID, false,
(Expr *) arg,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) makeBoolConst(false, false),
InvalidOid, InvalidOid);
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
}
if (clause && IsA(clause, BooleanTest))
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
BooleanTest *btest = (BooleanTest *) clause;
Node *arg = (Node *) btest->arg;
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
/* It must have matched the indexkey */
Add a back-link from IndexOptInfo structs to their parent RelOptInfo structs. There are many places in the planner where we were passing both a rel and an index to subroutines, and now need only pass the index struct. Notationally simpler, and perhaps a tad faster.
2005-03-27 08:29:49 +02:00
Assert(match_index_to_operand(arg, indexcol, index));
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
if (btest->booltesttype == IS_TRUE)
{
/* convert to indexkey = TRUE */
return make_opclause(BooleanEqualOperator, BOOLOID, false,
(Expr *) arg,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) makeBoolConst(true, false),
InvalidOid, InvalidOid);
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
}
if (btest->booltesttype == IS_FALSE)
{
/* convert to indexkey = FALSE */
return make_opclause(BooleanEqualOperator, BOOLOID, false,
(Expr *) arg,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) makeBoolConst(false, false),
InvalidOid, InvalidOid);
Expand the 'special index operator' machinery to handle special cases for boolean indexes. Previously we would only use such an index with WHERE clauses like 'indexkey = true' or 'indexkey = false'. The new code transforms the cases 'indexkey', 'NOT indexkey', 'indexkey IS TRUE', and 'indexkey IS FALSE' into one of these. While this is only marginally useful in itself, I intend soon to change constant-expression simplification so that 'foo = true' and 'foo = false' are reduced to just 'foo' and 'NOT foo' ... which would lose the ability to use boolean indexes for such queries at all, if the indexscan machinery couldn't make the reverse transformation.
2005-03-27 00:29:20 +01:00
}
/* Oops */
Assert(false);
}
return NULL;
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
/*
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* expand_indexqual_opclause --- expand a single indexqual condition
* that is an operator clause
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
*
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
* The input is a single RestrictInfo, the output a list of RestrictInfos.
*
* In the base case this is just list_make1(), but we have to be prepared to
* expand special cases that were accepted by match_special_index_operator().
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*/
static List *
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
expand_indexqual_opclause(RestrictInfo *rinfo, Oid opfamily, Oid idxcollation)
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
{
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
Expr *clause = rinfo->clause;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
/* we know these will succeed */
Node *leftop = get_leftop(clause);
Node *rightop = get_rightop(clause);
Oid expr_op = ((OpExpr *) clause)->opno;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
Oid expr_coll = ((OpExpr *) clause)->inputcollid;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Const *patt = (Const *) rightop;
Const *prefix = NULL;
Pattern_Prefix_Status pstatus;
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
/*
* LIKE and regex operators are not members of any btree index opfamily,
* but they can be members of opfamilies for more exotic index types such
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* as GIN. Therefore, we should only do expansion if the operator is
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
* actually not in the opfamily. But checking that requires a syscache
* lookup, so it's best to first see if the operator is one we are
* interested in.
*/
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
switch (expr_op)
{
case OID_TEXT_LIKE_OP:
case OID_BPCHAR_LIKE_OP:
case OID_NAME_LIKE_OP:
case OID_BYTEA_LIKE_OP:
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
if (!op_in_opfamily(expr_op, opfamily))
{
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Like, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return prefix_quals(leftop, opfamily, idxcollation, prefix, pstatus);
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case OID_TEXT_ICLIKE_OP:
case OID_BPCHAR_ICLIKE_OP:
case OID_NAME_ICLIKE_OP:
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
if (!op_in_opfamily(expr_op, opfamily))
{
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Like_IC, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return prefix_quals(leftop, opfamily, idxcollation, prefix, pstatus);
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case OID_TEXT_REGEXEQ_OP:
case OID_BPCHAR_REGEXEQ_OP:
case OID_NAME_REGEXEQ_OP:
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
if (!op_in_opfamily(expr_op, opfamily))
{
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Regex, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return prefix_quals(leftop, opfamily, idxcollation, prefix, pstatus);
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case OID_TEXT_ICREGEXEQ_OP:
case OID_BPCHAR_ICREGEXEQ_OP:
case OID_NAME_ICREGEXEQ_OP:
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
if (!op_in_opfamily(expr_op, opfamily))
{
/* the right-hand const is type text for all of these */
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, Pattern_Type_Regex_IC, expr_coll,
Refactor pattern_fixed_prefix() to avoid dealing in incomplete patterns. Previously, pattern_fixed_prefix() was defined to return whatever fixed prefix it could extract from the pattern, plus the "rest" of the pattern. That definition was sensible for LIKE patterns, but not so much for regexes, where reconstituting a valid pattern minus the prefix could be quite tricky (certainly the existing code wasn't doing that correctly). Since the only thing that callers ever did with the "rest" of the pattern was to pass it to like_selectivity() or regex_selectivity(), let's cut out the middle-man and just have pattern_fixed_prefix's subroutines do this directly. Then pattern_fixed_prefix can return a simple selectivity number, and the question of how to cope with partial patterns is removed from its API specification. While at it, adjust the API spec so that callers who don't actually care about the pattern's selectivity (which is a lot of them) can pass NULL for the selectivity pointer to skip doing the work of computing a selectivity estimate. This patch is only an API refactoring that doesn't actually change any processing, other than allowing a little bit of useless work to be skipped. However, it's necessary infrastructure for my upcoming fix to regex prefix extraction, because after that change there won't be any simple way to identify the "rest" of the regex, not even to the low level of fidelity needed by regex_selectivity. We can cope with that if regex_fixed_prefix and regex_selectivity communicate directly, but not if we have to work within the old API. Hence, back-patch to all active branches.
2012-07-10 05:22:55 +02:00
&prefix, NULL);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
return prefix_quals(leftop, opfamily, idxcollation, prefix, pstatus);
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case OID_INET_SUB_OP:
case OID_INET_SUBEQ_OP:
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
if (!op_in_opfamily(expr_op, opfamily))
{
return network_prefix_quals(leftop, expr_op, opfamily,
patt->constvalue);
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
}
Extend GIN to support partial-match searches, and extend tsquery to support prefix matching using this facility. Teodor Sigaev and Oleg Bartunov
2008-05-16 18:31:02 +02:00
/* Default case: just make a list of the unmodified indexqual */
return list_make1(rinfo);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/*
* expand_indexqual_rowcompare --- expand a single indexqual condition
* that is a RowCompareExpr
*
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
* This is a thin wrapper around adjust_rowcompare_for_index; we export the
* latter so that createplan.c can use it to re-discover which columns of the
* index are used by a row comparison indexqual.
*/
static RestrictInfo *
expand_indexqual_rowcompare(RestrictInfo *rinfo,
IndexOptInfo *index,
int indexcol)
{
RowCompareExpr *clause = (RowCompareExpr *) rinfo->clause;
Expr *newclause;
List *indexcolnos;
bool var_on_left;
newclause = adjust_rowcompare_for_index(clause,
index,
indexcol,
&indexcolnos,
&var_on_left);
/*
* If we didn't have to change the RowCompareExpr, return the original
* RestrictInfo.
*/
if (newclause == (Expr *) clause)
return rinfo;
/* Else we need a new RestrictInfo */
return make_simple_restrictinfo(newclause);
}
/*
* adjust_rowcompare_for_index --- expand a single indexqual condition
* that is a RowCompareExpr
*
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* It's already known that the first column of the row comparison matches
* the specified column of the index. We can use additional columns of the
* row comparison as index qualifications, so long as they match the index
* in the "same direction", ie, the indexkeys are all on the same side of the
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* clause and the operators are all the same-type members of the opfamilies.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* If all the columns of the RowCompareExpr match in this way, we just use it
* as-is. Otherwise, we build a shortened RowCompareExpr (if more than one
* column matches) or a simple OpExpr (if the first-column match is all
* there is). In these cases the modified clause is always "<=" or ">="
* even when the original was "<" or ">" --- this is necessary to match all
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* the rows that could match the original. (We are essentially building a
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* lossy version of the row comparison when we do this.)
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
*
* *indexcolnos receives an integer list of the index column numbers (zero
* based) used in the resulting expression. The reason we need to return
* that is that if the index is selected for use, createplan.c will need to
* call this again to extract that list. (This is a bit grotty, but row
* comparison indexquals aren't used enough to justify finding someplace to
* keep the information in the Path representation.) Since createplan.c
* also needs to know which side of the RowCompareExpr is the index side,
* we also return *var_on_left_p rather than re-deducing that there.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*/
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
Expr *
adjust_rowcompare_for_index(RowCompareExpr *clause,
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
IndexOptInfo *index,
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
int indexcol,
List **indexcolnos,
bool *var_on_left_p)
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
{
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
bool var_on_left;
int op_strategy;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid op_lefttype;
Oid op_righttype;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
int matching_cols;
Oid expr_op;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
List *opfamilies;
List *lefttypes;
List *righttypes;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
List *new_ops;
ListCell *largs_cell;
ListCell *rargs_cell;
ListCell *opnos_cell;
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
ListCell *collids_cell;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/* We have to figure out (again) how the first col matches */
var_on_left = match_index_to_operand((Node *) linitial(clause->largs),
indexcol, index);
Assert(var_on_left ||
match_index_to_operand((Node *) linitial(clause->rargs),
indexcol, index));
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
*var_on_left_p = var_on_left;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
expr_op = linitial_oid(clause->opnos);
if (!var_on_left)
expr_op = get_commutator(expr_op);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
get_op_opfamily_properties(expr_op, index->opfamily[indexcol], false,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
&op_strategy,
&op_lefttype,
Since createplan.c no longer cares whether index operators are lossy, it has no particular need to do get_op_opfamily_properties() while building an indexscan plan. Postpone that lookup until executor start. This simplifies createplan.c a lot more than it complicates nodeIndexscan.c, and makes things more uniform since we already had to do it that way for RowCompare expressions. Should be a bit faster too, at least for plans that aren't re-used many times, since we avoid palloc'ing and perhaps copying the intermediate list data structure.
2008-04-13 22:51:21 +02:00
&op_righttype);
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
/* Initialize returned list of which index columns are used */
*indexcolnos = list_make1_int(indexcol);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
/* Build lists of the opfamilies and operator datatypes in case needed */
opfamilies = list_make1_oid(index->opfamily[indexcol]);
lefttypes = list_make1_oid(op_lefttype);
righttypes = list_make1_oid(op_righttype);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* See how many of the remaining columns match some index column in the
Run pgindent on 9.2 source tree in preparation for first 9.3 commit-fest.
2012-06-10 21:20:04 +02:00
* same way. As in match_clause_to_indexcol(), the "other" side of any
* potential index condition is OK as long as it doesn't use Vars from the
* indexed relation.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*/
matching_cols = 1;
largs_cell = lnext(list_head(clause->largs));
rargs_cell = lnext(list_head(clause->rargs));
opnos_cell = lnext(list_head(clause->opnos));
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
collids_cell = lnext(list_head(clause->inputcollids));
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
while (largs_cell != NULL)
{
Node *varop;
Node *constop;
int i;
expr_op = lfirst_oid(opnos_cell);
if (var_on_left)
{
varop = (Node *) lfirst(largs_cell);
constop = (Node *) lfirst(rargs_cell);
}
else
{
varop = (Node *) lfirst(rargs_cell);
constop = (Node *) lfirst(largs_cell);
/* indexkey is on right, so commute the operator */
expr_op = get_commutator(expr_op);
if (expr_op == InvalidOid)
break; /* operator is not usable */
}
if (bms_is_member(index->rel->relid, pull_varnos(constop)))
break; /* no good, Var on wrong side */
if (contain_volatile_functions(constop))
break; /* no good, volatile comparison value */
/*
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
* The Var side can match any column of the index.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*/
for (i = 0; i < index->ncolumns; i++)
{
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
if (match_index_to_operand(varop, i, index) &&
get_op_opfamily_strategy(expr_op,
index->opfamily[i]) == op_strategy &&
IndexCollMatchesExprColl(index->indexcollations[i],
lfirst_oid(collids_cell)))
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
break;
}
if (i >= index->ncolumns)
break; /* no match found */
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
/* Add column number to returned list */
*indexcolnos = lappend_int(*indexcolnos, i);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
/* Add opfamily and datatypes to lists */
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
get_op_opfamily_properties(expr_op, index->opfamily[i], false,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
&op_strategy,
&op_lefttype,
Since createplan.c no longer cares whether index operators are lossy, it has no particular need to do get_op_opfamily_properties() while building an indexscan plan. Postpone that lookup until executor start. This simplifies createplan.c a lot more than it complicates nodeIndexscan.c, and makes things more uniform since we already had to do it that way for RowCompare expressions. Should be a bit faster too, at least for plans that aren't re-used many times, since we avoid palloc'ing and perhaps copying the intermediate list data structure.
2008-04-13 22:51:21 +02:00
&op_righttype);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamilies = lappend_oid(opfamilies, index->opfamily[i]);
lefttypes = lappend_oid(lefttypes, op_lefttype);
righttypes = lappend_oid(righttypes, op_righttype);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/* This column matches, keep scanning */
matching_cols++;
largs_cell = lnext(largs_cell);
rargs_cell = lnext(rargs_cell);
opnos_cell = lnext(opnos_cell);
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
collids_cell = lnext(collids_cell);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
/* Return clause as-is if it's all usable as index quals */
if (matching_cols == list_length(clause->opnos))
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
return (Expr *) clause;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* We have to generate a subset rowcompare (possibly just one OpExpr). The
* painful part of this is changing < to <= or > to >=, so deal with that
* first.
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
*/
if (op_strategy == BTLessEqualStrategyNumber ||
op_strategy == BTGreaterEqualStrategyNumber)
{
/* easy, just use the same operators */
new_ops = list_truncate(list_copy(clause->opnos), matching_cols);
}
else
{
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
ListCell *opfamilies_cell;
ListCell *lefttypes_cell;
ListCell *righttypes_cell;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
if (op_strategy == BTLessStrategyNumber)
op_strategy = BTLessEqualStrategyNumber;
else if (op_strategy == BTGreaterStrategyNumber)
op_strategy = BTGreaterEqualStrategyNumber;
else
elog(ERROR, "unexpected strategy number %d", op_strategy);
new_ops = NIL;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
lefttypes_cell = list_head(lefttypes);
righttypes_cell = list_head(righttypes);
foreach(opfamilies_cell, opfamilies)
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Oid opfam = lfirst_oid(opfamilies_cell);
Oid lefttype = lfirst_oid(lefttypes_cell);
Oid righttype = lfirst_oid(righttypes_cell);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
expr_op = get_opfamily_member(opfam, lefttype, righttype,
op_strategy);
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
if (!OidIsValid(expr_op)) /* should not happen */
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "could not find member %d(%u,%u) of opfamily %u",
op_strategy, lefttype, righttype, opfam);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
if (!var_on_left)
{
expr_op = get_commutator(expr_op);
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
if (!OidIsValid(expr_op)) /* should not happen */
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "could not find commutator of member %d(%u,%u) of opfamily %u",
op_strategy, lefttype, righttype, opfam);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
new_ops = lappend_oid(new_ops, expr_op);
Fix silly mistake in expand_indexqual_rowcompare --- in converting a forboth() into an iteration over three parallel lists, I had accidentally put the lnext steps outside the loop. Sigh. Per bug #3938.
2008-02-07 18:53:53 +01:00
lefttypes_cell = lnext(lefttypes_cell);
righttypes_cell = lnext(righttypes_cell);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
}
/* If we have more than one matching col, create a subset rowcompare */
if (matching_cols > 1)
{
RowCompareExpr *rc = makeNode(RowCompareExpr);
if (var_on_left)
rc->rctype = (RowCompareType) op_strategy;
else
rc->rctype = (op_strategy == BTLessEqualStrategyNumber) ?
ROWCOMPARE_GE : ROWCOMPARE_LE;
rc->opnos = new_ops;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
rc->opfamilies = list_truncate(list_copy(clause->opfamilies),
matching_cols);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
rc->inputcollids = list_truncate(list_copy(clause->inputcollids),
matching_cols);
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
rc->largs = list_truncate((List *) copyObject(clause->largs),
matching_cols);
rc->rargs = list_truncate((List *) copyObject(clause->rargs),
matching_cols);
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
return (Expr *) rc;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
else
{
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
return make_opclause(linitial_oid(new_ops), BOOLOID, false,
copyObject(linitial(clause->largs)),
copyObject(linitial(clause->rargs)),
InvalidOid,
linitial_oid(clause->inputcollids));
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
}
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
/*
* Given a fixed prefix that all the "leftop" values must have,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* generate suitable indexqual condition(s). opfamily is the index
* operator family; we use it to deduce the appropriate comparison
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
* operators and operand datatypes. collation is the input collation to use.
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
*/
static List *
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
prefix_quals(Node *leftop, Oid opfamily, Oid collation,
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *prefix_const, Pattern_Prefix_Status pstatus)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
List *result;
Oid datatype;
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
Oid oproid;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Expr *expr;
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
FmgrInfo ltproc;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Const *greaterstr;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Assert(pstatus != Pattern_Prefix_None);
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
switch (opfamily)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
case TEXT_BTREE_FAM_OID:
case TEXT_PATTERN_BTREE_FAM_OID:
Allow spgist's text_ops to handle pattern-matching operators. This was presumably intended to work this way all along, but a few key bits of indxpath.c didn't get the memo. Robert Haas and Tom Lane
2012-02-02 19:10:56 +01:00
case TEXT_SPGIST_FAM_OID:
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
datatype = TEXTOID;
break;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
case BPCHAR_BTREE_FAM_OID:
case BPCHAR_PATTERN_BTREE_FAM_OID:
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
datatype = BPCHAROID;
break;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
case NAME_BTREE_FAM_OID:
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
datatype = NAMEOID;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
break;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
case BYTEA_BTREE_FAM_OID:
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
datatype = BYTEAOID;
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
break;
default:
Error message editing in backend/optimizer, backend/rewrite.
2003-07-25 02:01:09 +02:00
/* shouldn't get here */
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "unexpected opfamily: %u", opfamily);
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
return NIL;
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If necessary, coerce the prefix constant to the right type. The given
* prefix constant is either text or bytea type.
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*/
if (prefix_const->consttype != datatype)
{
pgindent run.
2003-08-04 02:43:34 +02:00
char *prefix;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
switch (prefix_const->consttype)
{
case TEXTOID:
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
prefix = TextDatumGetCString(prefix_const->constvalue);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case BYTEAOID:
prefix = DatumGetCString(DirectFunctionCall1(byteaout,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
prefix_const->constvalue));
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
default:
Error message editing in backend/optimizer, backend/rewrite.
2003-07-25 02:01:09 +02:00
elog(ERROR, "unexpected const type: %u",
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
prefix_const->consttype);
return NIL;
}
prefix_const = string_to_const(prefix, datatype);
pfree(prefix);
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
/*
* If we found an exact-match pattern, generate an "=" indexqual.
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (pstatus == Pattern_Prefix_Exact)
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
{
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
oproid = get_opfamily_member(opfamily, datatype, datatype,
BTEqualStrategyNumber);
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
if (oproid == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no = operator for opfamily %u", opfamily);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr = make_opclause(oproid, BOOLOID, false,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) leftop, (Expr *) prefix_const,
InvalidOid, collation);
Fix planner to restore its previous level of intelligence about pushing constants through full joins, as in select * from tenk1 a full join tenk1 b using (unique1) where unique1 = 42; which should generate a fairly cheap plan where we apply the constraint unique1 = 42 in each relation scan. This had been broken by my patch of 2008-06-27, which is now reverted in favor of a more invasive but hopefully less incorrect approach. That patch was meant to prevent incorrect extraction of OR'd indexclauses from OR conditions above an outer join. To do that correctly we need more information than the outerjoin_delay flag can provide, so add a nullable_relids field to RestrictInfo that records exactly which relations are nulled by outer joins that are underneath a particular qual clause. A side benefit is that we can make the test in create_or_index_quals more specific: it is now smart enough to extract an OR'd indexclause into the outer side of an outer join, even though it must not do so in the inner side. The old coding couldn't distinguish these cases so it could not do either.
2009-04-16 22:42:16 +02:00
result = list_make1(make_simple_restrictinfo(expr));
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
return result;
}
/*
* Otherwise, we have a nonempty required prefix of the values.
*
* We can always say "x >= prefix".
*/
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
oproid = get_opfamily_member(opfamily, datatype, datatype,
BTGreaterEqualStrategyNumber);
Clean up loose end in LIKE optimization fix: parser's code would generate <= and >= indexquals from a LIKE even if the index in question didn't support those operators. (As, for example, a hash index does not.)
1999-12-31 04:41:03 +01:00
if (oproid == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no >= operator for opfamily %u", opfamily);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr = make_opclause(oproid, BOOLOID, false,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) leftop, (Expr *) prefix_const,
InvalidOid, collation);
Fix planner to restore its previous level of intelligence about pushing constants through full joins, as in select * from tenk1 a full join tenk1 b using (unique1) where unique1 = 42; which should generate a fairly cheap plan where we apply the constraint unique1 = 42 in each relation scan. This had been broken by my patch of 2008-06-27, which is now reverted in favor of a more invasive but hopefully less incorrect approach. That patch was meant to prevent incorrect extraction of OR'd indexclauses from OR conditions above an outer join. To do that correctly we need more information than the outerjoin_delay flag can provide, so add a nullable_relids field to RestrictInfo that records exactly which relations are nulled by outer joins that are underneath a particular qual clause. A side benefit is that we can make the test in create_or_index_quals more specific: it is now smart enough to extract an OR'd indexclause into the outer side of an outer join, even though it must not do so in the inner side. The old coding couldn't distinguish these cases so it could not do either.
2009-04-16 22:42:16 +02:00
result = list_make1(make_simple_restrictinfo(expr));
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Fix comments that were mis-wrapped, for Tom Lane.
2001-03-23 05:49:58 +01:00
/*-------
* If we can create a string larger than the prefix, we can say
Clean up minor collation issues in indxpath.c. Get rid of bogus collation test in match_special_index_operator (even for ILIKE, the pattern match operator's collation doesn't matter here, and even if it did the test was testing the wrong thing). Fix broken looping logic in expand_indexqual_rowcompare. Add collation check in match_clause_to_ordering_op. Make naming and argument ordering more consistent; improve comments.
2011-04-09 01:19:17 +02:00
* "x < greaterstr". NB: we rely on make_greater_string() to generate
* a guaranteed-greater string, not just a probably-greater string.
* In general this is only guaranteed in C locale, so we'd better be
* using a C-locale index collation.
Fix comments that were mis-wrapped, for Tom Lane.
2001-03-23 05:49:58 +01:00
*-------
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
*/
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
oproid = get_opfamily_member(opfamily, datatype, datatype,
BTLessStrategyNumber);
if (oproid == InvalidOid)
elog(ERROR, "no < operator for opfamily %u", opfamily);
fmgr_info(get_opcode(oproid), &ltproc);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
greaterstr = make_greater_string(prefix_const, &ltproc, collation);
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
if (greaterstr)
{
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr = make_opclause(oproid, BOOLOID, false,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
(Expr *) leftop, (Expr *) greaterstr,
InvalidOid, collation);
Fix planner to restore its previous level of intelligence about pushing constants through full joins, as in select * from tenk1 a full join tenk1 b using (unique1) where unique1 = 42; which should generate a fairly cheap plan where we apply the constraint unique1 = 42 in each relation scan. This had been broken by my patch of 2008-06-27, which is now reverted in favor of a more invasive but hopefully less incorrect approach. That patch was meant to prevent incorrect extraction of OR'd indexclauses from OR conditions above an outer join. To do that correctly we need more information than the outerjoin_delay flag can provide, so add a nullable_relids field to RestrictInfo that records exactly which relations are nulled by outer joins that are underneath a particular qual clause. A side benefit is that we can make the test in create_or_index_quals more specific: it is now smart enough to extract an OR'd indexclause into the outer side of an outer join, even though it must not do so in the inner side. The old coding couldn't distinguish these cases so it could not do either.
2009-04-16 22:42:16 +02:00
result = lappend(result, make_simple_restrictinfo(expr));
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
}
First cut at doing LIKE/regex indexing optimization in optimizer rather than parser. This has many advantages, such as not getting fooled by chance uses of operator names ~ and ~~ (the operators are identified by OID now), and not creating useless comparison operations in contexts where the comparisons will not actually be used as indexquals. The new code also recognizes exact-match LIKE and regex patterns, and produces an = indexqual instead of >= and <=. This change does NOT fix the problem with non-ASCII locales: the code still doesn't know how to generate an upper bound indexqual for non-ASCII collation order. But it's no worse than before, just the same deficiency in a different place... Also, dike out loc_restrictinfo fields in Plan nodes. These were doing nothing useful in the absence of 'expensive functions' optimization, and they took a considerable amount of processing to fill in.
1999-07-27 05:51:11 +02:00
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
return result;
}
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
/*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* Given a leftop and a rightop, and a inet-family sup/sub operator,
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
* generate suitable indexqual condition(s). expr_op is the original
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* operator, and opfamily is the index opfamily.
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
*/
static List *
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
network_prefix_quals(Node *leftop, Oid expr_op, Oid opfamily, Datum rightop)
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
{
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
bool is_eq;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Oid datatype;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Oid opr1oid;
Oid opr2oid;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
Datum opr1right;
Datum opr2right;
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
List *result;
Expr *expr;
switch (expr_op)
{
case OID_INET_SUB_OP:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
datatype = INETOID;
is_eq = false;
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
break;
case OID_INET_SUBEQ_OP:
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
datatype = INETOID;
is_eq = true;
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
break;
default:
Error message editing in backend/optimizer, backend/rewrite.
2003-07-25 02:01:09 +02:00
elog(ERROR, "unexpected operator: %u", expr_op);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
return NIL;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
}
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* create clause "key >= network_scan_first( rightop )", or ">" if the
* operator disallows equality.
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
*/
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
if (is_eq)
{
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opr1oid = get_opfamily_member(opfamily, datatype, datatype,
BTGreaterEqualStrategyNumber);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
if (opr1oid == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no >= operator for opfamily %u", opfamily);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
else
{
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opr1oid = get_opfamily_member(opfamily, datatype, datatype,
BTGreaterStrategyNumber);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
if (opr1oid == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no > operator for opfamily %u", opfamily);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
opr1right = network_scan_first(rightop);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr = make_opclause(opr1oid, BOOLOID, false,
(Expr *) leftop,
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
(Expr *) makeConst(datatype, -1,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
InvalidOid, /* not collatable */
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
-1, opr1right,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
false, false),
InvalidOid, InvalidOid);
Fix planner to restore its previous level of intelligence about pushing constants through full joins, as in select * from tenk1 a full join tenk1 b using (unique1) where unique1 = 42; which should generate a fairly cheap plan where we apply the constraint unique1 = 42 in each relation scan. This had been broken by my patch of 2008-06-27, which is now reverted in favor of a more invasive but hopefully less incorrect approach. That patch was meant to prevent incorrect extraction of OR'd indexclauses from OR conditions above an outer join. To do that correctly we need more information than the outerjoin_delay flag can provide, so add a nullable_relids field to RestrictInfo that records exactly which relations are nulled by outer joins that are underneath a particular qual clause. A side benefit is that we can make the test in create_or_index_quals more specific: it is now smart enough to extract an OR'd indexclause into the outer side of an outer join, even though it must not do so in the inner side. The old coding couldn't distinguish these cases so it could not do either.
2009-04-16 22:42:16 +02:00
result = list_make1(make_simple_restrictinfo(expr));
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
/* create clause "key <= network_scan_last( rightop )" */
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opr2oid = get_opfamily_member(opfamily, datatype, datatype,
BTLessEqualStrategyNumber);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
if (opr2oid == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no <= operator for opfamily %u", opfamily);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
opr2right = network_scan_last(rightop);
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
expr = make_opclause(opr2oid, BOOLOID, false,
(Expr *) leftop,
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
(Expr *) makeConst(datatype, -1,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
InvalidOid, /* not collatable */
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
-1, opr2right,
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
false, false),
InvalidOid, InvalidOid);
Fix planner to restore its previous level of intelligence about pushing constants through full joins, as in select * from tenk1 a full join tenk1 b using (unique1) where unique1 = 42; which should generate a fairly cheap plan where we apply the constraint unique1 = 42 in each relation scan. This had been broken by my patch of 2008-06-27, which is now reverted in favor of a more invasive but hopefully less incorrect approach. That patch was meant to prevent incorrect extraction of OR'd indexclauses from OR conditions above an outer join. To do that correctly we need more information than the outerjoin_delay flag can provide, so add a nullable_relids field to RestrictInfo that records exactly which relations are nulled by outer joins that are underneath a particular qual clause. A side benefit is that we can make the test in create_or_index_quals more specific: it is now smart enough to extract an OR'd indexclause into the outer side of an outer join, even though it must not do so in the inner side. The old coding couldn't distinguish these cases so it could not do either.
2009-04-16 22:42:16 +02:00
result = lappend(result, make_simple_restrictinfo(expr));
Make inet/cidr << and <<= operators indexable. From Alex Pilosov <alex@pilosoft.com>.
2001-06-17 04:05:20 +02:00
return result;
}
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
/*
* Handy subroutines for match_special_index_operator() and friends.
*/
/*
* Generate a Datum of the appropriate type from a C string.
* Note that all of the supported types are pass-by-ref, so the
* returned value should be pfree'd if no longer needed.
*/
static Datum
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
string_to_datum(const char *str, Oid datatype)
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
{
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
* We cheat a little by assuming that CStringGetTextDatum() will do for
* bpchar and varchar constants too...
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
*/
if (datatype == NAMEOID)
More functions updated to new fmgr style --- money, name, tid datatypes. We're reaching the mopup stage here (good thing too, this is getting tedious).
2000-08-03 18:35:08 +02:00
return DirectFunctionCall1(namein, CStringGetDatum(str));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
else if (datatype == BYTEAOID)
return DirectFunctionCall1(byteain, CStringGetDatum(str));
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
else
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
return CStringGetTextDatum(str);
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
}
/*
* Generate a Const node of the appropriate type from a C string.
*/
static Const *
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
string_to_const(const char *str, Oid datatype)
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
{
Datum conval = string_to_datum(str, datatype);
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
Oid collation;
int constlen;
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
/*
* We only need to support a few datatypes here, so hard-wire properties
* instead of incurring the expense of catalog lookups.
*/
switch (datatype)
{
case TEXTOID:
case VARCHAROID:
case BPCHAROID:
collation = DEFAULT_COLLATION_OID;
constlen = -1;
break;
case NAMEOID:
collation = InvalidOid;
constlen = NAMEDATALEN;
break;
case BYTEAOID:
collation = InvalidOid;
constlen = -1;
break;
default:
elog(ERROR, "unexpected datatype in string_to_const: %u",
datatype);
return NULL;
}
return makeConst(datatype, -1, collation, constlen,
Remove unused constisset and constiscast fields of Const nodes. Clean up code and documentation associated with Param nodes.
2002-11-25 22:29:42 +01:00
conval, false, false);
Generate double-sided LIKE indexquals that work even in weird locales, by continuing to increment the rightmost character until we get a string that is demonstrably greater than the pattern prefix.
1999-12-31 06:38:25 +01:00
}