rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/utils/adt/selfuncs.c

7171 lines
204 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* selfuncs.c
Revise handling of index-type-specific indexscan cost estimation, per pghackers discussion of 5-Jan-2000. The amopselect and amopnpages estimators are gone, and in their place is a per-AM amcostestimate procedure (linked to from pg_am, not pg_amop).
2000-01-23 00:50:30 +01:00
* Selectivity functions and index cost estimation functions for
* standard operators and index access methods.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Revise handling of index-type-specific indexscan cost estimation, per pghackers discussion of 5-Jan-2000. The amopselect and amopnpages estimators are gone, and in their place is a per-AM amcostestimate procedure (linked to from pg_am, not pg_amop).
2000-01-23 00:50:30 +01:00
* Selectivity routines are registered in the pg_operator catalog
* in the "oprrest" and "oprjoin" attributes.
*
* Index cost functions are registered in the pg_am catalog
* in the "amcostestimate" attribute.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyright notices for year 2012.
2012-01-02 00:01:58 +01:00
* Portions Copyright (c) 1996-2012, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/utils/adt/selfuncs.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*----------
* Operator selectivity estimation functions are called to estimate the
* selectivity of WHERE clauses whose top-level operator is their operator.
* We divide the problem into two cases:
* Restriction clause estimation: the clause involves vars of just
* one relation.
* Join clause estimation: the clause involves vars of multiple rels.
* Join selectivity estimation is far more difficult and usually less accurate
* than restriction estimation.
*
* When dealing with the inner scan of a nestloop join, we consider the
* join's joinclauses as restriction clauses for the inner relation, and
* treat vars of the outer relation as parameters (a/k/a constants of unknown
* values). So, restriction estimators need to be able to accept an argument
* telling which relation is to be treated as the variable.
*
* The call convention for a restriction estimator (oprrest function) is
*
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
* Selectivity oprrest (PlannerInfo *root,
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
* Oid operator,
* List *args,
* int varRelid);
*
* root: general information about the query (rtable and RelOptInfo lists
* are particularly important for the estimator).
* operator: OID of the specific operator in question.
* args: argument list from the operator clause.
* varRelid: if not zero, the relid (rtable index) of the relation to
* be treated as the variable relation. May be zero if the args list
* is known to contain vars of only one relation.
*
* This is represented at the SQL level (in pg_proc) as
*
Add a bunch of pseudo-types to replace the behavior formerly associated with OPAQUE, as per recent pghackers discussion. I still want to do some more work on the 'cstring' pseudo-type, but I'm going to commit the bulk of the changes now before the tree starts shifting under me ...
2002-08-22 02:01:51 +02:00
* float8 oprrest (internal, oid, internal, int4);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
* The result is a selectivity, that is, a fraction (0 to 1) of the rows
* of the relation that are expected to produce a TRUE result for the
* given operator.
*
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
* The call convention for a join estimator (oprjoin function) is similar
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
* except that varRelid is not needed, and instead join information is
Extend join-selectivity API (oprjoin interface) so that join type is passed to join selectivity estimators. Make use of this in eqjoinsel to derive non-bogus selectivity for IN clauses. Further tweaking of cost estimation for IN. initdb forced because of pg_proc.h changes.
2003-01-28 23:13:41 +01:00
* supplied:
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
* Selectivity oprjoin (PlannerInfo *root,
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
* Oid operator,
Extend join-selectivity API (oprjoin interface) so that join type is passed to join selectivity estimators. Make use of this in eqjoinsel to derive non-bogus selectivity for IN clauses. Further tweaking of cost estimation for IN. initdb forced because of pg_proc.h changes.
2003-01-28 23:13:41 +01:00
* List *args,
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
* JoinType jointype,
* SpecialJoinInfo *sjinfo);
*
* float8 oprjoin (internal, oid, internal, int2, internal);
*
* (Before Postgres 8.4, join estimators had only the first four of these
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* parameters. That signature is still allowed, but deprecated.) The
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
* relationship between jointype and sjinfo is explained in the comments for
* clause_selectivity() --- the short version is that jointype is usually
* best ignored in favor of examining sjinfo.
*
* Join selectivity for regular inner and outer joins is defined as the
* fraction (0 to 1) of the cross product of the relations that is expected
* to produce a TRUE result for the given operator. For both semi and anti
* joins, however, the selectivity is defined as the fraction of the left-hand
* side relation's rows that are expected to have a match (ie, at least one
* row with a TRUE result) in the right-hand side.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*----------
*/
Produce a clean compile of backend...
1996-11-03 07:54:38 +01:00
#include "postgres.h"
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
#include <ctype.h>
Tweak indexscan cost estimation: round estimated # of tuples visited up to next integer. Previously, if selectivity was small, we could compute very tiny scan cost on the basis of estimating that only 0.001 tuple would be fetched, which is silly. This naturally led to some rather silly plans...
2000-03-30 02:53:30 +02:00
#include <math.h>
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
#include "access/gin.h"
Restructure some header files a bit, in particular heapam.h, by removing some unnecessary #include lines in it. Also, move some tuple routine prototypes and macros to htup.h, which allows removal of heapam.h inclusion from some .c files. For this to work, a new header file access/sysattr.h needed to be created, initially containing attribute numbers of system columns, for pg_dump usage. While at it, make contrib ltree, intarray and hstore header files more consistent with our header style.
2008-05-12 02:00:54 +02:00
#include "access/sysattr.h"
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
#include "catalog/index.h"
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
#include "catalog/pg_collation.h"
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
#include "catalog/pg_opfamily.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include "catalog/pg_statistic.h"
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
#include "catalog/pg_type.h"
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
#include "executor/executor.h"
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
#include "mb/pg_wchar.h"
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
#include "nodes/makefuncs.h"
Move exprType(), exprTypmod(), expression_tree_walker(), and related routines into nodes/nodeFuncs, so as to reduce wanton cross-subsystem #includes inside the backend. There's probably more that should be done along this line, but this is a start anyway.
2008-08-26 00:42:34 +02:00
#include "nodes/nodeFuncs.h"
Third round of fmgr updates: eliminate calls using fmgr() and fmgr_faddr() in favor of new-style calls. Lots of cleanup of sloppy casts to use XXXGetDatum and DatumGetXXX ...
2000-05-30 06:25:00 +02:00
#include "optimizer/clauses.h"
Revise handling of index-type-specific indexscan cost estimation, per pghackers discussion of 5-Jan-2000. The amopselect and amopnpages estimators are gone, and in their place is a per-AM amcostestimate procedure (linked to from pg_am, not pg_amop).
2000-01-23 00:50:30 +01:00
#include "optimizer/cost.h"
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
#include "optimizer/pathnode.h"
Modify planner's implied-equality-deduction code so that when a set of known-equal expressions includes any constant expressions (including Params from outer queries), we actively suppress any 'var = var' clauses that are or could be deduced from the set, generating only the deducible 'var = const' clauses instead. The idea here is to push down the restrictions implied by the equality set to base relations whenever possible. Once we have applied the 'var = const' clauses, the 'var = var' clauses are redundant, and should be suppressed both to save work at execution and to avoid double-counting restrictivity.
2003-01-24 04:58:44 +01:00
#include "optimizer/paths.h"
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
#include "optimizer/plancat.h"
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
#include "optimizer/predtest.h"
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
#include "optimizer/restrictinfo.h"
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
#include "optimizer/var.h"
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
#include "parser/parse_coerce.h"
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
#include "parser/parsetree.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "utils/builtins.h"
Support hex-string input and output for type BYTEA. Both hex format and the traditional "escape" format are automatically handled on input. The output format is selected by the new GUC variable bytea_output. As committed, bytea_output defaults to HEX, which is an *incompatible change*. We will keep it this way for awhile for testing purposes, but should consider whether to switch to the more backwards-compatible default of ESCAPE before 8.5 is released. Peter Eisentraut
2009-08-04 18:08:37 +02:00
#include "utils/bytea.h"
Clean up #include's.
2000-06-15 05:33:12 +02:00
#include "utils/date.h"
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
#include "utils/datum.h"
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
#include "utils/fmgroids.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "utils/lsyscache.h"
Clean up the rather historically encumbered interface to now() and current time: provide a GetCurrentTimestamp() function that returns current time in the form of a TimestampTz, instead of separate time_t and microseconds fields. This is what all the callers really want anyway, and it eliminates low-level dependencies on AbsoluteTime, which is a deprecated datatype that will have to disappear eventually.
2005-06-30 00:51:57 +02:00
#include "utils/nabstime.h"
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
#include "utils/pg_locale.h"
Move Trigger and TriggerDesc structs out of rel.h into a new reltrigger.h This lets us stop including rel.h into execnodes.h, which is a widely used header.
2011-02-23 18:18:09 +01:00
#include "utils/rel.h"
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
#include "utils/selfuncs.h"
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
#include "utils/spccache.h"
Final cleanup.
1999-07-16 07:00:38 +02:00
#include "utils/syscache.h"
Move Timestamp/Interval typedefs and basic macros into datatype/timestamp.h. As per my recent proposal, this refactors things so that these typedefs and macros are available in a header that can be included in frontend-ish code. I also changed various headers that were undesirably including utils/timestamp.h to include datatype/timestamp.h instead. Unsurprisingly, this showed that half the system was getting utils/timestamp.h by way of xlog.h. No actual code changes here, just header refactoring.
2011-09-09 19:23:41 +02:00
#include "utils/timestamp.h"
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
#include "utils/tqual.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
/* Hooks for plugins to get control when we ask for stats */
get_relation_stats_hook_type get_relation_stats_hook = NULL;
get_index_stats_hook_type get_index_stats_hook = NULL;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
static double var_eq_const(VariableStatData *vardata, Oid operator,
Datum constval, bool constisnull,
bool varonleft);
static double var_eq_non_const(VariableStatData *vardata, Oid operator,
Node *other,
bool varonleft);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
static double ineq_histogram_selectivity(PlannerInfo *root,
VariableStatData *vardata,
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
FmgrInfo *opproc, bool isgt,
Datum constval, Oid consttype);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
static double eqjoinsel_inner(Oid operator,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
VariableStatData *vardata1, VariableStatData *vardata2);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
static double eqjoinsel_semi(Oid operator,
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
VariableStatData *vardata1, VariableStatData *vardata2,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
RelOptInfo *inner_rel);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static bool convert_to_scalar(Datum value, Oid valuetypid, double *scaledvalue,
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
Datum lobound, Datum hibound, Oid boundstypid,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
double *scaledlobound, double *scaledhibound);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static double convert_numeric_to_scalar(Datum value, Oid typid);
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
static void convert_string_to_scalar(char *value,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
double *scaledvalue,
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
char *lobound,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
double *scaledlobound,
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
char *hibound,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
double *scaledhibound);
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
static void convert_bytea_to_scalar(Datum value,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double *scaledvalue,
Datum lobound,
double *scaledlobound,
Datum hibound,
double *scaledhibound);
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
static double convert_one_string_to_scalar(char *value,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
int rangelo, int rangehi);
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
static double convert_one_bytea_to_scalar(unsigned char *value, int valuelen,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
int rangelo, int rangehi);
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
static char *convert_string_datum(Datum value, Oid typid);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static double convert_timevalue_to_scalar(Datum value, Oid typid);
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
static void examine_simple_variable(PlannerInfo *root, Var *var,
VariableStatData *vardata);
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
static bool get_variable_range(PlannerInfo *root, VariableStatData *vardata,
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
Oid sortop, Datum *min, Datum *max);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
static bool get_actual_variable_range(PlannerInfo *root,
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
VariableStatData *vardata,
Oid sortop,
Datum *min, Datum *max);
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
static RelOptInfo *find_join_input_rel(PlannerInfo *root, Relids relids);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
static Selectivity prefix_selectivity(PlannerInfo *root,
VariableStatData *vardata,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid vartype, Oid opfamily, Const *prefixcon);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
static Selectivity pattern_selectivity(Const *patt, Pattern_Type ptype);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static Datum string_to_datum(const char *str, Oid datatype);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
static Const *string_to_const(const char *str, Oid datatype);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
static Const *string_to_bytea_const(const char *str, size_t str_len);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
* eqsel - Selectivity of "=" for any data types.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*
* Note: this routine is also used to estimate selectivity for some
* operators that are not "=" but have comparable selectivity behavior,
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
* such as "~=" (geometric approximate-match). Even for "=", we must
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
* keep in mind that the left and right datatypes may differ.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
eqsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
int varRelid = PG_GETARG_INT32(3);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Node *other;
bool varonleft;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
double selec;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If expression is not variable = something or something = variable, then
* punt and return a default estimate.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!get_restriction_variable(root, args, varRelid,
&vardata, &other, &varonleft))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
PG_RETURN_FLOAT8(DEFAULT_EQ_SEL);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
* We can do a lot better if the something is a constant. (Note: the
* Const might result from estimation rather than being a simple constant
* in the query.)
*/
if (IsA(other, Const))
selec = var_eq_const(&vardata, operator,
((Const *) other)->constvalue,
((Const *) other)->constisnull,
varonleft);
else
selec = var_eq_non_const(&vardata, operator, other,
varonleft);
ReleaseVariableStats(vardata);
PG_RETURN_FLOAT8((float8) selec);
}
/*
* var_eq_const --- eqsel for var = const case
*
* This is split out so that some other estimation functions can use it.
*/
static double
var_eq_const(VariableStatData *vardata, Oid operator,
Datum constval, bool constisnull,
bool varonleft)
{
double selec;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* If the constant is NULL, assume operator is strict and return zero, ie,
* operator will never return TRUE.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (constisnull)
return 0.0;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
/*
* If we matched the var to a unique index, assume there is exactly one
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* match regardless of anything else. (This is slightly bogus, since the
* index's equality operator might be different from ours, but it's more
* likely to be right than ignoring the information.)
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
*/
if (vardata->isunique && vardata->rel && vardata->rel->tuples >= 1.0)
return 1.0 / vardata->rel->tuples;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (HeapTupleIsValid(vardata->statsTuple))
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
{
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Form_pg_statistic stats;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
Datum *values;
int nvalues;
float4 *numbers;
int nnumbers;
bool match = false;
int i;
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
stats = (Form_pg_statistic) GETSTRUCT(vardata->statsTuple);
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/*
* Is the constant "=" to any of the column's most common values?
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* (Although the given operator may not really be "=", we will assume
* that seeing whether it returns TRUE is an appropriate test. If you
* don't like this, maybe you shouldn't be using eqsel for your
* operator...)
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
if (get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
&values, &nvalues,
&numbers, &nnumbers))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
FmgrInfo eqproc;
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
fmgr_info(get_opcode(operator), &eqproc);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
for (i = 0; i < nvalues; i++)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/* be careful to apply operator right way 'round */
if (varonleft)
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
match = DatumGetBool(FunctionCall2Coll(&eqproc,
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
DEFAULT_COLLATION_OID,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
values[i],
constval));
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
else
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
match = DatumGetBool(FunctionCall2Coll(&eqproc,
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
DEFAULT_COLLATION_OID,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
constval,
values[i]));
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (match)
break;
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
}
else
{
/* no most-common-value info available */
values = NULL;
numbers = NULL;
i = nvalues = nnumbers = 0;
}
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (match)
{
/*
* Constant is "=" to this common value. We know selectivity
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* exactly (or as exactly as ANALYZE could calculate it, anyway).
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
selec = numbers[i];
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
else
{
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Comparison is against a constant that is neither NULL nor any
* of the common values. Its selectivity cannot be more than
* this:
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
double sumcommon = 0.0;
double otherdistinct;
for (i = 0; i < nnumbers; i++)
sumcommon += numbers[i];
selec = 1.0 - sumcommon - stats->stanullfrac;
CLAMP_PROBABILITY(selec);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* and in fact it's probably a good deal less. We approximate that
* all the not-common values share this remaining fraction
* equally, so we divide by the number of other distinct values.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
otherdistinct = get_variable_numdistinct(vardata, &isdefault) - nnumbers;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (otherdistinct > 1)
selec /= otherdistinct;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Another cross-check: selectivity shouldn't be estimated as more
* than the least common "most common value".
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
*/
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (nnumbers > 0 && selec > numbers[nnumbers - 1])
selec = numbers[nnumbers - 1];
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
free_attstatsslot(vardata->atttype, values, nvalues,
numbers, nnumbers);
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
else
{
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* No ANALYZE stats available, so make a guess using estimated number
* of distinct values and assuming they are equally common. (The guess
* is unlikely to be very good, but we do know a few special cases.)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
selec = 1.0 / get_variable_numdistinct(vardata, &isdefault);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/* result should be in range, but make sure... */
CLAMP_PROBABILITY(selec);
return selec;
}
/*
* var_eq_non_const --- eqsel for var = something-other-than-const case
*/
static double
var_eq_non_const(VariableStatData *vardata, Oid operator,
Node *other,
bool varonleft)
{
double selec;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
/*
* If we matched the var to a unique index, assume there is exactly one
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* match regardless of anything else. (This is slightly bogus, since the
* index's equality operator might be different from ours, but it's more
* likely to be right than ignoring the information.)
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
*/
if (vardata->isunique && vardata->rel && vardata->rel->tuples >= 1.0)
return 1.0 / vardata->rel->tuples;
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (HeapTupleIsValid(vardata->statsTuple))
{
Form_pg_statistic stats;
double ndistinct;
float4 *numbers;
int nnumbers;
stats = (Form_pg_statistic) GETSTRUCT(vardata->statsTuple);
/*
* Search is for a value that we do not know a priori, but we will
* assume it is not NULL. Estimate the selectivity as non-null
* fraction divided by number of distinct values, so that we get a
* result averaged over all possible values whether common or
* uncommon. (Essentially, we are assuming that the not-yet-known
* comparison value is equally likely to be any of the possible
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* values, regardless of their frequency in the table. Is that a good
* idea?)
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
selec = 1.0 - stats->stanullfrac;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
ndistinct = get_variable_numdistinct(vardata, &isdefault);
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
if (ndistinct > 1)
selec /= ndistinct;
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Cross-check: selectivity should never be estimated as more than the
* most common value's.
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
if (get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
NULL, NULL,
&numbers, &nnumbers))
{
if (nnumbers > 0 && selec > numbers[0])
selec = numbers[0];
free_attstatsslot(vardata->atttype, NULL, 0, numbers, nnumbers);
}
}
else
{
/*
* No ANALYZE stats available, so make a guess using estimated number
* of distinct values and assuming they are equally common. (The guess
* is unlikely to be very good, but we do know a few special cases.)
*/
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
selec = 1.0 / get_variable_numdistinct(vardata, &isdefault);
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/* result should be in range, but make sure... */
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
CLAMP_PROBABILITY(selec);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
return selec;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
* neqsel - Selectivity of "!=" for any data types.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*
* This routine is also used for some operators that are not "!="
* but have comparable selectivity behavior. See above comments
* for eqsel().
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
neqsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
int varRelid = PG_GETARG_INT32(3);
Oid eqop;
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
float8 result;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We want 1 - eqsel() where the equality operator is the one associated
* with this != operator, that is, its negator.
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
eqop = get_negator(operator);
if (eqop)
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
{
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
result = DatumGetFloat8(DirectFunctionCall4(eqsel,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
PointerGetDatum(root),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ObjectIdGetDatum(eqop),
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
PointerGetDatum(args),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Int32GetDatum(varRelid)));
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
}
else
{
/* Use default selectivity (should we raise an error instead?) */
result = DEFAULT_EQ_SEL;
}
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
result = 1.0 - result;
PG_RETURN_FLOAT8(result);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
* scalarineqsel - Selectivity of "<", "<=", ">", ">=" for scalars.
*
* This is the guts of both scalarltsel and scalargtsel. The caller has
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* commuted the clause, if necessary, so that we can treat the variable as
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
* being on the left. The caller must also make sure that the other side
* of the clause is a non-null Const, and dissect same into a value and
* datatype.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
* This routine works for any datatype (or pair of datatypes) known to
* convert_to_scalar(). If it is applied to some other datatype,
* it will return a default estimate.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
static double
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
scalarineqsel(PlannerInfo *root, Oid operator, bool isgt,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData *vardata, Datum constval, Oid consttype)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Form_pg_statistic stats;
FmgrInfo opproc;
double mcv_selec,
hist_selec,
sumcommon;
double selec;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!HeapTupleIsValid(vardata->statsTuple))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
/* no stats available, so default result */
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
return DEFAULT_INEQ_SEL;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
stats = (Form_pg_statistic) GETSTRUCT(vardata->statsTuple);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
fmgr_info(get_opcode(operator), &opproc);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* If we have most-common-values info, add up the fractions of the MCV
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* entries that satisfy MCV OP CONST. These fractions contribute directly
* to the result selectivity. Also add up the total fraction represented
* by MCV entries.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Generalize mcv_selectivity() to support both VAR OP CONST and CONST OP VAR cases. This was not needed in the existing uses within selfuncs.c, but if we're gonna export it for general use, the extra generality seems helpful. Motivated by looking at ltree example.
2006-04-27 19:52:40 +02:00
mcv_selec = mcv_selectivity(vardata, &opproc, constval, true,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
&sumcommon);
/*
* If there is a histogram, determine which bin the constant falls in, and
* compute the resulting contribution to selectivity.
*/
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
hist_selec = ineq_histogram_selectivity(root, vardata, &opproc, isgt,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
constval, consttype);
/*
* Now merge the results from the MCV and histogram calculations,
* realizing that the histogram covers only the non-null values that are
* not listed in MCV.
*/
selec = 1.0 - stats->stanullfrac - sumcommon;
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
if (hist_selec >= 0.0)
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
selec *= hist_selec;
else
{
/*
* If no histogram but there are values not accounted for by MCV,
* arbitrarily assume half of them will match.
*/
selec *= 0.5;
}
selec += mcv_selec;
/* result should be in range, but make sure... */
CLAMP_PROBABILITY(selec);
return selec;
}
/*
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* mcv_selectivity - Examine the MCV list for selectivity estimates
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
*
* Determine the fraction of the variable's MCV population that satisfies
Generalize mcv_selectivity() to support both VAR OP CONST and CONST OP VAR cases. This was not needed in the existing uses within selfuncs.c, but if we're gonna export it for general use, the extra generality seems helpful. Motivated by looking at ltree example.
2006-04-27 19:52:40 +02:00
* the predicate (VAR OP CONST), or (CONST OP VAR) if !varonleft. Also
* compute the fraction of the total column population represented by the MCV
* list. This code will work for any boolean-returning predicate operator.
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
*
* The function result is the MCV selectivity, and the fraction of the
* total population is returned into *sumcommonp. Zeroes are returned
* if there is no MCV list.
*/
If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too.
2006-04-27 02:46:59 +02:00
double
Generalize mcv_selectivity() to support both VAR OP CONST and CONST OP VAR cases. This was not needed in the existing uses within selfuncs.c, but if we're gonna export it for general use, the extra generality seems helpful. Motivated by looking at ltree example.
2006-04-27 19:52:40 +02:00
mcv_selectivity(VariableStatData *vardata, FmgrInfo *opproc,
Datum constval, bool varonleft,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
double *sumcommonp)
{
double mcv_selec,
sumcommon;
Datum *values;
int nvalues;
float4 *numbers;
int nnumbers;
int i;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
mcv_selec = 0.0;
sumcommon = 0.0;
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
if (HeapTupleIsValid(vardata->statsTuple) &&
get_attstatsslot(vardata->statsTuple,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->atttype, vardata->atttypmod,
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
&values, &nvalues,
&numbers, &nnumbers))
{
for (i = 0; i < nvalues; i++)
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
{
Generalize mcv_selectivity() to support both VAR OP CONST and CONST OP VAR cases. This was not needed in the existing uses within selfuncs.c, but if we're gonna export it for general use, the extra generality seems helpful. Motivated by looking at ltree example.
2006-04-27 19:52:40 +02:00
if (varonleft ?
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
DatumGetBool(FunctionCall2Coll(opproc,
DEFAULT_COLLATION_OID,
values[i],
constval)) :
DatumGetBool(FunctionCall2Coll(opproc,
DEFAULT_COLLATION_OID,
constval,
values[i])))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
mcv_selec += numbers[i];
sumcommon += numbers[i];
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
free_attstatsslot(vardata->atttype, values, nvalues,
numbers, nnumbers);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
*sumcommonp = sumcommon;
return mcv_selec;
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
/*
* histogram_selectivity - Examine the histogram for selectivity estimates
*
* Determine the fraction of the variable's histogram entries that satisfy
* the predicate (VAR OP CONST), or (CONST OP VAR) if !varonleft.
*
* This code will work for any boolean-returning predicate operator, whether
* or not it has anything to do with the histogram sort operator. We are
* essentially using the histogram just as a representative sample. However,
* small histograms are unlikely to be all that representative, so the caller
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
* should be prepared to fall back on some other estimation approach when the
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* histogram is missing or very small. It may also be prudent to combine this
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
* approach with another one when the histogram is small.
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
*
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
* If the actual histogram size is not at least min_hist_size, we won't bother
* to do the calculation at all. Also, if the n_skip parameter is > 0, we
* ignore the first and last n_skip histogram elements, on the grounds that
* they are outliers and hence not very representative. Typical values for
* these parameters are 10 and 1.
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
*
* The function result is the selectivity, or -1 if there is no histogram
* or it's smaller than min_hist_size.
*
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
* The output parameter *hist_size receives the actual histogram size,
* or zero if no histogram. Callers may use this number to decide how
* much faith to put in the function result.
*
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* Note that the result disregards both the most-common-values (if any) and
* null entries. The caller is expected to combine this result with
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* statistics for those portions of the column population. It may also be
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* prudent to clamp the result range, ie, disbelieve exact 0 or 1 outputs.
*/
double
histogram_selectivity(VariableStatData *vardata, FmgrInfo *opproc,
Datum constval, bool varonleft,
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
int min_hist_size, int n_skip,
int *hist_size)
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
{
double result;
Datum *values;
int nvalues;
/* check sanity of parameters */
Assert(n_skip >= 0);
Assert(min_hist_size > 2 * n_skip);
if (HeapTupleIsValid(vardata->statsTuple) &&
get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_HISTOGRAM, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
&values, &nvalues,
NULL, NULL))
{
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
*hist_size = nvalues;
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (nvalues >= min_hist_size)
{
int nmatch = 0;
int i;
for (i = n_skip; i < nvalues - n_skip; i++)
{
if (varonleft ?
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
DatumGetBool(FunctionCall2Coll(opproc,
DEFAULT_COLLATION_OID,
values[i],
constval)) :
DatumGetBool(FunctionCall2Coll(opproc,
DEFAULT_COLLATION_OID,
constval,
values[i])))
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
nmatch++;
}
result = ((double) nmatch) / ((double) (nvalues - 2 * n_skip));
}
else
result = -1;
free_attstatsslot(vardata->atttype, values, nvalues, NULL, 0);
}
else
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
{
*hist_size = 0;
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
result = -1;
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
return result;
}
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
/*
* ineq_histogram_selectivity - Examine the histogram for scalarineqsel
*
* Determine the fraction of the variable's histogram population that
* satisfies the inequality condition, ie, VAR < CONST or VAR > CONST.
*
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
* Returns -1 if there is no histogram (valid results will always be >= 0).
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
*
* Note that the result disregards both the most-common-values (if any) and
* null entries. The caller is expected to combine this result with
* statistics for those portions of the column population.
*/
static double
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
ineq_histogram_selectivity(PlannerInfo *root,
VariableStatData *vardata,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
FmgrInfo *opproc, bool isgt,
Datum constval, Oid consttype)
{
double hist_selec;
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
Oid hist_op;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
Datum *values;
int nvalues;
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
hist_selec = -1.0;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* Someday, ANALYZE might store more than one histogram per rel/att,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* corresponding to more than one possible sort ordering defined for the
* column type. However, to make that work we will need to figure out
* which staop to search for --- it's not necessarily the one we have at
* hand! (For example, we might have a '<=' operator rather than the '<'
* operator that will appear in staop.) For now, assume that whatever
* appears in pg_statistic is sorted the same way our operator sorts, or
* the reverse way if isgt is TRUE.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
if (HeapTupleIsValid(vardata->statsTuple) &&
get_attstatsslot(vardata->statsTuple,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->atttype, vardata->atttypmod,
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
STATISTIC_KIND_HISTOGRAM, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
&hist_op,
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
&values, &nvalues,
NULL, NULL))
{
if (nvalues > 1)
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
{
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Use binary search to find proper location, ie, the first slot
* at which the comparison fails. (If the given operator isn't
* actually sort-compatible with the histogram, you'll get garbage
* results ... but probably not any more garbage-y than you would
* from the old linear search.)
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* If the binary search accesses the first or last histogram
* entry, we try to replace that endpoint with the true column min
* or max as found by get_actual_variable_range(). This
* ameliorates misestimates when the min or max is moving as a
* result of changes since the last ANALYZE. Note that this could
* result in effectively including MCVs into the histogram that
* weren't there before, but we don't try to correct for that.
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
*/
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
double histfrac;
int lobound = 0; /* first possible slot to search */
int hibound = nvalues; /* last+1 slot to search */
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
bool have_end = false;
/*
* If there are only two histogram entries, we'll want up-to-date
* values for both. (If there are more than two, we need at most
* one of them to be updated, so we deal with that within the
* loop.)
*/
if (nvalues == 2)
have_end = get_actual_variable_range(root,
vardata,
hist_op,
&values[0],
&values[1]);
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
while (lobound < hibound)
{
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
int probe = (lobound + hibound) / 2;
bool ltcmp;
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/*
* If we find ourselves about to compare to the first or last
* histogram entry, first try to replace it with the actual
* current min or max (unless we already did so above).
*/
if (probe == 0 && nvalues > 2)
have_end = get_actual_variable_range(root,
vardata,
hist_op,
&values[0],
NULL);
else if (probe == nvalues - 1 && nvalues > 2)
have_end = get_actual_variable_range(root,
vardata,
hist_op,
NULL,
&values[probe]);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
ltcmp = DatumGetBool(FunctionCall2Coll(opproc,
DEFAULT_COLLATION_OID,
values[probe],
constval));
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (isgt)
ltcmp = !ltcmp;
if (ltcmp)
lobound = probe + 1;
else
hibound = probe;
}
if (lobound <= 0)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
/* Constant is below lower histogram boundary. */
histfrac = 0.0;
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
else if (lobound >= nvalues)
{
/* Constant is above upper histogram boundary. */
histfrac = 1.0;
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
else
{
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
int i = lobound;
double val,
high,
low;
double binfrac;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
* We have values[i-1] <= constant <= values[i].
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
*
* Convert the constant and the two nearest bin boundary
* values to a uniform comparison scale, and do a linear
* interpolation within this bin.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (convert_to_scalar(constval, consttype, &val,
values[i - 1], values[i],
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
vardata->vartype,
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
&low, &high))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (high <= low)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
/* cope if bin boundaries appear identical */
binfrac = 0.5;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
else if (val <= low)
binfrac = 0.0;
else if (val >= high)
binfrac = 1.0;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
else
{
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
binfrac = (val - low) / (high - low);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Watch out for the possibility that we got a NaN or
* Infinity from the division. This can happen
* despite the previous checks, if for example "low"
* is -Infinity.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (isnan(binfrac) ||
binfrac < 0.0 || binfrac > 1.0)
binfrac = 0.5;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
}
else
{
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Ideally we'd produce an error here, on the grounds that
* the given operator shouldn't have scalarXXsel
* registered as its selectivity func unless we can deal
* with its operand types. But currently, all manner of
* stuff is invoking scalarXXsel, so give a default
* estimate until that can be fixed.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
binfrac = 0.5;
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
/*
* Now, compute the overall selectivity across the values
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* represented by the histogram. We have i-1 full bins and
* binfrac partial bin below the constant.
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
*/
histfrac = (double) (i - 1) + binfrac;
histfrac /= (double) (nvalues - 1);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Now histfrac = fraction of histogram entries below the
* constant.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*
* Account for "<" vs ">"
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
*/
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
hist_selec = isgt ? (1.0 - histfrac) : histfrac;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* The histogram boundaries are only approximate to begin with,
* and may well be out of date anyway. Therefore, don't believe
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
* extremely small or large selectivity estimates --- unless we
* got actual current endpoint values from the table.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*/
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
if (have_end)
CLAMP_PROBABILITY(hist_selec);
else
{
if (hist_selec < 0.0001)
hist_selec = 0.0001;
else if (hist_selec > 0.9999)
hist_selec = 0.9999;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
free_attstatsslot(vardata->atttype, values, nvalues, NULL, 0);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
return hist_selec;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
/*
* scalarltsel - Selectivity of "<" (also "<=") for scalars.
*/
Datum
scalarltsel(PG_FUNCTION_ARGS)
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
int varRelid = PG_GETARG_INT32(3);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Node *other;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
bool varonleft;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Datum constval;
Oid consttype;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
bool isgt;
double selec;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If expression is not variable op something or something op variable,
* then punt and return a default estimate.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!get_restriction_variable(root, args, varRelid,
&vardata, &other, &varonleft))
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Can't do anything useful if the something is not a constant, either.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
if (!IsA(other, Const))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If the constant is NULL, assume operator is strict and return zero, ie,
* operator will never return TRUE.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
if (((Const *) other)->constisnull)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
PG_RETURN_FLOAT8(0.0);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
constval = ((Const *) other)->constvalue;
consttype = ((Const *) other)->consttype;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
* Force the var to be on the left to simplify logic in scalarineqsel.
*/
if (varonleft)
{
/* we have var < other */
isgt = false;
}
else
{
/* we have other < var, commute to make var > other */
operator = get_commutator(operator);
if (!operator)
{
/* Use default selectivity (should we raise an error instead?) */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
}
isgt = true;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
selec = scalarineqsel(root, operator, isgt, &vardata, constval, consttype);
ReleaseVariableStats(vardata);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
PG_RETURN_FLOAT8((float8) selec);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
* scalargtsel - Selectivity of ">" (also ">=") for integers.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
scalargtsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
int varRelid = PG_GETARG_INT32(3);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Node *other;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
bool varonleft;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Datum constval;
Oid consttype;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
bool isgt;
double selec;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If expression is not variable op something or something op variable,
* then punt and return a default estimate.
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!get_restriction_variable(root, args, varRelid,
&vardata, &other, &varonleft))
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Can't do anything useful if the something is not a constant, either.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
if (!IsA(other, Const))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If the constant is NULL, assume operator is strict and return zero, ie,
* operator will never return TRUE.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
if (((Const *) other)->constisnull)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
PG_RETURN_FLOAT8(0.0);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
constval = ((Const *) other)->constvalue;
consttype = ((Const *) other)->consttype;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
* Force the var to be on the left to simplify logic in scalarineqsel.
*/
if (varonleft)
{
/* we have var > other */
isgt = true;
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
else
{
/* we have other > var, commute to make var < other */
operator = get_commutator(operator);
if (!operator)
{
/* Use default selectivity (should we raise an error instead?) */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
}
isgt = false;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
selec = scalarineqsel(root, operator, isgt, &vardata, constval, consttype);
ReleaseVariableStats(vardata);
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
PG_RETURN_FLOAT8((float8) selec);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* patternsel - Generic code for pattern-match selectivity.
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
static double
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
patternsel(PG_FUNCTION_ARGS, Pattern_Type ptype, bool negate)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
int varRelid = PG_GETARG_INT32(3);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Node *other;
bool varonleft;
Datum constval;
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
Oid consttype;
Oid vartype;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid opfamily;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Pattern_Prefix_Status pstatus;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *patt = NULL;
Const *prefix = NULL;
Const *rest = NULL;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
double result;
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
/*
* If this is for a NOT LIKE or similar operator, get the corresponding
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* positive-match operator and work with that. Set result to the correct
* default estimate, too.
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
*/
if (negate)
{
operator = get_negator(operator);
if (!OidIsValid(operator))
elog(ERROR, "patternsel called for operator without a negator");
result = 1.0 - DEFAULT_MATCH_SEL;
}
else
{
result = DEFAULT_MATCH_SEL;
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* If expression is not variable op constant, then punt and return a
* default estimate.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!get_restriction_variable(root, args, varRelid,
&vardata, &other, &varonleft))
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return result;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
if (!varonleft || !IsA(other, Const))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return result;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If the constant is NULL, assume operator is strict and return zero, ie,
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
* operator will never return TRUE. (It's zero even for a negator op.)
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
if (((Const *) other)->constisnull)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
return 0.0;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
constval = ((Const *) other)->constvalue;
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
consttype = ((Const *) other)->consttype;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* The right-hand const is type text or bytea for all supported operators.
* We do not expect to see binary-compatible types here, since
* const-folding should have relabeled the const to exactly match the
* operator's declared type.
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
*/
if (consttype != TEXTOID && consttype != BYTEAOID)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
ReleaseVariableStats(vardata);
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return result;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pgindent run.
2002-09-04 22:31:48 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Similarly, the exposed type of the left-hand side should be one of
* those we know. (Do not look at vardata.atttype, which might be
* something binary-compatible but different.) We can use it to choose
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* the index opfamily from which we must draw the comparison operators.
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* NOTE: It would be more correct to use the PATTERN opfamilies than the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* simple ones, but at the moment ANALYZE will not generate statistics for
* the PATTERN operators. But our results are so approximate anyway that
* it probably hardly matters.
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*/
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
vartype = vardata.vartype;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
switch (vartype)
{
case TEXTOID:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamily = TEXT_BTREE_FAM_OID;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case BPCHAROID:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamily = BPCHAR_BTREE_FAM_OID;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case NAMEOID:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamily = NAME_BTREE_FAM_OID;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case BYTEAOID:
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamily = BYTEA_BTREE_FAM_OID;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
default:
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return result;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
/*
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
* Divide pattern into fixed prefix and remainder. XXX we have to assume
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
* default collation here, because we don't have access to the actual
* input collation for the operator. FIXME ...
*/
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
patt = (Const *) other;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pstatus = pattern_fixed_prefix(patt, ptype, DEFAULT_COLLATION_OID,
&prefix, &rest);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If necessary, coerce the prefix constant to the right type. (The "rest"
* constant need not be changed.)
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*/
if (prefix && prefix->consttype != vartype)
{
pgindent run.
2003-08-04 02:43:34 +02:00
char *prefixstr;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
switch (prefix->consttype)
{
case TEXTOID:
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
prefixstr = TextDatumGetCString(prefix->constvalue);
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
case BYTEAOID:
prefixstr = DatumGetCString(DirectFunctionCall1(byteaout,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
prefix->constvalue));
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
break;
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized consttype: %u",
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
prefix->consttype);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return result;
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
}
prefix = string_to_const(prefixstr, vartype);
pfree(prefixstr);
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
if (pstatus == Pattern_Prefix_Exact)
{
/*
* Pattern specifies an exact match, so pretend operator is '='
*/
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid eqopr = get_opfamily_member(opfamily, vartype, vartype,
BTEqualStrategyNumber);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
if (eqopr == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no = operator for opfamily %u", opfamily);
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
result = var_eq_const(&vardata, eqopr, prefix->constvalue,
false, true);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
else
{
/*
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* Not exact-match pattern. If we have a sufficiently large
* histogram, estimate selectivity for the histogram part of the
* population by counting matches in the histogram. If not, estimate
* selectivity of the fixed prefix and remainder of pattern
* separately, then combine the two to get an estimate of the
* selectivity for the part of the column population represented by
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* the histogram. (For small histograms, we combine these
* approaches.)
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
*
* We then add up data for any most-common-values values; these are
* not in the histogram population, and we can get exact answers for
* them by applying the pattern operator, so there's no reason to
* approximate. (If the MCVs cover a significant part of the total
* population, this gives us a big leg up in accuracy.)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Selectivity selec;
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
int hist_size;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
FmgrInfo opproc;
double nullfrac,
mcv_selec,
sumcommon;
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
/* Try to use the histogram entries to get selectivity */
fmgr_info(get_opcode(operator), &opproc);
selec = histogram_selectivity(&vardata, &opproc, constval, true,
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
10, 1, &hist_size);
/* If not at least 100 entries, use the heuristic method */
if (hist_size < 100)
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
{
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
Selectivity heursel;
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
Selectivity prefixsel;
Selectivity restsel;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (pstatus == Pattern_Prefix_Partial)
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
prefixsel = prefix_selectivity(root, &vardata, vartype,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
opfamily, prefix);
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
else
prefixsel = 1.0;
restsel = pattern_selectivity(rest, ptype);
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
heursel = prefixsel * restsel;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
if (selec < 0) /* fewer than 10 histogram entries? */
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
selec = heursel;
else
{
/*
* For histogram sizes from 10 to 100, we combine the
* histogram and heuristic selectivities, putting increasingly
* more trust in the histogram for larger sizes.
*/
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
double hist_weight = hist_size / 100.0;
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
selec = selec * hist_weight + heursel * (1.0 - hist_weight);
}
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
}
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
Change patternsel() so that instead of switching from a pure pattern-examination heuristic method to purely histogram-driven selectivity at histogram size 100, we compute both estimates and use a weighted average. The weight put on the heuristic estimate decreases linearly with histogram size, dropping to zero for 100 or more histogram entries. Likewise in ltreeparentsel(). After a patch by Greg Stark, though I reorganized the logic a bit to give the caller of histogram_selectivity() more control.
2008-03-09 01:32:09 +01:00
/* In any case, don't believe extremely small or large estimates. */
if (selec < 0.0001)
selec = 0.0001;
else if (selec > 0.9999)
selec = 0.9999;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
/*
* If we have most-common-values info, add up the fractions of the MCV
* entries that satisfy MCV OP PATTERN. These fractions contribute
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* directly to the result selectivity. Also add up the total fraction
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
* represented by MCV entries.
*/
Generalize mcv_selectivity() to support both VAR OP CONST and CONST OP VAR cases. This was not needed in the existing uses within selfuncs.c, but if we're gonna export it for general use, the extra generality seems helpful. Motivated by looking at ltree example.
2006-04-27 19:52:40 +02:00
mcv_selec = mcv_selectivity(&vardata, &opproc, constval, true,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
&sumcommon);
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
if (HeapTupleIsValid(vardata.statsTuple))
nullfrac = ((Form_pg_statistic) GETSTRUCT(vardata.statsTuple))->stanullfrac;
else
nullfrac = 0.0;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
/*
* Now merge the results from the MCV and histogram calculations,
* realizing that the histogram covers only the non-null values that
* are not listed in MCV.
*/
selec *= 1.0 - nullfrac - sumcommon;
selec += mcv_selec;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/* result should be in range, but make sure... */
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
CLAMP_PROBABILITY(selec);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
result = selec;
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
if (prefix)
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
{
pfree(DatumGetPointer(prefix->constvalue));
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
pfree(prefix);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
return negate ? (1.0 - result) : result;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* regexeqsel - Selectivity of regular-expression pattern match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
regexeqsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Regex, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* icregexeqsel - Selectivity of case-insensitive regex match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
icregexeqsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Regex_IC, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* likesel - Selectivity of LIKE pattern match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
likesel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Like, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
/*
* iclikesel - Selectivity of ILIKE pattern match.
*/
Datum
iclikesel(PG_FUNCTION_ARGS)
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Like_IC, false));
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* regexnesel - Selectivity of regular-expression pattern non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
regexnesel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Regex, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* icregexnesel - Selectivity of case-insensitive regex non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
icregexnesel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Regex_IC, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* nlikesel - Selectivity of LIKE pattern non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
nlikesel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Like, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
/*
* icnlikesel - Selectivity of ILIKE pattern non-match.
*/
Datum
icnlikesel(PG_FUNCTION_ARGS)
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternsel(fcinfo, Pattern_Type_Like_IC, true));
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
}
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
* booltestsel - Selectivity of BooleanTest Node.
*/
Selectivity
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
booltestsel(PlannerInfo *root, BoolTestType booltesttype, Node *arg,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
int varRelid, JoinType jointype, SpecialJoinInfo *sjinfo)
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double selec;
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
examine_variable(root, arg, varRelid, &vardata);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (HeapTupleIsValid(vardata.statsTuple))
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
{
Form_pg_statistic stats;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double freq_null;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Datum *values;
int nvalues;
float4 *numbers;
int nnumbers;
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
stats = (Form_pg_statistic) GETSTRUCT(vardata.statsTuple);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
freq_null = stats->stanullfrac;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (get_attstatsslot(vardata.statsTuple,
vardata.atttype, vardata.atttypmod,
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
&values, &nvalues,
&numbers, &nnumbers)
&& nnumbers > 0)
{
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double freq_true;
double freq_false;
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
* Get first MCV frequency and derive frequency for true.
*/
if (DatumGetBool(values[0]))
freq_true = numbers[0];
else
freq_true = 1.0 - numbers[0] - freq_null;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Next derive frequency for false. Then use these as appropriate
* to derive frequency for each case.
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
freq_false = 1.0 - freq_true - freq_null;
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
switch (booltesttype)
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
{
case IS_UNKNOWN:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select only NULL values */
selec = freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_NOT_UNKNOWN:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select non-NULL values */
selec = 1.0 - freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_TRUE:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select only TRUE values */
selec = freq_true;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_NOT_TRUE:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select non-TRUE values */
selec = 1.0 - freq_true;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_FALSE:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select only FALSE values */
selec = freq_false;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_NOT_FALSE:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* select non-FALSE values */
selec = 1.0 - freq_false;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized booltesttype: %d",
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
(int) booltesttype);
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
selec = 0.0; /* Keep compiler quiet */
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
break;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
free_attstatsslot(vardata.atttype, values, nvalues,
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
numbers, nnumbers);
}
else
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* No most-common-value info available. Still have null fraction
* information, so use it for IS [NOT] UNKNOWN. Otherwise adjust
* for null fraction and assume an even split for boolean tests.
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
switch (booltesttype)
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
{
case IS_UNKNOWN:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
* Use freq_null directly.
*/
selec = freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_NOT_UNKNOWN:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Select not unknown (not null) values. Calculate from
* freq_null.
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
selec = 1.0 - freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_TRUE:
case IS_NOT_TRUE:
case IS_FALSE:
case IS_NOT_FALSE:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
selec = (1.0 - freq_null) / 2.0;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized booltesttype: %d",
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
(int) booltesttype);
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
selec = 0.0; /* Keep compiler quiet */
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
break;
}
}
}
else
{
/*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* If we can't get variable statistics for the argument, perhaps
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* clause_selectivity can do something with it. We ignore the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* possibility of a NULL value when using clause_selectivity, and just
* assume the value is either TRUE or FALSE.
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
switch (booltesttype)
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
{
case IS_UNKNOWN:
selec = DEFAULT_UNK_SEL;
break;
case IS_NOT_UNKNOWN:
selec = DEFAULT_NOT_UNK_SEL;
break;
case IS_TRUE:
case IS_NOT_FALSE:
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
selec = (double) clause_selectivity(root, arg,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
varRelid,
jointype, sjinfo);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
break;
case IS_FALSE:
case IS_NOT_TRUE:
selec = 1.0 - (double) clause_selectivity(root, arg,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
varRelid,
jointype, sjinfo);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
break;
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized booltesttype: %d",
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
(int) booltesttype);
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
selec = 0.0; /* Keep compiler quiet */
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
break;
}
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* result should be in range, but make sure... */
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
CLAMP_PROBABILITY(selec);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
return (Selectivity) selec;
}
/*
* nulltestsel - Selectivity of NullTest Node.
*/
Selectivity
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
nulltestsel(PlannerInfo *root, NullTestType nulltesttype, Node *arg,
int varRelid, JoinType jointype, SpecialJoinInfo *sjinfo)
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
double selec;
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
examine_variable(root, arg, varRelid, &vardata);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (HeapTupleIsValid(vardata.statsTuple))
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
{
Form_pg_statistic stats;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
double freq_null;
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
stats = (Form_pg_statistic) GETSTRUCT(vardata.statsTuple);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
freq_null = stats->stanullfrac;
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
switch (nulltesttype)
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
{
case IS_NULL:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
* Use freq_null directly.
*/
selec = freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
case IS_NOT_NULL:
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Select not unknown (not null) values. Calculate from
* freq_null.
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
selec = 1.0 - freq_null;
break;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized nulltesttype: %d",
Fix range-query estimation to not double-exclude NULLs, per gripe from Ray Ontko 28-June-02. Also, fix prefix_selectivity for NAME lefthand variables (it was bogusly assuming binary compatibility), and adjust make_greater_string() to not call pg_mbcliplen() with invalid multibyte data (this last per bug report that I can't find at the moment, but it was in July '02).
2002-10-19 04:56:16 +02:00
(int) nulltesttype);
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
return (Selectivity) 0; /* keep compiler quiet */
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
}
}
else
{
/*
Change patternsel (LIKE/regex selectivity estimation) so that if there is a large enough histogram, it will use the number of matches in the histogram to derive a selectivity estimate, rather than the admittedly pretty bogus heuristics involving examining the pattern contents. I set 'large enough' at 100, but perhaps we should change that later. Also apply the same technique in contrib/ltree's <@ and @> estimator. Per discussion with Stefan Kaltenbrunner and Matteo Beccati.
2006-09-20 21:50:21 +02:00
* No ANALYZE stats available, so make a guess
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
switch (nulltesttype)
{
case IS_NULL:
selec = DEFAULT_UNK_SEL;
break;
case IS_NOT_NULL:
selec = DEFAULT_NOT_UNK_SEL;
break;
default:
elog(ERROR, "unrecognized nulltesttype: %d",
(int) nulltesttype);
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
return (Selectivity) 0; /* keep compiler quiet */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
/* result should be in range, but make sure... */
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
CLAMP_PROBABILITY(selec);
Optimizer can now estimate selectivity of IS NULL, IS NOT NULL, IS TRUE, etc, with some degree of verisimilitude. Split out selectivity support functions from builtins.h into a new header file selfuncs.h, so as to reduce the number of header files builtins.h must depend on. Fix a few missing inclusions exposed thereby. From Joe Conway, with some kibitzing from Tom Lane.
2001-06-25 23:11:45 +02:00
return (Selectivity) selec;
}
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
/*
* strip_array_coercion - strip binary-compatible relabeling from an array expr
*
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
* For array values, the parser normally generates ArrayCoerceExpr conversions,
* but it seems possible that RelabelType might show up. Also, the planner
* is not currently tense about collapsing stacked ArrayCoerceExpr nodes,
* so we need to be ready to deal with more than one level.
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
*/
static Node *
strip_array_coercion(Node *node)
{
for (;;)
{
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
if (node && IsA(node, ArrayCoerceExpr) &&
((ArrayCoerceExpr *) node)->elemfuncid == InvalidOid)
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
{
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
node = (Node *) ((ArrayCoerceExpr *) node)->arg;
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
}
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
else if (node && IsA(node, RelabelType))
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
{
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
/* We don't really expect this case, but may as well cope */
node = (Node *) ((RelabelType *) node)->arg;
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
}
else
break;
}
return node;
}
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
* scalararraysel - Selectivity of ScalarArrayOpExpr Node.
*/
Selectivity
scalararraysel(PlannerInfo *root,
ScalarArrayOpExpr *clause,
bool is_join_clause,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
int varRelid,
JoinType jointype,
SpecialJoinInfo *sjinfo)
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
{
Oid operator = clause->opno;
bool useOr = clause->useOr;
Node *leftop;
Node *rightop;
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
Oid nominal_element_type;
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
Oid nominal_element_collation;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
RegProcedure oprsel;
FmgrInfo oprselproc;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
Selectivity s1;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* First, look up the underlying operator's selectivity estimator. Punt if
* it hasn't got one.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*/
if (is_join_clause)
oprsel = get_oprjoin(operator);
else
oprsel = get_oprrest(operator);
if (!oprsel)
return (Selectivity) 0.5;
fmgr_info(oprsel, &oprselproc);
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
/* deconstruct the expression */
Assert(list_length(clause->args) == 2);
leftop = (Node *) linitial(clause->args);
rightop = (Node *) lsecond(clause->args);
/* get nominal (after relabeling) element type of rightop */
Improve handling of domains over arrays. This patch eliminates various bizarre behaviors caused by sloppy thinking about the difference between a domain type and its underlying array type. In particular, the operation of updating one element of such an array has to be considered as yielding a value of the underlying array type, *not* a value of the domain, because there's no assurance that the domain's CHECK constraints are still satisfied. If we're intending to store the result back into a domain column, we have to re-cast to the domain type so that constraints are re-checked. For similar reasons, such a domain can't be blindly matched to an ANYARRAY polymorphic parameter, because the polymorphic function is likely to apply array-ish operations that could invalidate the domain constraints. For the moment, we just forbid such matching. We might later wish to insert an automatic downcast to the underlying array type, but such a change should also change matching of domains to ANYELEMENT for consistency. To ensure that all such logic is rechecked, this patch removes the original hack of setting a domain's pg_type.typelem field to match its base type; the typelem will always be zero instead. In those places where it's really okay to look through the domain type with no other logic changes, use the newly added get_base_element_type function in place of get_element_type. catversion bumped due to change in pg_type contents. Per bug #5717 from Richard Huxton and subsequent discussion.
2010-10-21 22:07:17 +02:00
nominal_element_type = get_base_element_type(exprType(rightop));
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
if (!OidIsValid(nominal_element_type))
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
return (Selectivity) 0.5; /* probably shouldn't happen */
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
/* get nominal collation, too, for generating constants */
nominal_element_collation = exprCollation(rightop);
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
/* look through any binary-compatible relabeling of rightop */
rightop = strip_array_coercion(rightop);
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
* We consider three cases:
*
* 1. rightop is an Array constant: deconstruct the array, apply the
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* operator's selectivity function for each array element, and merge the
* results in the same way that clausesel.c does for AND/OR combinations.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*
* 2. rightop is an ARRAY[] construct: apply the operator's selectivity
* function for each element of the ARRAY[] construct, and merge.
*
* 3. otherwise, make a guess ...
*/
if (rightop && IsA(rightop, Const))
{
Datum arraydatum = ((Const *) rightop)->constvalue;
bool arrayisnull = ((Const *) rightop)->constisnull;
ArrayType *arrayval;
int16 elmlen;
bool elmbyval;
char elmalign;
int num_elems;
Datum *elem_values;
bool *elem_nulls;
int i;
if (arrayisnull) /* qual can't succeed if null array */
return (Selectivity) 0.0;
arrayval = DatumGetArrayTypeP(arraydatum);
get_typlenbyvalalign(ARR_ELEMTYPE(arrayval),
&elmlen, &elmbyval, &elmalign);
deconstruct_array(arrayval,
ARR_ELEMTYPE(arrayval),
elmlen, elmbyval, elmalign,
&elem_values, &elem_nulls, &num_elems);
s1 = useOr ? 0.0 : 1.0;
for (i = 0; i < num_elems; i++)
{
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
List *args;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
Selectivity s2;
args = list_make2(leftop,
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
makeConst(nominal_element_type,
Fix up the remaining places where the expression node structure would lose available information about the typmod of an expression; namely, Const, ArrayRef, ArrayExpr, and EXPR and ARRAY SubLinks. In the ArrayExpr and SubLink cases it wasn't really the data structure's fault, but exprTypmod() being lazy. This seems like a good idea in view of the expected increase in typmod usage from Teodor's work to allow user-defined types to have typmods. In particular this responds to the concerns we had about eliminating the special-purpose hack that exprTypmod() used to have for BPCHAR Consts. We can now tell whether or not such a Const has been cast to a specific length, and report or display properly if so. initdb forced due to changes in stored rules.
2007-03-17 01:11:05 +01:00
-1,
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
nominal_element_collation,
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
elmlen,
elem_values[i],
elem_nulls[i],
elmbyval));
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (is_join_clause)
s2 = DatumGetFloat8(FunctionCall5(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int16GetDatum(jointype),
PointerGetDatum(sjinfo)));
else
s2 = DatumGetFloat8(FunctionCall4(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int32GetDatum(varRelid)));
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (useOr)
s1 = s1 + s2 - s1 * s2;
else
s1 = s1 * s2;
}
}
else if (rightop && IsA(rightop, ArrayExpr) &&
!((ArrayExpr *) rightop)->multidims)
{
ArrayExpr *arrayexpr = (ArrayExpr *) rightop;
int16 elmlen;
bool elmbyval;
ListCell *l;
get_typlenbyval(arrayexpr->element_typeid,
&elmlen, &elmbyval);
s1 = useOr ? 0.0 : 1.0;
foreach(l, arrayexpr->elements)
{
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
Node *elem = (Node *) lfirst(l);
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
List *args;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
Selectivity s2;
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
/*
* Theoretically, if elem isn't of nominal_element_type we should
* insert a RelabelType, but it seems unlikely that any operator
* estimation function would really care ...
*/
args = list_make2(leftop, elem);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (is_join_clause)
s2 = DatumGetFloat8(FunctionCall5(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int16GetDatum(jointype),
PointerGetDatum(sjinfo)));
else
s2 = DatumGetFloat8(FunctionCall4(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int32GetDatum(varRelid)));
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (useOr)
s1 = s1 + s2 - s1 * s2;
else
s1 = s1 * s2;
}
}
else
{
CaseTestExpr *dummyexpr;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
List *args;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
Selectivity s2;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
int i;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
* We need a dummy rightop to pass to the operator selectivity
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* routine. It can be pretty much anything that doesn't look like a
* constant; CaseTestExpr is a convenient choice.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*/
dummyexpr = makeNode(CaseTestExpr);
Fix scalararraysel() to cope with binary-compatible cases, such as text[] versus varchar[]. This oversight probably explains Ryan Holmes' recent complaint --- he was getting a generic selectivity estimate instead of anything intelligent.
2007-01-28 02:37:38 +01:00
dummyexpr->typeId = nominal_element_type;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
dummyexpr->typeMod = -1;
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
dummyexpr->collation = clause->inputcollid;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
args = list_make2(leftop, dummyexpr);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (is_join_clause)
s2 = DatumGetFloat8(FunctionCall5(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int16GetDatum(jointype),
PointerGetDatum(sjinfo)));
else
s2 = DatumGetFloat8(FunctionCall4(&oprselproc,
PointerGetDatum(root),
ObjectIdGetDatum(operator),
PointerGetDatum(args),
Int32GetDatum(varRelid)));
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
s1 = useOr ? 0.0 : 1.0;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Arbitrarily assume 10 elements in the eventual array value (see
* also estimate_array_length)
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*/
for (i = 0; i < 10; i++)
{
if (useOr)
s1 = s1 + s2 - s1 * s2;
else
s1 = s1 * s2;
}
}
/* result should be in range, but make sure... */
CLAMP_PROBABILITY(s1);
return s1;
}
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
/*
* Estimate number of elements in the array yielded by an expression.
*
* It's important that this agree with scalararraysel.
*/
int
estimate_array_length(Node *arrayexpr)
{
Dept of second thoughts: the IQ of estimate_array_length() needs to be kept on par with that of scalararraysel(), else estimates that should track might not. Hence teach it about binary-compatible cases, too.
2007-01-28 03:53:34 +01:00
/* look through any binary-compatible relabeling of arrayexpr */
arrayexpr = strip_array_coercion(arrayexpr);
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
if (arrayexpr && IsA(arrayexpr, Const))
{
Datum arraydatum = ((Const *) arrayexpr)->constvalue;
bool arrayisnull = ((Const *) arrayexpr)->constisnull;
ArrayType *arrayval;
if (arrayisnull)
return 0;
arrayval = DatumGetArrayTypeP(arraydatum);
return ArrayGetNItems(ARR_NDIM(arrayval), ARR_DIMS(arrayval));
}
else if (arrayexpr && IsA(arrayexpr, ArrayExpr) &&
!((ArrayExpr *) arrayexpr)->multidims)
{
return list_length(((ArrayExpr *) arrayexpr)->elements);
}
else
{
/* default guess --- see also scalararraysel */
return 10;
}
}
Add selectivity-calculation code for RowCompareExpr nodes. Simplistic, but a lot better than nothing at all ...
2006-01-14 01:14:12 +01:00
/*
* rowcomparesel - Selectivity of RowCompareExpr Node.
*
* We estimate RowCompare selectivity by considering just the first (high
* order) columns, which makes it equivalent to an ordinary OpExpr. While
* this estimate could be refined by considering additional columns, it
* seems unlikely that we could do a lot better without multi-column
* statistics.
*/
Selectivity
rowcomparesel(PlannerInfo *root,
RowCompareExpr *clause,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
int varRelid, JoinType jointype, SpecialJoinInfo *sjinfo)
Add selectivity-calculation code for RowCompareExpr nodes. Simplistic, but a lot better than nothing at all ...
2006-01-14 01:14:12 +01:00
{
Selectivity s1;
Oid opno = linitial_oid(clause->opnos);
List *opargs;
bool is_join_clause;
/* Build equivalent arg list for single operator */
opargs = list_make2(linitial(clause->largs), linitial(clause->rargs));
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
/*
* Decide if it's a join clause. This should match clausesel.c's
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* treat_as_join_clause(), except that we intentionally consider only the
* leading columns and not the rest of the clause.
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
*/
Add selectivity-calculation code for RowCompareExpr nodes. Simplistic, but a lot better than nothing at all ...
2006-01-14 01:14:12 +01:00
if (varRelid != 0)
{
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Caller is forcing restriction mode (eg, because we are examining an
* inner indexscan qual).
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
*/
is_join_clause = false;
}
else if (sjinfo == NULL)
{
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* It must be a restriction clause, since it's being evaluated at a
* scan node.
Add selectivity-calculation code for RowCompareExpr nodes. Simplistic, but a lot better than nothing at all ...
2006-01-14 01:14:12 +01:00
*/
is_join_clause = false;
}
else
{
/*
* Otherwise, it's a join if there's more than one relation used.
*/
is_join_clause = (NumRelids((Node *) opargs) > 1);
}
if (is_join_clause)
{
/* Estimate selectivity for a join clause. */
s1 = join_selectivity(root, opno,
opargs,
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
jointype,
sjinfo);
Add selectivity-calculation code for RowCompareExpr nodes. Simplistic, but a lot better than nothing at all ...
2006-01-14 01:14:12 +01:00
}
else
{
/* Estimate selectivity for a restriction clause. */
s1 = restriction_selectivity(root, opno,
opargs,
varRelid);
}
return s1;
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* eqjoinsel - Join selectivity of "="
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
eqjoinsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
#ifdef NOT_USED
Extend join-selectivity API (oprjoin interface) so that join type is passed to join selectivity estimators. Make use of this in eqjoinsel to derive non-bogus selectivity for IN clauses. Further tweaking of cost estimation for IN. initdb forced because of pg_proc.h changes.
2003-01-28 23:13:41 +01:00
JoinType jointype = (JoinType) PG_GETARG_INT16(3);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
#endif
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) PG_GETARG_POINTER(4);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
double selec;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata1;
VariableStatData vardata2;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
bool join_is_reversed;
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
RelOptInfo *inner_rel;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
get_join_variables(root, args, sjinfo,
&vardata1, &vardata2, &join_is_reversed);
switch (sjinfo->jointype)
{
case JOIN_INNER:
case JOIN_LEFT:
case JOIN_FULL:
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
selec = eqjoinsel_inner(operator, &vardata1, &vardata2);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
break;
case JOIN_SEMI:
case JOIN_ANTI:
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
/*
* Look up the join's inner relation. min_righthand is sufficient
* information because neither SEMI nor ANTI joins permit any
* reassociation into or out of their RHS, so the righthand will
* always be exactly that set of rels.
*/
inner_rel = find_join_input_rel(root, sjinfo->min_righthand);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (!join_is_reversed)
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
selec = eqjoinsel_semi(operator, &vardata1, &vardata2,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
inner_rel);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
else
selec = eqjoinsel_semi(get_commutator(operator),
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
&vardata2, &vardata1,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
inner_rel);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
break;
default:
/* other values not expected here */
elog(ERROR, "unrecognized join type: %d",
(int) sjinfo->jointype);
selec = 0; /* keep compiler quiet */
break;
}
ReleaseVariableStats(vardata1);
ReleaseVariableStats(vardata2);
CLAMP_PROBABILITY(selec);
PG_RETURN_FLOAT8((float8) selec);
}
/*
* eqjoinsel_inner --- eqjoinsel for normal inner join
*
* We also use this for LEFT/FULL outer joins; it's not presently clear
* that it's worth trying to distinguish them here.
*/
static double
eqjoinsel_inner(Oid operator,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
VariableStatData *vardata1, VariableStatData *vardata2)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
{
double selec;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
double nd1;
double nd2;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault1;
bool isdefault2;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Form_pg_statistic stats1 = NULL;
Form_pg_statistic stats2 = NULL;
bool have_mcvs1 = false;
Datum *values1 = NULL;
int nvalues1 = 0;
float4 *numbers1 = NULL;
int nnumbers1 = 0;
bool have_mcvs2 = false;
Datum *values2 = NULL;
int nvalues2 = 0;
float4 *numbers2 = NULL;
int nnumbers2 = 0;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
nd1 = get_variable_numdistinct(vardata1, &isdefault1);
nd2 = get_variable_numdistinct(vardata2, &isdefault2);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (HeapTupleIsValid(vardata1->statsTuple))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
stats1 = (Form_pg_statistic) GETSTRUCT(vardata1->statsTuple);
have_mcvs1 = get_attstatsslot(vardata1->statsTuple,
vardata1->atttype,
vardata1->atttypmod,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
STATISTIC_KIND_MCV,
InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
&values1, &nvalues1,
&numbers1, &nnumbers1);
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (HeapTupleIsValid(vardata2->statsTuple))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
stats2 = (Form_pg_statistic) GETSTRUCT(vardata2->statsTuple);
have_mcvs2 = get_attstatsslot(vardata2->statsTuple,
vardata2->atttype,
vardata2->atttypmod,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
STATISTIC_KIND_MCV,
InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
&values2, &nvalues2,
&numbers2, &nnumbers2);
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (have_mcvs1 && have_mcvs2)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We have most-common-value lists for both relations. Run through
* the lists to see which MCVs actually join to each other with the
* given operator. This allows us to determine the exact join
* selectivity for the portion of the relations represented by the MCV
* lists. We still have to estimate for the remaining population, but
* in a skewed distribution this gives us a big leg up in accuracy.
* For motivation see the analysis in Y. Ioannidis and S.
* Christodoulakis, "On the propagation of errors in the size of join
* results", Technical Report 1018, Computer Science Dept., University
* of Wisconsin, Madison, March 1991 (available from ftp.cs.wisc.edu).
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
FmgrInfo eqproc;
bool *hasmatch1;
bool *hasmatch2;
double nullfrac1 = stats1->stanullfrac;
double nullfrac2 = stats2->stanullfrac;
double matchprodfreq,
matchfreq1,
matchfreq2,
unmatchfreq1,
unmatchfreq2,
otherfreq1,
otherfreq2,
totalsel1,
totalsel2;
int i,
nmatches;
fmgr_info(get_opcode(operator), &eqproc);
hasmatch1 = (bool *) palloc0(nvalues1 * sizeof(bool));
hasmatch2 = (bool *) palloc0(nvalues2 * sizeof(bool));
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Note we assume that each MCV will match at most one member of the
* other MCV list. If the operator isn't really equality, there could
* be multiple matches --- but we don't look for them, both for speed
* and because the math wouldn't add up...
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
matchprodfreq = 0.0;
nmatches = 0;
for (i = 0; i < nvalues1; i++)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
int j;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
for (j = 0; j < nvalues2; j++)
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (hasmatch2[j])
continue;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (DatumGetBool(FunctionCall2Coll(&eqproc,
DEFAULT_COLLATION_OID,
values1[i],
values2[j])))
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
hasmatch1[i] = hasmatch2[j] = true;
matchprodfreq += numbers1[i] * numbers2[j];
nmatches++;
break;
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
CLAMP_PROBABILITY(matchprodfreq);
/* Sum up frequencies of matched and unmatched MCVs */
matchfreq1 = unmatchfreq1 = 0.0;
for (i = 0; i < nvalues1; i++)
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (hasmatch1[i])
matchfreq1 += numbers1[i];
else
unmatchfreq1 += numbers1[i];
}
CLAMP_PROBABILITY(matchfreq1);
CLAMP_PROBABILITY(unmatchfreq1);
matchfreq2 = unmatchfreq2 = 0.0;
for (i = 0; i < nvalues2; i++)
{
if (hasmatch2[i])
matchfreq2 += numbers2[i];
else
unmatchfreq2 += numbers2[i];
}
CLAMP_PROBABILITY(matchfreq2);
CLAMP_PROBABILITY(unmatchfreq2);
pfree(hasmatch1);
pfree(hasmatch2);
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Compute total frequency of non-null values that are not in the MCV
* lists.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
otherfreq1 = 1.0 - nullfrac1 - matchfreq1 - unmatchfreq1;
otherfreq2 = 1.0 - nullfrac2 - matchfreq2 - unmatchfreq2;
CLAMP_PROBABILITY(otherfreq1);
CLAMP_PROBABILITY(otherfreq2);
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* We can estimate the total selectivity from the point of view of
* relation 1 as: the known selectivity for matched MCVs, plus
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* unmatched MCVs that are assumed to match against random members of
* relation 2's non-MCV population, plus non-MCV values that are
* assumed to match against random members of relation 2's unmatched
* MCVs plus non-MCV values.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
totalsel1 = matchprodfreq;
if (nd2 > nvalues2)
totalsel1 += unmatchfreq1 * otherfreq2 / (nd2 - nvalues2);
if (nd2 > nmatches)
totalsel1 += otherfreq1 * (otherfreq2 + unmatchfreq2) /
(nd2 - nmatches);
/* Same estimate from the point of view of relation 2. */
totalsel2 = matchprodfreq;
if (nd1 > nvalues1)
totalsel2 += unmatchfreq2 * otherfreq1 / (nd1 - nvalues1);
if (nd1 > nmatches)
totalsel2 += otherfreq2 * (otherfreq1 + unmatchfreq1) /
(nd1 - nmatches);
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* Use the smaller of the two estimates. This can be justified in
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* essentially the same terms as given below for the no-stats case: to
* a first approximation, we are estimating from the point of view of
* the relation with smaller nd.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
selec = (totalsel1 < totalsel2) ? totalsel1 : totalsel2;
}
else
{
/*
* We do not have MCV lists for both sides. Estimate the join
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* selectivity as MIN(1/nd1,1/nd2)*(1-nullfrac1)*(1-nullfrac2). This
* is plausible if we assume that the join operator is strict and the
* non-null values are about equally distributed: a given non-null
* tuple of rel1 will join to either zero or N2*(1-nullfrac2)/nd2 rows
* of rel2, so total join rows are at most
* N1*(1-nullfrac1)*N2*(1-nullfrac2)/nd2 giving a join selectivity of
* not more than (1-nullfrac1)*(1-nullfrac2)/nd2. By the same logic it
* is not more than (1-nullfrac1)*(1-nullfrac2)/nd1, so the expression
* with MIN() is an upper bound. Using the MIN() means we estimate
* from the point of view of the relation with smaller nd (since the
* larger nd is determining the MIN). It is reasonable to assume that
* most tuples in this rel will have join partners, so the bound is
* probably reasonably tight and should be taken as-is.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* XXX Can we be smarter if we have an MCV list for just one side? It
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* seems that if we assume equal distribution for the other side, we
* end up with the same answer anyway.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
double nullfrac1 = stats1 ? stats1->stanullfrac : 0.0;
double nullfrac2 = stats2 ? stats2->stanullfrac : 0.0;
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
selec = (1.0 - nullfrac1) * (1.0 - nullfrac2);
if (nd1 > nd2)
selec /= nd1;
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
else
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
selec /= nd2;
}
eqjoinsel's logic for case where MCV lists are not present should account for NULLs; in hindsight this is obvious since the code for the MCV-lists case would reduce to this when there are zero entries in both lists. Per example from Alec Mitchell.
2003-04-15 07:18:12 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (have_mcvs1)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
free_attstatsslot(vardata1->atttype, values1, nvalues1,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
numbers1, nnumbers1);
if (have_mcvs2)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
free_attstatsslot(vardata2->atttype, values2, nvalues2,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
numbers2, nnumbers2);
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
return selec;
}
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
/*
* eqjoinsel_semi --- eqjoinsel for semi join
*
* (Also used for anti join, which we are supposed to estimate the same way.)
* Caller has ensured that vardata1 is the LHS variable.
*/
static double
eqjoinsel_semi(Oid operator,
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
VariableStatData *vardata1, VariableStatData *vardata2,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
RelOptInfo *inner_rel)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
{
double selec;
double nd1;
double nd2;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault1;
bool isdefault2;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
Form_pg_statistic stats1 = NULL;
bool have_mcvs1 = false;
Datum *values1 = NULL;
int nvalues1 = 0;
float4 *numbers1 = NULL;
int nnumbers1 = 0;
bool have_mcvs2 = false;
Datum *values2 = NULL;
int nvalues2 = 0;
float4 *numbers2 = NULL;
int nnumbers2 = 0;
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
nd1 = get_variable_numdistinct(vardata1, &isdefault1);
nd2 = get_variable_numdistinct(vardata2, &isdefault2);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
/*
* We clamp nd2 to be not more than what we estimate the inner relation's
* size to be. This is intuitively somewhat reasonable since obviously
* there can't be more than that many distinct values coming from the
* inner rel. The reason for the asymmetry (ie, that we don't clamp nd1
* likewise) is that this is the only pathway by which restriction clauses
* applied to the inner rel will affect the join result size estimate,
* since set_joinrel_size_estimates will multiply SEMI/ANTI selectivity by
* only the outer rel's size. If we clamped nd1 we'd be double-counting
* the selectivity of outer-rel restrictions.
*
* We can apply this clamping both with respect to the base relation from
* which the join variable comes (if there is just one), and to the
* immediate inner input relation of the current join.
*/
if (vardata2->rel)
nd2 = Min(nd2, vardata2->rel->rows);
nd2 = Min(nd2, inner_rel->rows);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (HeapTupleIsValid(vardata1->statsTuple))
{
stats1 = (Form_pg_statistic) GETSTRUCT(vardata1->statsTuple);
have_mcvs1 = get_attstatsslot(vardata1->statsTuple,
vardata1->atttype,
vardata1->atttypmod,
STATISTIC_KIND_MCV,
InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
&values1, &nvalues1,
&numbers1, &nnumbers1);
}
if (HeapTupleIsValid(vardata2->statsTuple))
{
have_mcvs2 = get_attstatsslot(vardata2->statsTuple,
vardata2->atttype,
vardata2->atttypmod,
STATISTIC_KIND_MCV,
InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
&values2, &nvalues2,
&numbers2, &nnumbers2);
}
if (have_mcvs1 && have_mcvs2 && OidIsValid(operator))
{
/*
* We have most-common-value lists for both relations. Run through
* the lists to see which MCVs actually join to each other with the
* given operator. This allows us to determine the exact join
* selectivity for the portion of the relations represented by the MCV
* lists. We still have to estimate for the remaining population, but
* in a skewed distribution this gives us a big leg up in accuracy.
*/
FmgrInfo eqproc;
bool *hasmatch1;
bool *hasmatch2;
double nullfrac1 = stats1->stanullfrac;
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
double matchfreq1,
uncertainfrac,
uncertain;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
int i,
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
nmatches,
clamped_nvalues2;
/*
* The clamping above could have resulted in nd2 being less than
* nvalues2; in which case, we assume that precisely the nd2 most
* common values in the relation will appear in the join input, and so
* compare to only the first nd2 members of the MCV list. Of course
* this is frequently wrong, but it's the best bet we can make.
*/
clamped_nvalues2 = Min(nvalues2, nd2);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
fmgr_info(get_opcode(operator), &eqproc);
hasmatch1 = (bool *) palloc0(nvalues1 * sizeof(bool));
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
hasmatch2 = (bool *) palloc0(clamped_nvalues2 * sizeof(bool));
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
/*
* Note we assume that each MCV will match at most one member of the
* other MCV list. If the operator isn't really equality, there could
* be multiple matches --- but we don't look for them, both for speed
* and because the math wouldn't add up...
*/
nmatches = 0;
for (i = 0; i < nvalues1; i++)
{
int j;
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
for (j = 0; j < clamped_nvalues2; j++)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
{
if (hasmatch2[j])
continue;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (DatumGetBool(FunctionCall2Coll(&eqproc,
DEFAULT_COLLATION_OID,
values1[i],
values2[j])))
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
{
hasmatch1[i] = hasmatch2[j] = true;
nmatches++;
break;
}
}
}
/* Sum up frequencies of matched MCVs */
matchfreq1 = 0.0;
for (i = 0; i < nvalues1; i++)
{
if (hasmatch1[i])
matchfreq1 += numbers1[i];
}
CLAMP_PROBABILITY(matchfreq1);
pfree(hasmatch1);
pfree(hasmatch2);
/*
* Now we need to estimate the fraction of relation 1 that has at
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* least one join partner. We know for certain that the matched MCVs
* do, so that gives us a lower bound, but we're really in the dark
* about everything else. Our crude approach is: if nd1 <= nd2 then
* assume all non-null rel1 rows have join partners, else assume for
* the uncertain rows that a fraction nd2/nd1 have join partners. We
* can discount the known-matched MCVs from the distinct-values counts
* before doing the division.
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
*
* Crude as the above is, it's completely useless if we don't have
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
* reliable ndistinct values for both sides. Hence, if either nd1 or
* nd2 is default, punt and assume half of the uncertain rows have
* join partners.
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
*/
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
if (!isdefault1 && !isdefault2)
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
{
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
nd1 -= nmatches;
nd2 -= nmatches;
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
if (nd1 <= nd2 || nd2 < 0)
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
uncertainfrac = 1.0;
else
uncertainfrac = nd2 / nd1;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
}
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
else
uncertainfrac = 0.5;
uncertain = 1.0 - matchfreq1 - nullfrac1;
CLAMP_PROBABILITY(uncertain);
selec = matchfreq1 + uncertainfrac * uncertain;
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
}
else
{
/*
* Without MCV lists for both sides, we can only use the heuristic
* about nd1 vs nd2.
*/
double nullfrac1 = stats1 ? stats1->stanullfrac : 0.0;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
if (!isdefault1 && !isdefault2)
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
{
Further repair of eqjoinsel ndistinct-clamping logic. Examination of examples provided by Mark Kirkwood and others has convinced me that actually commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 was quite a few bricks shy of a load. The useful part of that patch was clamping ndistinct for the inner side of a semi or anti join, and the reason why that's needed is that it's the only way that restriction clauses eliminating rows from the inner relation can affect the estimated size of the join result. I had not clearly understood why the clamping was appropriate, and so mis-extrapolated to conclude that we should clamp ndistinct for the outer side too, as well as for both sides of regular joins. These latter actions were all wrong, and are reverted with this patch. In addition, the clamping logic is now made to affect the behavior of both paths in eqjoinsel_semi, with or without MCV lists to compare. When we have MCVs, we suppose that the most common values are the ones that are most likely to survive the decimation resulting from a lower restriction clause, so we think of the clamping as eliminating non-MCV values, or potentially even the least-common MCVs for the inner relation. Back-patch to 8.4, same as previous fixes in this area.
2011-09-01 06:18:28 +02:00
if (nd1 <= nd2 || nd2 < 0)
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
selec = 1.0 - nullfrac1;
else
selec = (nd2 / nd1) * (1.0 - nullfrac1);
}
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
else
Be more wary of missing statistics in eqjoinsel_semi(). In particular, if we don't have real ndistinct estimates for both sides, fall back to assuming that half of the left-hand rows have join partners. This is what was done in 8.2 and 8.3 (cf nulltestsel() in those versions). It's pretty stupid but it won't lead us to think that an antijoin produces no rows out, as seen in recent example from Uwe Schroeder.
2011-04-12 07:59:34 +02:00
selec = 0.5 * (1.0 - nullfrac1);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
}
if (have_mcvs1)
free_attstatsslot(vardata1->atttype, values1, nvalues1,
numbers1, nnumbers1);
if (have_mcvs2)
free_attstatsslot(vardata2->atttype, values2, nvalues2,
numbers2, nnumbers2);
return selec;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* neqjoinsel - Join selectivity of "!="
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
neqjoinsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
Oid operator = PG_GETARG_OID(1);
List *args = (List *) PG_GETARG_POINTER(2);
Extend join-selectivity API (oprjoin interface) so that join type is passed to join selectivity estimators. Make use of this in eqjoinsel to derive non-bogus selectivity for IN clauses. Further tweaking of cost estimation for IN. initdb forced because of pg_proc.h changes.
2003-01-28 23:13:41 +01:00
JoinType jointype = (JoinType) PG_GETARG_INT16(3);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
SpecialJoinInfo *sjinfo = (SpecialJoinInfo *) PG_GETARG_POINTER(4);
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
Oid eqop;
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
float8 result;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
/*
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
* We want 1 - eqjoinsel() where the equality operator is the one
* associated with this != operator, that is, its negator.
Repair bug noted by Paul Caskey: neqsel() has been generating a bogus result, in fact nearly the opposite of what it should, because it was passing the not-equal operator to eqsel() which would use it to compare the value against the most common value in the column, and of course obtain the wrong result therefrom. Must pass the equality operator to eqsel() instead. Fortunately that's easy to get from the oprnegate link.
2000-08-03 02:58:22 +02:00
*/
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
eqop = get_negator(operator);
if (eqop)
{
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
result = DatumGetFloat8(DirectFunctionCall5(eqjoinsel,
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
PointerGetDatum(root),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
ObjectIdGetDatum(eqop),
Extend join-selectivity API (oprjoin interface) so that join type is passed to join selectivity estimators. Make use of this in eqjoinsel to derive non-bogus selectivity for IN clauses. Further tweaking of cost estimation for IN. initdb forced because of pg_proc.h changes.
2003-01-28 23:13:41 +01:00
PointerGetDatum(args),
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
Int16GetDatum(jointype),
PointerGetDatum(sjinfo)));
Fix eqjoinsel() to make use of new statistics.
2001-05-27 19:37:48 +02:00
}
else
{
/* Use default selectivity (should we raise an error instead?) */
result = DEFAULT_EQ_SEL;
}
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
result = 1.0 - result;
PG_RETURN_FLOAT8(result);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
* scalarltjoinsel - Join selectivity of "<" and "<=" for scalars
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
scalarltjoinsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
/*
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
* scalargtjoinsel - Join selectivity of ">" and ">=" for scalars
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
scalargtjoinsel(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
PG_RETURN_FLOAT8(DEFAULT_INEQ_SEL);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
/*
* patternjoinsel - Generic code for pattern-match join selectivity.
*/
static double
patternjoinsel(PG_FUNCTION_ARGS, Pattern_Type ptype, bool negate)
{
/* For the moment we just punt. */
return negate ? (1.0 - DEFAULT_MATCH_SEL) : DEFAULT_MATCH_SEL;
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* regexeqjoinsel - Join selectivity of regular-expression pattern match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
regexeqjoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Regex, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* icregexeqjoinsel - Join selectivity of case-insensitive regex match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
icregexeqjoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Regex_IC, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* likejoinsel - Join selectivity of LIKE pattern match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
likejoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Like, false));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
/*
* iclikejoinsel - Join selectivity of ILIKE pattern match.
*/
Datum
iclikejoinsel(PG_FUNCTION_ARGS)
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Like_IC, false));
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* regexnejoinsel - Join selectivity of regex non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
regexnejoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Regex, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* icregexnejoinsel - Join selectivity of case-insensitive regex non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
icregexnejoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Regex_IC, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* nlikejoinsel - Join selectivity of LIKE pattern non-match.
*/
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
nlikejoinsel(PG_FUNCTION_ARGS)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Like, true));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
/*
* icnlikejoinsel - Join selectivity of ILIKE pattern non-match.
*/
Datum
icnlikejoinsel(PG_FUNCTION_ARGS)
{
Fix patternsel() and callers to do the right thing for NOT LIKE and the other negated-match operators. patternsel had been using the supplied operator as though it were a positive-match operator, and thus obtaining a wrong result, which was even more wrong after the caller subtracted it from 1. Seems cleanest to give patternsel an explicit "negate" argument so that it knows what's going on. Also install the same factorization scheme for pattern join selectivity estimators; even though they are just stubs at the moment, this may keep someone from making the same type of mistake when they get filled out. Per report from Greg Mullane. Backpatch to 8.2 --- previous releases do not show the problem because patternsel() doesn't actually use the operator directly.
2007-11-07 22:00:37 +01:00
PG_RETURN_FLOAT8(patternjoinsel(fcinfo, Pattern_Type_Like_IC, true));
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
* mergejoinscansel - Scan selectivity of merge join.
*
* A merge join will stop as soon as it exhausts either input stream.
* Therefore, if we can estimate the ranges of both input variables,
* we can estimate how much of the input will actually be read. This
* can have a considerable impact on the cost when using indexscans.
*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* Also, we can estimate how much of each input has to be read before the
* first join pair is found, which will affect the join's startup time.
*
Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.
2007-01-22 21:00:40 +01:00
* clause should be a clause already known to be mergejoinable. opfamily,
* strategy, and nulls_first specify the sort ordering being used.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* The outputs are:
* *leftstart is set to the fraction of the left-hand variable expected
* to be scanned before the first join pair is found (0 to 1).
* *leftend is set to the fraction of the left-hand variable expected
* to be scanned before the join terminates (0 to 1).
* *rightstart, *rightend similarly for the right-hand variable.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
void
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
mergejoinscansel(PlannerInfo *root, Node *clause,
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
Oid opfamily, int strategy, bool nulls_first,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
Selectivity *leftstart, Selectivity *leftend,
Selectivity *rightstart, Selectivity *rightend)
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Node *left,
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*right;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData leftvar,
rightvar;
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
int op_strategy;
Oid op_lefttype;
Oid op_righttype;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Oid opno,
lsortop,
rsortop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
lstatop,
rstatop,
ltop,
Make estimation of mergejoin scan selectivities more robust, per recent example from RaÇl GutiÅrrez.
2003-01-22 21:16:42 +01:00
leop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
revltop,
Make estimation of mergejoin scan selectivities more robust, per recent example from RaÇl GutiÅrrez.
2003-01-22 21:16:42 +01:00
revleop;
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
bool isgt;
Datum leftmin,
leftmax,
rightmin,
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
rightmax;
double selec;
/* Set default results if we can't figure anything out. */
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
/* XXX should default "start" fraction be a bit more than 0? */
*leftstart = *rightstart = 0.0;
*leftend = *rightend = 1.0;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/* Deconstruct the merge clause */
if (!is_opclause(clause))
return; /* shouldn't happen */
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
opno = ((OpExpr *) clause)->opno;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
left = get_leftop((Expr *) clause);
right = get_rightop((Expr *) clause);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
if (!right)
return; /* shouldn't happen */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Look for stats for the inputs */
examine_variable(root, left, 0, &leftvar);
examine_variable(root, right, 0, &rightvar);
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
/* Extract the operator's declared left/right datatypes */
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
get_op_opfamily_properties(opno, opfamily, false,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
&op_strategy,
&op_lefttype,
Since createplan.c no longer cares whether index operators are lossy, it has no particular need to do get_op_opfamily_properties() while building an indexscan plan. Postpone that lookup until executor start. This simplifies createplan.c a lot more than it complicates nodeIndexscan.c, and makes things more uniform since we already had to do it that way for RowCompare expressions. Should be a bit faster too, at least for plans that aren't re-used many times, since we avoid palloc'ing and perhaps copying the intermediate list data structure.
2008-04-13 22:51:21 +02:00
&op_righttype);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Assert(op_strategy == BTEqualStrategyNumber);
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Look up the various operators we need. If we don't find them all, it
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* probably means the opfamily is broken, but we just fail silently.
*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Note: we expect that pg_statistic histograms will be sorted by the '<'
* operator, regardless of which sort direction we are considering.
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
*/
switch (strategy)
{
case BTLessStrategyNumber:
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
isgt = false;
if (op_lefttype == op_righttype)
{
/* easy case */
ltop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTLessStrategyNumber);
leop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTLessEqualStrategyNumber);
lsortop = ltop;
rsortop = ltop;
lstatop = lsortop;
rstatop = rsortop;
revltop = ltop;
revleop = leop;
}
else
{
ltop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTLessStrategyNumber);
leop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTLessEqualStrategyNumber);
lsortop = get_opfamily_member(opfamily,
op_lefttype, op_lefttype,
BTLessStrategyNumber);
rsortop = get_opfamily_member(opfamily,
op_righttype, op_righttype,
BTLessStrategyNumber);
lstatop = lsortop;
rstatop = rsortop;
revltop = get_opfamily_member(opfamily,
op_righttype, op_lefttype,
BTLessStrategyNumber);
revleop = get_opfamily_member(opfamily,
op_righttype, op_lefttype,
BTLessEqualStrategyNumber);
}
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
break;
case BTGreaterStrategyNumber:
/* descending-order case */
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
isgt = true;
if (op_lefttype == op_righttype)
{
/* easy case */
ltop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTGreaterStrategyNumber);
leop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTGreaterEqualStrategyNumber);
lsortop = ltop;
rsortop = ltop;
lstatop = get_opfamily_member(opfamily,
op_lefttype, op_lefttype,
BTLessStrategyNumber);
rstatop = lstatop;
revltop = ltop;
revleop = leop;
}
else
{
ltop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTGreaterStrategyNumber);
leop = get_opfamily_member(opfamily,
op_lefttype, op_righttype,
BTGreaterEqualStrategyNumber);
lsortop = get_opfamily_member(opfamily,
op_lefttype, op_lefttype,
BTGreaterStrategyNumber);
rsortop = get_opfamily_member(opfamily,
op_righttype, op_righttype,
BTGreaterStrategyNumber);
lstatop = get_opfamily_member(opfamily,
op_lefttype, op_lefttype,
BTLessStrategyNumber);
rstatop = get_opfamily_member(opfamily,
op_righttype, op_righttype,
BTLessStrategyNumber);
revltop = get_opfamily_member(opfamily,
op_righttype, op_lefttype,
BTGreaterStrategyNumber);
revleop = get_opfamily_member(opfamily,
op_righttype, op_lefttype,
BTGreaterEqualStrategyNumber);
}
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
break;
default:
goto fail; /* shouldn't get here */
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
if (!OidIsValid(lsortop) ||
!OidIsValid(rsortop) ||
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
!OidIsValid(lstatop) ||
!OidIsValid(rstatop) ||
!OidIsValid(ltop) ||
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
!OidIsValid(leop) ||
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
!OidIsValid(revltop) ||
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
!OidIsValid(revleop))
goto fail; /* insufficient info in catalogs */
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
/* Try to get ranges of both inputs */
if (!isgt)
{
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
if (!get_variable_range(root, &leftvar, lstatop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
&leftmin, &leftmax))
goto fail; /* no range available from stats */
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
if (!get_variable_range(root, &rightvar, rstatop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
&rightmin, &rightmax))
goto fail; /* no range available from stats */
}
else
{
/* need to swap the max and min */
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
if (!get_variable_range(root, &leftvar, lstatop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
&leftmax, &leftmin))
goto fail; /* no range available from stats */
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
if (!get_variable_range(root, &rightvar, rstatop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
&rightmax, &rightmin))
goto fail; /* no range available from stats */
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
* Now, the fraction of the left variable that will be scanned is the
* fraction that's <= the right-side maximum value. But only believe
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* non-default estimates, else stick with our 1.0.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
selec = scalarineqsel(root, leop, isgt, &leftvar,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
rightmax, op_righttype);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
if (selec != DEFAULT_INEQ_SEL)
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*leftend = selec;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/* And similarly for the right variable. */
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
selec = scalarineqsel(root, revleop, isgt, &rightvar,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
leftmax, op_lefttype);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
if (selec != DEFAULT_INEQ_SEL)
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*rightend = selec;
/*
* Only one of the two "end" fractions can really be less than 1.0;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* believe the smaller estimate and reset the other one to exactly 1.0. If
* we get exactly equal estimates (as can easily happen with self-joins),
* believe neither.
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*/
if (*leftend > *rightend)
*leftend = 1.0;
else if (*leftend < *rightend)
*rightend = 1.0;
else
*leftend = *rightend = 1.0;
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Also, the fraction of the left variable that will be scanned before the
* first join pair is found is the fraction that's < the right-side
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* minimum value. But only believe non-default estimates, else stick with
* our own default.
*/
selec = scalarineqsel(root, ltop, isgt, &leftvar,
rightmin, op_righttype);
if (selec != DEFAULT_INEQ_SEL)
*leftstart = selec;
/* And similarly for the right variable. */
selec = scalarineqsel(root, revltop, isgt, &rightvar,
leftmin, op_lefttype);
if (selec != DEFAULT_INEQ_SEL)
*rightstart = selec;
/*
* Only one of the two "start" fractions can really be more than zero;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* believe the larger estimate and reset the other one to exactly 0.0. If
* we get exactly equal estimates (as can easily happen with self-joins),
* believe neither.
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*/
if (*leftstart < *rightstart)
*leftstart = 0.0;
else if (*leftstart > *rightstart)
*rightstart = 0.0;
else
*leftstart = *rightstart = 0.0;
/*
* If the sort order is nulls-first, we're going to have to skip over any
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* nulls too. These would not have been counted by scalarineqsel, and we
* can safely add in this fraction regardless of whether we believe
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* scalarineqsel's results or not. But be sure to clamp the sum to 1.0!
*/
if (nulls_first)
Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.
2007-01-22 21:00:40 +01:00
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
Form_pg_statistic stats;
Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.
2007-01-22 21:00:40 +01:00
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
if (HeapTupleIsValid(leftvar.statsTuple))
{
stats = (Form_pg_statistic) GETSTRUCT(leftvar.statsTuple);
*leftstart += stats->stanullfrac;
CLAMP_PROBABILITY(*leftstart);
*leftend += stats->stanullfrac;
CLAMP_PROBABILITY(*leftend);
}
if (HeapTupleIsValid(rightvar.statsTuple))
{
Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.
2007-01-22 21:00:40 +01:00
stats = (Form_pg_statistic) GETSTRUCT(rightvar.statsTuple);
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*rightstart += stats->stanullfrac;
CLAMP_PROBABILITY(*rightstart);
*rightend += stats->stanullfrac;
CLAMP_PROBABILITY(*rightend);
Put back planner's ability to cache the results of mergejoinscansel(), which I had removed in the first cut of the EquivalenceClass rewrite to simplify that patch a little. But it's still important --- in a four-way join problem mergejoinscansel() was eating about 40% of the planning time according to gprof. Also, improve the EquivalenceClass code to re-use join RestrictInfos rather than generating fresh ones for each join considered. This saves some memory space but more importantly improves the effectiveness of caching planning info in RestrictInfos.
2007-01-22 21:00:40 +01:00
}
}
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
/* Disbelieve start >= end, just in case that can happen */
if (*leftstart >= *leftend)
{
*leftstart = 0.0;
*leftend = 1.0;
}
if (*rightstart >= *rightend)
{
*rightstart = 0.0;
*rightend = 1.0;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
fail:
ReleaseVariableStats(leftvar);
ReleaseVariableStats(rightvar);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
}
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/*
* Helper routine for estimate_num_groups: add an item to a list of
* GroupVarInfos, but only if it's not known equal to any of the existing
* entries.
*/
typedef struct
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
Node *var; /* might be an expression, not just a Var */
RelOptInfo *rel; /* relation it belongs to */
double ndistinct; /* # distinct values */
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
} GroupVarInfo;
static List *
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
add_unique_group_var(PlannerInfo *root, List *varinfos,
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
Node *var, VariableStatData *vardata)
{
GroupVarInfo *varinfo;
double ndistinct;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault;
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
ListCell *lc;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
ndistinct = get_variable_numdistinct(vardata, &isdefault);
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/* cannot use foreach here because of possible list_delete */
lc = list_head(varinfos);
while (lc)
{
varinfo = (GroupVarInfo *) lfirst(lc);
/* must advance lc before list_delete possibly pfree's it */
lc = lnext(lc);
/* Drop exact duplicates */
if (equal(var, varinfo->var))
return varinfos;
/*
* Drop known-equal vars, but only if they belong to different
* relations (see comments for estimate_num_groups)
*/
if (vardata->rel != varinfo->rel &&
exprs_known_equal(root, var, varinfo->var))
{
if (varinfo->ndistinct <= ndistinct)
{
/* Keep older item, forget new one */
return varinfos;
}
else
{
/* Delete the older item */
varinfos = list_delete_ptr(varinfos, varinfo);
}
}
}
varinfo = (GroupVarInfo *) palloc(sizeof(GroupVarInfo));
varinfo->var = var;
varinfo->rel = vardata->rel;
varinfo->ndistinct = ndistinct;
varinfos = lappend(varinfos, varinfo);
return varinfos;
}
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/*
* estimate_num_groups - Estimate number of groups in a grouped query
*
* Given a query having a GROUP BY clause, estimate how many groups there
* will be --- ie, the number of distinct combinations of the GROUP BY
* expressions.
*
* This routine is also used to estimate the number of rows emitted by
* a DISTINCT filtering step; that is an isomorphic problem. (Note:
* actually, we only use it for DISTINCT when there's no grouping or
* aggregation ahead of the DISTINCT.)
*
* Inputs:
* root - the query
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
2003-01-20 19:55:07 +01:00
* groupExprs - list of expressions being grouped by
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* input_rows - number of rows estimated to arrive at the group/unique
* filter step
*
* Given the lack of any cross-correlation statistics in the system, it's
* impossible to do anything really trustworthy with GROUP BY conditions
* involving multiple Vars. We should however avoid assuming the worst
* case (all possible cross-product terms actually appear as groups) since
* very often the grouped-by Vars are highly correlated. Our current approach
* is as follows:
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* 1. Expressions yielding boolean are assumed to contribute two groups,
* independently of their content, and are ignored in the subsequent
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* steps. This is mainly because tests like "col IS NULL" break the
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* heuristic used in step 2 especially badly.
* 2. Reduce the given expressions to a list of unique Vars used. For
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* example, GROUP BY a, a + b is treated the same as GROUP BY a, b.
* It is clearly correct not to count the same Var more than once.
* It is also reasonable to treat f(x) the same as x: f() cannot
* increase the number of distinct values (unless it is volatile,
* which we consider unlikely for grouping), but it probably won't
* reduce the number of distinct values much either.
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
* As a special case, if a GROUP BY expression can be matched to an
* expressional index for which we have statistics, then we treat the
* whole expression as though it were just a Var.
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* 3. If the list contains Vars of different relations that are known equal
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
* due to equivalence classes, then drop all but one of the Vars from each
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* known-equal set, keeping the one with smallest estimated # of values
* (since the extra values of the others can't appear in joined rows).
* Note the reason we only consider Vars of different relations is that
* if we considered ones of the same rel, we'd be double-counting the
* restriction selectivity of the equality in the next step.
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* 4. For Vars within a single source rel, we multiply together the numbers
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
* of values, clamp to the number of rows in the rel (divided by 10 if
* more than one Var), and then multiply by the selectivity of the
* restriction clauses for that rel. When there's more than one Var,
* the initial product is probably too high (it's the worst case) but
* clamping to a fraction of the rel's rows seems to be a helpful
* heuristic for not letting the estimate get out of hand. (The factor
* of 10 is derived from pre-Postgres-7.4 practice.) Multiplying
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* by the restriction selectivity is effectively assuming that the
* restriction clauses are independent of the grouping, which is a crummy
* assumption, but it's hard to do better.
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* 5. If there are Vars from multiple rels, we repeat step 4 for each such
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* rel, and multiply the results together.
* Note that rels not containing grouped Vars are ignored completely, as are
Refactor planner's pathkeys data structure to create a separate, explicit representation of equivalence classes of variables. This is an extensive rewrite, but it brings a number of benefits: * planner no longer fails in the presence of "incomplete" operator families that don't offer operators for every possible combination of datatypes. * avoid generating and then discarding redundant equality clauses. * remove bogus assumption that derived equalities always use operators named "=". * mergejoins can work with a variety of sort orders (e.g., descending) now, instead of tying each mergejoinable operator to exactly one sort order. * better recognition of redundant sort columns. * can make use of equalities appearing underneath an outer join.
2007-01-20 21:45:41 +01:00
* join clauses. Such rels cannot increase the number of groups, and we
* assume such clauses do not reduce the number either (somewhat bogus,
* but we don't have the info to do better).
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*/
double
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
estimate_num_groups(PlannerInfo *root, List *groupExprs, double input_rows)
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
{
List *varinfos = NIL;
double numdistinct;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *l;
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/* We should not be called unless query has GROUP BY (or DISTINCT) */
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
2003-01-20 19:55:07 +01:00
Assert(groupExprs != NIL);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* Count groups derived from boolean grouping expressions. For other
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* expressions, find the unique Vars used, treating an expression as a Var
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
* if we can find stats for it. For each one, record the statistical
* estimate of number of distinct values (total in its table, without
* regard for filtering).
*/
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
numdistinct = 1.0;
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
2003-01-20 19:55:07 +01:00
foreach(l, groupExprs)
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
{
IN clauses appearing at top level of WHERE can now be handled as joins. There are two implementation techniques: the executor understands a new JOIN_IN jointype, which emits at most one matching row per left-hand row, or the result of the IN's sub-select can be fed through a DISTINCT filter and then joined as an ordinary relation. Along the way, some minor code cleanup in the optimizer; notably, break out most of the jointree-rearrangement preprocessing in planner.c and put it in a new file prep/prepjointree.c.
2003-01-20 19:55:07 +01:00
Node *groupexpr = (Node *) lfirst(l);
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
VariableStatData vardata;
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
List *varshere;
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
ListCell *l2;
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
/* Short-circuit for expressions returning boolean */
if (exprType(groupexpr) == BOOLOID)
{
numdistinct *= 2.0;
continue;
}
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/*
* If examine_variable is able to deduce anything about the GROUP BY
* expression, treat it as a single variable even if it's really more
* complicated.
*/
examine_variable(root, groupexpr, 0, &vardata);
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
if (HeapTupleIsValid(vardata.statsTuple) || vardata.isunique)
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
{
varinfos = add_unique_group_var(root, varinfos,
groupexpr, &vardata);
ReleaseVariableStats(vardata);
continue;
}
ReleaseVariableStats(vardata);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/*
Fix estimate_num_groups() to not fail on PlaceHolderVars, per report from Stefan Kaltenbrunner. The most reasonable behavior (at least for the near term) seems to be to ignore the PlaceHolderVar and examine its argument instead. In support of this, change the API of pull_var_clause() to allow callers to request recursion into PlaceHolderVars. Currently estimate_num_groups() is the only customer for that behavior, but where there's one there may be others.
2009-04-19 21:46:33 +02:00
* Else pull out the component Vars. Handle PlaceHolderVars by
* recursing into their arguments (effectively assuming that the
* PlaceHolderVar doesn't change the number of groups, which boils
* down to ignoring the possible addition of nulls to the result set).
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
*/
Avoid listing ungrouped Vars in the targetlist of Agg-underneath-Window. Regular aggregate functions in combination with, or within the arguments of, window functions are OK per spec; they have the semantics that the aggregate output rows are computed and then we run the window functions over that row set. (Thus, this combination is not really useful unless there's a GROUP BY so that more than one aggregate output row is possible.) The case without GROUP BY could fail, as recently reported by Jeff Davis, because sloppy construction of the Agg node's targetlist resulted in extra references to possibly-ungrouped Vars appearing outside the aggregate function calls themselves. See the added regression test case for an example. Fixing this requires modifying the API of flatten_tlist and its underlying function pull_var_clause. I chose to make pull_var_clause's API for aggregates identical to what it was already doing for placeholders, since the useful behaviors turn out to be the same (error, report node as-is, or recurse into it). I also tightened the error checking in this area a bit: if it was ever valid to see an uplevel Var, Aggref, or PlaceHolderVar here, that was a long time ago, so complain instead of ignoring them. Backpatch into 9.1. The failure exists in 8.4 and 9.0 as well, but seeing that it only occurs in a basically-useless corner case, it doesn't seem worth the risks of changing a function API in a minor release. There might be third-party code using pull_var_clause.
2011-07-13 00:23:55 +02:00
varshere = pull_var_clause(groupexpr,
PVC_RECURSE_AGGREGATES,
PVC_RECURSE_PLACEHOLDERS);
pgindent run.
2003-08-04 02:43:34 +02:00
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/*
pgindent run.
2003-08-04 02:43:34 +02:00
* If we find any variable-free GROUP BY item, then either it is a
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* constant (and we can ignore it) or it contains a volatile function;
* in the latter case we punt and assume that each input row will
* yield a distinct group.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*/
if (varshere == NIL)
{
if (contain_volatile_functions(groupexpr))
return input_rows;
continue;
}
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
/*
* Else add variables to varinfos list
*/
foreach(l2, varshere)
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
{
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
Node *var = (Node *) lfirst(l2);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
examine_variable(root, var, 0, &vardata);
varinfos = add_unique_group_var(root, varinfos, var, &vardata);
ReleaseVariableStats(vardata);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
}
}
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
/*
* If now no Vars, we must have an all-constant or all-boolean GROUP BY
* list.
*/
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
if (varinfos == NIL)
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
{
/* Guard against out-of-range answers */
if (numdistinct > input_rows)
numdistinct = input_rows;
return numdistinct;
}
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/*
Fix estimate_num_groups() to assume that GROUP BY expressions yielding boolean results always contribute two groups, regardless of the expression contents. This is very substantially more accurate than the regular heuristic for certain boolean tests like "col IS NULL". Per gripe from Sam Mason. Back-patch to all supported releases, since the behavior of estimate_num_groups() hasn't changed all that much since 7.4.
2008-07-07 22:24:55 +02:00
* Group Vars by relation and estimate total numdistinct.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*
pgindent run.
2003-08-04 02:43:34 +02:00
* For each iteration of the outer loop, we process the frontmost Var in
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* varinfos, plus all other Vars in the same relation. We remove these
* Vars from the newvarinfos list for the next iteration. This is the
* easiest way to group Vars of same rel together.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*/
do
{
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
GroupVarInfo *varinfo1 = (GroupVarInfo *) linitial(varinfos);
RelOptInfo *rel = varinfo1->rel;
pgindent run.
2003-08-04 02:43:34 +02:00
double reldistinct = varinfo1->ndistinct;
Adjust estimate_num_groups() to not clamp per-relation group count estimate to less than the number of values estimated for any one grouping Var, as suggested by Manfred. This is intuitively right, and what's more it puts the plan choices in the subselect regression test back the way they were before ...
2005-02-02 00:08:13 +01:00
double relmaxndistinct = reldistinct;
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
int relvarcount = 1;
pgindent run.
2003-08-04 02:43:34 +02:00
List *newvarinfos = NIL;
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/*
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
* Get the product of numdistinct estimates of the Vars for this rel.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
* Also, construct new varinfos list of remaining Vars.
*/
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
for_each_cell(l, lnext(list_head(varinfos)))
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
{
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
GroupVarInfo *varinfo2 = (GroupVarInfo *) lfirst(l);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
Fix estimate_num_groups to be able to use expression-index statistics when there is an expressional index matching a GROUP BY item.
2004-09-18 21:39:50 +02:00
if (varinfo2->rel == varinfo1->rel)
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
{
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
reldistinct *= varinfo2->ndistinct;
Adjust estimate_num_groups() to not clamp per-relation group count estimate to less than the number of values estimated for any one grouping Var, as suggested by Manfred. This is intuitively right, and what's more it puts the plan choices in the subselect regression test back the way they were before ...
2005-02-02 00:08:13 +01:00
if (relmaxndistinct < varinfo2->ndistinct)
relmaxndistinct = varinfo2->ndistinct;
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
relvarcount++;
}
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
else
{
/* not time to process varinfo2 yet */
newvarinfos = lcons(varinfo2, newvarinfos);
}
}
/*
Avoid division by zero in estimate_num_groups() when table has no rows.
2003-10-16 23:37:54 +02:00
* Sanity check --- don't divide by zero if empty relation.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*/
Assert(rel->reloptkind == RELOPT_BASEREL);
Avoid division by zero in estimate_num_groups() when table has no rows.
2003-10-16 23:37:54 +02:00
if (rel->tuples > 0)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Clamp to size of rel, or size of rel / 10 if multiple Vars. The
* fudge factor is because the Vars are probably correlated but we
* don't know by how much. We should never clamp to less than the
* largest ndistinct value for any of the Vars, though, since
* there will surely be at least that many groups.
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
*/
double clamp = rel->tuples;
if (relvarcount > 1)
Adjust estimate_num_groups() to not clamp per-relation group count estimate to less than the number of values estimated for any one grouping Var, as suggested by Manfred. This is intuitively right, and what's more it puts the plan choices in the subselect regression test back the way they were before ...
2005-02-02 00:08:13 +01:00
{
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
clamp *= 0.1;
Adjust estimate_num_groups() to not clamp per-relation group count estimate to less than the number of values estimated for any one grouping Var, as suggested by Manfred. This is intuitively right, and what's more it puts the plan choices in the subselect regression test back the way they were before ...
2005-02-02 00:08:13 +01:00
if (clamp < relmaxndistinct)
{
clamp = relmaxndistinct;
/* for sanity in case some ndistinct is too large: */
if (clamp > rel->tuples)
clamp = rel->tuples;
}
}
When dealing with multiple grouping columns coming from the same table, clamp the estimated number of groups to table row count over 10, instead of table row count; this reflects a heuristic that people probably won't group over a near-unique set of columns, and the knowledge that we don't currently have any way to estimate the correlation of the columns better than guessing. This change creates a trivial plan change in one of the regression tests.
2005-01-28 21:34:27 +01:00
if (reldistinct > clamp)
reldistinct = clamp;
/*
* Multiply by restriction selectivity.
Avoid division by zero in estimate_num_groups() when table has no rows.
2003-10-16 23:37:54 +02:00
*/
reldistinct *= rel->rows / rel->tuples;
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
Avoid division by zero in estimate_num_groups() when table has no rows.
2003-10-16 23:37:54 +02:00
/*
* Update estimate of total distinct groups.
*/
numdistinct *= reldistinct;
}
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
varinfos = newvarinfos;
} while (varinfos != NIL);
Upgrade cost estimation for joins, per discussion with Bradley Baetz. Try to model the effect of rescanning input tuples in mergejoins; account for JOIN_IN short-circuiting where appropriate. Also, recognize that mergejoin and hashjoin clauses may now be more than single operator calls, so we have to charge appropriate execution costs.
2003-01-27 21:51:54 +01:00
numdistinct = ceil(numdistinct);
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
/* Guard against out-of-range answers */
if (numdistinct > input_rows)
numdistinct = input_rows;
if (numdistinct < 1.0)
numdistinct = 1.0;
return numdistinct;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* Estimate hash bucketsize fraction (ie, number of entries in a bucket
* divided by total tuples in relation) if the specified expression is used
* as a hash key.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* XXX This is really pretty bogus since we're effectively assuming that the
* distribution of hash keys will be the same after applying restriction
* clauses as it was in the underlying relation. However, we are not nearly
* smart enough to figure out how the restrict clauses might change the
* distribution, so this will have to do for now.
Add an at-least-marginally-plausible method of estimating the number of groups produced by GROUP BY. This improves the accuracy of planning estimates for grouped subselects, and is needed to check whether a hashed aggregation plan risks memory overflow.
2002-11-20 00:22:00 +01:00
*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* We are passed the number of buckets the executor will use for the given
* input relation. If the data were perfectly distributed, with the same
* number of tuples going into each available bucket, then the bucketsize
* fraction would be 1/nbuckets. But this happy state of affairs will occur
* only if (a) there are at least nbuckets distinct data values, and (b)
* we have a not-too-skewed data distribution. Otherwise the buckets will
* be nonuniformly occupied. If the other relation in the join has a key
* distribution similar to this one's, then the most-loaded buckets are
* exactly those that will be probed most often. Therefore, the "average"
* bucket size for costing purposes should really be taken as something close
* to the "worst case" bucket size. We try to estimate this by adjusting the
* fraction if there are too few distinct data values, and then scaling up
* by the ratio of the most common value's frequency to the average frequency.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* If no statistics are available, use a default estimate of 0.1. This will
* discourage use of a hash rather strongly if the inner relation is large,
* which is what we want. We do not want to hash unless we know that the
* inner rel is well-dispersed (or the alternatives seem much worse).
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Selectivity
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
estimate_hash_bucketsize(PlannerInfo *root, Node *hashkey, double nbuckets)
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData vardata;
double estfract,
ndistinct,
stanullfrac,
mcvfreq,
avgfreq;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
bool isdefault;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
float4 *numbers;
int nnumbers;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
examine_variable(root, hashkey, 0, &vardata);
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
/* Get number of distinct values */
ndistinct = get_variable_numdistinct(&vardata, &isdefault);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
/* If ndistinct isn't real, punt and return 0.1, per comments above */
if (isdefault)
{
ReleaseVariableStats(vardata);
return (Selectivity) 0.1;
}
/* Get fraction that are null */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (HeapTupleIsValid(vardata.statsTuple))
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Form_pg_statistic stats;
stats = (Form_pg_statistic) GETSTRUCT(vardata.statsTuple);
stanullfrac = stats->stanullfrac;
}
else
stanullfrac = 0.0;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Compute avg freq of all distinct data values in raw relation */
avgfreq = (1.0 - stanullfrac) / ndistinct;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Adjust ndistinct to account for restriction clauses. Observe we are
* assuming that the data distribution is affected uniformly by the
* restriction clauses!
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* XXX Possibly better way, but much more expensive: multiply by
* selectivity of rel's restriction clauses that mention the target Var.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (vardata.rel)
ndistinct *= vardata.rel->rows / vardata.rel->tuples;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Initial estimate of bucketsize fraction is 1/nbuckets as long as the
* number of buckets is less than the expected number of distinct values;
* otherwise it is 1/ndistinct.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
Revise hash join code so that we can increase the number of batches on-the-fly, and thereby avoid blowing out memory when the planner has underestimated the hash table size. Hash join will now obey the work_mem limit with some faithfulness. Per my recent proposal (hash aggregate part isn't done yet though).
2005-03-06 23:15:05 +01:00
if (ndistinct > nbuckets)
estfract = 1.0 / nbuckets;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
else
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
estfract = 1.0 / ndistinct;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* Look up the frequency of the most common value, if available.
*/
mcvfreq = 0.0;
if (HeapTupleIsValid(vardata.statsTuple))
{
if (get_attstatsslot(vardata.statsTuple,
vardata.atttype, vardata.atttypmod,
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
NULL, NULL,
&numbers, &nnumbers))
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* The first MCV stat is for the most common value.
*/
if (nnumbers > 0)
mcvfreq = numbers[0];
free_attstatsslot(vardata.atttype, NULL, 0,
numbers, nnumbers);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
}
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Adjust estimated bucketsize upward to account for skewed distribution.
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (avgfreq > 0.0 && mcvfreq > avgfreq)
estfract *= mcvfreq / avgfreq;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* Clamp bucketsize to sane range (the above adjustment could easily
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* produce an out-of-range result). We set the lower bound a little above
* zero, since zero isn't a very sane result.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (estfract < 1.0e-6)
estfract = 1.0e-6;
else if (estfract > 1.0)
estfract = 1.0;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ReleaseVariableStats(vardata);
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return (Selectivity) estfract;
Teach planner about the idea that a mergejoin won't necessarily read both input streams to the end. If one variable's range is much less than the other, an indexscan-based merge can win by not scanning all of the other table. Per example from Reinhard Max.
2002-03-01 05:09:28 +01:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*-------------------------------------------------------------------------
*
* Support routines
*
*-------------------------------------------------------------------------
*/
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
/*
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
* convert_to_scalar
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* Convert non-NULL values of the indicated types to the comparison
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
* scale needed by scalarineqsel().
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
* Returns "true" if successful.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
* XXX this routine is a hack: ideally we should look up the conversion
* subroutines in pg_type.
*
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
* All numeric datatypes are simply converted to their equivalent
Make selectivity routines cope gracefully with NaNs, infinities, and NUMERIC values that are out of the range of 'double'. Per trouble report from Mike Quinn.
2001-10-14 01:32:34 +02:00
* "double" values. (NUMERIC values that are outside the range of "double"
* are clamped to +/- HUGE_VAL.)
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
*
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
* String datatypes are converted by convert_string_to_scalar(),
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* which is explained below. The reason why this routine deals with
* three values at a time, not just one, is that we need it for strings.
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
*
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
* The bytea datatype is just enough different from strings that it has
* to be treated separately.
*
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
* The several datatypes representing absolute times are all converted
* to Timestamp, which is actually a double, and then we just use that
Make selectivity routines cope gracefully with NaNs, infinities, and NUMERIC values that are out of the range of 'double'. Per trouble report from Mike Quinn.
2001-10-14 01:32:34 +02:00
* double value. Note this will give correct results even for the "special"
* values of Timestamp, since those are chosen to compare correctly;
* see timestamp_cmp.
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
*
* The several datatypes representing relative times (intervals) are all
* converted to measurements expressed in seconds.
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static bool
convert_to_scalar(Datum value, Oid valuetypid, double *scaledvalue,
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
Datum lobound, Datum hibound, Oid boundstypid,
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
double *scaledlobound, double *scaledhibound)
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
{
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Both the valuetypid and the boundstypid should exactly match the
* declared input type(s) of the operator we are invoked for, so we just
* error out if either is not recognized.
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* XXX The histogram we are interpolating between points of could belong
* to a column that's only binary-compatible with the declared type. In
* essence we are assuming that the semantics of binary-compatible types
* are enough alike that we can use a histogram generated with one type's
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* operators to estimate selectivity for the other's. This is outright
* wrong in some cases --- in particular signed versus unsigned
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
* interpretation could trip us up. But it's useful enough in the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* majority of cases that we do it anyway. Should think about more
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
* rigorous ways to do it.
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
switch (valuetypid)
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
{
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
* Built-in numeric types
*/
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
case BOOLOID:
case INT2OID:
case INT4OID:
case INT8OID:
case FLOAT4OID:
case FLOAT8OID:
case NUMERICOID:
case OIDOID:
case REGPROCOID:
Implement types regprocedure, regoper, regoperator, regclass, regtype per pghackers discussion. Add some more typsanity tests, and clean up some problems exposed thereby (broken or missing array types for some built-in types). Also, clean up loose ends from unknownin/out patch.
2002-04-25 04:56:56 +02:00
case REGPROCEDUREOID:
case REGOPEROID:
case REGOPERATOROID:
case REGCLASSOID:
case REGTYPEOID:
Tsearch2 functionality migrates to core. The bulk of this work is by Oleg Bartunov and Teodor Sigaev, but I did a lot of editorializing, so anything that's broken is probably my fault. Documentation is nonexistent as yet, but let's land the patch so we can get some portability testing done.
2007-08-21 03:11:32 +02:00
case REGCONFIGOID:
case REGDICTIONARYOID:
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*scaledvalue = convert_numeric_to_scalar(value, valuetypid);
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
*scaledlobound = convert_numeric_to_scalar(lobound, boundstypid);
*scaledhibound = convert_numeric_to_scalar(hibound, boundstypid);
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
return true;
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
* Built-in string types
*/
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
case CHAROID:
case BPCHAROID:
case VARCHAROID:
Ooops ... I had left some test coding in selfuncs.c that failed on 'field < textconstant' ...
1999-08-21 02:56:18 +02:00
case TEXTOID:
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
case NAMEOID:
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
{
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
char *valstr = convert_string_datum(value, valuetypid);
char *lostr = convert_string_datum(lobound, boundstypid);
char *histr = convert_string_datum(hibound, boundstypid);
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
convert_string_to_scalar(valstr, scaledvalue,
lostr, scaledlobound,
histr, scaledhibound);
pfree(valstr);
pfree(lostr);
pfree(histr);
return true;
}
First cut at making useful selectivity estimates for range queries (ie, WHERE x > lowbound AND x < highbound). It's not very bright yet but it does something useful. Also, rename intltsel/intgtsel to scalarltsel/scalargtsel to reflect usage better. Extend convert_to_scalar to do something a little bit useful with string data types. Still need to make it do something with date/time datatypes, but I'll wait for Thomas's datetime unification dust to settle first. Eventually the routine ought not have any type-specific knowledge at all; it ought to be calling a type-dependent routine found via a pg_type column; but that's a task for another day.
2000-01-24 08:16:52 +01:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
* Built-in bytea type
*/
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
case BYTEAOID:
{
convert_bytea_to_scalar(value, scaledvalue,
lobound, scaledlobound,
hibound, scaledhibound);
return true;
}
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
* Built-in time types
*/
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
case TIMESTAMPOID:
Add missing TIMESTAMPTZ case.
2001-10-03 20:25:59 +02:00
case TIMESTAMPTZOID:
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
case ABSTIMEOID:
case DATEOID:
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
case INTERVALOID:
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
case RELTIMEOID:
case TINTERVALOID:
case TIMEOID:
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
case TIMETZOID:
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*scaledvalue = convert_timevalue_to_scalar(value, valuetypid);
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
*scaledlobound = convert_timevalue_to_scalar(lobound, boundstypid);
*scaledhibound = convert_timevalue_to_scalar(hibound, boundstypid);
Add date and time datatype handling to convert_to_scalar. (I was waiting for Thomas to do the datetime consolidation before touching this, but it's done now...)
2000-02-27 00:03:12 +01:00
return true;
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
* Built-in network types
*/
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
case INETOID:
case CIDROID:
case MACADDROID:
*scaledvalue = convert_network_to_scalar(value, valuetypid);
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
*scaledlobound = convert_network_to_scalar(lobound, boundstypid);
*scaledhibound = convert_network_to_scalar(hibound, boundstypid);
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
return true;
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
}
/* Don't know how to convert */
Clean up possibly-uninitialized-variable warnings reported by gcc 4.x.
2005-09-25 00:54:44 +02:00
*scaledvalue = *scaledlobound = *scaledhibound = 0;
Further selectivity-estimation work. Speed up eqsel() (it should just call the given operator, not look up an = operator). Fix intltsel() so that all numeric data types are converted to double before trying to estimate where the given comparison value is in the known range of column values. intltsel() still needs work, or replacement, for non-numeric data types ... but for nonintegral numeric types it should now be delivering reasonable estimates.
1999-08-02 04:05:41 +02:00
return false;
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* Do convert_to_scalar()'s work for any numeric data type.
*/
static double
convert_numeric_to_scalar(Datum value, Oid typid)
{
switch (typid)
{
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
case BOOLOID:
First round of changes for new fmgr interface. fmgr itself and the key call sites are changed, but most called functions are still oldstyle. An exception is that the PL managers are updated (so, for example, NULL handling now behaves as expected in plperl and plpgsql functions). NOTE initdb is forced due to added column in pg_proc.
2000-05-28 19:56:29 +02:00
return (double) DatumGetBool(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case INT2OID:
return (double) DatumGetInt16(value);
case INT4OID:
return (double) DatumGetInt32(value);
case INT8OID:
First round of changes for new fmgr interface. fmgr itself and the key call sites are changed, but most called functions are still oldstyle. An exception is that the PL managers are updated (so, for example, NULL handling now behaves as expected in plperl and plpgsql functions). NOTE initdb is forced due to added column in pg_proc.
2000-05-28 19:56:29 +02:00
return (double) DatumGetInt64(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case FLOAT4OID:
First round of changes for new fmgr interface. fmgr itself and the key call sites are changed, but most called functions are still oldstyle. An exception is that the PL managers are updated (so, for example, NULL handling now behaves as expected in plperl and plpgsql functions). NOTE initdb is forced due to added column in pg_proc.
2000-05-28 19:56:29 +02:00
return (double) DatumGetFloat4(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case FLOAT8OID:
First round of changes for new fmgr interface. fmgr itself and the key call sites are changed, but most called functions are still oldstyle. An exception is that the PL managers are updated (so, for example, NULL handling now behaves as expected in plperl and plpgsql functions). NOTE initdb is forced due to added column in pg_proc.
2000-05-28 19:56:29 +02:00
return (double) DatumGetFloat8(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case NUMERICOID:
Make selectivity routines cope gracefully with NaNs, infinities, and NUMERIC values that are out of the range of 'double'. Per trouble report from Mike Quinn.
2001-10-14 01:32:34 +02:00
/* Note: out-of-range values will be clamped to +-HUGE_VAL */
return (double)
DatumGetFloat8(DirectFunctionCall1(numeric_float8_no_overflow,
value));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case OIDOID:
case REGPROCOID:
Implement types regprocedure, regoper, regoperator, regclass, regtype per pghackers discussion. Add some more typsanity tests, and clean up some problems exposed thereby (broken or missing array types for some built-in types). Also, clean up loose ends from unknownin/out patch.
2002-04-25 04:56:56 +02:00
case REGPROCEDUREOID:
case REGOPEROID:
case REGOPERATOROID:
case REGCLASSOID:
case REGTYPEOID:
Tsearch2 functionality migrates to core. The bulk of this work is by Oleg Bartunov and Teodor Sigaev, but I did a lot of editorializing, so anything that's broken is probably my fault. Documentation is nonexistent as yet, but let's land the patch so we can get some portability testing done.
2007-08-21 03:11:32 +02:00
case REGCONFIGOID:
case REGDICTIONARYOID:
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/* we can treat OIDs as integers... */
return (double) DatumGetObjectId(value);
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Can't get here unless someone tries to use scalarltsel/scalargtsel on
* an operator with one numeric and one non-numeric operand.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unsupported type: %u", typid);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return 0;
}
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
/*
* Do convert_to_scalar()'s work for any character-string data type.
*
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* String datatypes are converted to a scale that ranges from 0 to 1,
* where we visualize the bytes of the string as fractional digits.
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
*
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* We do not want the base to be 256, however, since that tends to
* generate inflated selectivity estimates; few databases will have
* occurrences of all 256 possible byte values at each position.
* Instead, use the smallest and largest byte values seen in the bounds
* as the estimated range for each byte, after some fudging to deal with
* the fact that we probably aren't going to see the full range that way.
*
* An additional refinement is that we discard any common prefix of the
* three strings before computing the scaled values. This allows us to
* "zoom in" when we encounter a narrow data range. An example is a phone
* number database where all the values begin with the same area code.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
* (Actually, the bounds will be adjacent histogram-bin-boundary values,
* so this is more likely to happen than you might think.)
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
static void
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
convert_string_to_scalar(char *value,
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
double *scaledvalue,
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
char *lobound,
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
double *scaledlobound,
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
char *hibound,
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
double *scaledhibound)
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
{
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
int rangelo,
rangehi;
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
char *sptr;
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
rangelo = rangehi = (unsigned char) hibound[0];
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
for (sptr = lobound; *sptr; sptr++)
{
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
if (rangelo > (unsigned char) *sptr)
rangelo = (unsigned char) *sptr;
if (rangehi < (unsigned char) *sptr)
rangehi = (unsigned char) *sptr;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
for (sptr = hibound; *sptr; sptr++)
{
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
if (rangelo > (unsigned char) *sptr)
rangelo = (unsigned char) *sptr;
if (rangehi < (unsigned char) *sptr)
rangehi = (unsigned char) *sptr;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/* If range includes any upper-case ASCII chars, make it include all */
if (rangelo <= 'Z' && rangehi >= 'A')
{
if (rangelo > 'A')
rangelo = 'A';
if (rangehi < 'Z')
rangehi = 'Z';
}
/* Ditto lower-case */
if (rangelo <= 'z' && rangehi >= 'a')
{
if (rangelo > 'a')
rangelo = 'a';
if (rangehi < 'z')
rangehi = 'z';
}
/* Ditto digits */
if (rangelo <= '9' && rangehi >= '0')
{
if (rangelo > '0')
rangelo = '0';
if (rangehi < '9')
rangehi = '9';
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
* If range includes less than 10 chars, assume we have not got enough
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* data, and make it include regular ASCII set.
*/
if (rangehi - rangelo < 9)
{
rangelo = ' ';
rangehi = 127;
}
/*
* Now strip any common prefix of the three strings.
*/
while (*lobound)
{
if (*lobound != *hibound || *lobound != *value)
break;
lobound++, hibound++, value++;
}
/*
* Now we can do the conversions.
*/
*scaledvalue = convert_one_string_to_scalar(value, rangelo, rangehi);
*scaledlobound = convert_one_string_to_scalar(lobound, rangelo, rangehi);
*scaledhibound = convert_one_string_to_scalar(hibound, rangelo, rangehi);
}
static double
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
convert_one_string_to_scalar(char *value, int rangelo, int rangehi)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
int slen = strlen(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
double num,
denom,
base;
if (slen <= 0)
return 0.0; /* empty string has scalar value 0 */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Since base is at least 10, need not consider more than about 20 chars
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (slen > 20)
slen = 20;
/* Convert initial characters to fraction */
base = rangehi - rangelo + 1;
num = 0.0;
denom = base;
while (slen-- > 0)
{
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
int ch = (unsigned char) *value++;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (ch < rangelo)
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
ch = rangelo - 1;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
else if (ch > rangehi)
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
ch = rangehi + 1;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
num += ((double) (ch - rangelo)) / denom;
denom *= base;
}
return num;
}
/*
* Convert a string-type Datum into a palloc'd, null-terminated string.
*
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
* When using a non-C locale, we must pass the string through strxfrm()
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* before continuing, so as to generate correct locale-specific results.
*/
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
static char *
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
convert_string_datum(Datum value, Oid typid)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
char *val;
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
switch (typid)
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
{
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case CHAROID:
val = (char *) palloc(2);
val[0] = DatumGetChar(value);
val[1] = '\0';
break;
case BPCHAROID:
case VARCHAROID:
case TEXTOID:
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
val = TextDatumGetCString(value);
break;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case NAMEOID:
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
{
NameData *nm = (NameData *) DatumGetPointer(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
val = pstrdup(NameStr(*nm));
break;
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
default:
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Can't get here unless someone tries to use scalarltsel on an
* operator with one string and one non-string operand.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unsupported type: %u", typid);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return NULL;
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
}
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
if (!lc_collate_is_c(DEFAULT_COLLATION_OID))
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
{
Work around buggy strxfrm() present in some Solaris releases.
2003-07-17 22:52:36 +02:00
char *xfrmstr;
size_t xfrmlen;
size_t xfrmlen2;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Note: originally we guessed at a suitable output buffer size, and
* only needed to call strxfrm twice if our guess was too small.
* However, it seems that some versions of Solaris have buggy strxfrm
* that can write past the specified buffer length in that scenario.
* So, do it the dumb way for portability.
Oh, for crying in a bucket ... relax Assert so that glibc's strxfrm does not dump core.
2003-07-18 00:20:14 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Yet other systems (e.g., glibc) sometimes return a smaller value
* from the second call than the first; thus the Assert must be <= not
* == as you'd expect. Can't any of these people program their way
* out of a paper bag?
Check return code from strxfrm on Windows since it has a non-standard way of indicating errors, so we don't try to allocate INT_MAX bytes to store a result in.
2007-05-05 19:05:48 +02:00
*
* XXX: strxfrm doesn't support UTF-8 encoding on Win32, it can return
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* bogus data or set an error. This is not really a problem unless it
* crashes since it will only give an estimation error and nothing
* fatal.
Work around buggy strxfrm() present in some Solaris releases.
2003-07-17 22:52:36 +02:00
*/
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
#if _MSC_VER == 1400 /* VS.Net 2005 */
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
*
* http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* FeedbackID=99694 */
Work around bug in strxfmt() but in MS VS2005. William ZHANG
2006-07-26 19:17:28 +02:00
{
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
char x[1];
Work around bug in strxfmt() but in MS VS2005. William ZHANG
2006-07-26 19:17:28 +02:00
xfrmlen = strxfrm(x, val, 0);
}
#else
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
xfrmlen = strxfrm(NULL, val, 0);
Check return code from strxfrm on Windows since it has a non-standard way of indicating errors, so we don't try to allocate INT_MAX bytes to store a result in.
2007-05-05 19:05:48 +02:00
#endif
#ifdef WIN32
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Check return code from strxfrm on Windows since it has a non-standard way of indicating errors, so we don't try to allocate INT_MAX bytes to store a result in.
2007-05-05 19:05:48 +02:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* On Windows, strxfrm returns INT_MAX when an error occurs. Instead
* of trying to allocate this much memory (and fail), just return the
Check return code from strxfrm on Windows since it has a non-standard way of indicating errors, so we don't try to allocate INT_MAX bytes to store a result in.
2007-05-05 19:05:48 +02:00
* original string unmodified as if we were in the C locale.
*/
if (xfrmlen == INT_MAX)
return val;
Work around bug in strxfmt() but in MS VS2005. William ZHANG
2006-07-26 19:17:28 +02:00
#endif
Work around buggy strxfrm() present in some Solaris releases.
2003-07-17 22:52:36 +02:00
xfrmstr = (char *) palloc(xfrmlen + 1);
xfrmlen2 = strxfrm(xfrmstr, val, xfrmlen + 1);
Oh, for crying in a bucket ... relax Assert so that glibc's strxfrm does not dump core.
2003-07-18 00:20:14 +02:00
Assert(xfrmlen2 <= xfrmlen);
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
pfree(val);
val = xfrmstr;
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
}
Suppress signed-vs-unsigned-char warnings.
2005-09-24 19:53:28 +02:00
return val;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
/*
* Do convert_to_scalar()'s work for any bytea data type.
*
* Very similar to convert_string_to_scalar except we can't assume
* null-termination and therefore pass explicit lengths around.
*
* Also, assumptions about likely "normal" ranges of characters have been
* removed - a data range of 0..255 is always used, for now. (Perhaps
* someday we will add information about actual byte data range to
* pg_statistic.)
*/
static void
convert_bytea_to_scalar(Datum value,
double *scaledvalue,
Datum lobound,
double *scaledlobound,
Datum hibound,
double *scaledhibound)
{
int rangelo,
rangehi,
valuelen = VARSIZE(DatumGetPointer(value)) - VARHDRSZ,
loboundlen = VARSIZE(DatumGetPointer(lobound)) - VARHDRSZ,
hiboundlen = VARSIZE(DatumGetPointer(hibound)) - VARHDRSZ,
i,
minlen;
unsigned char *valstr = (unsigned char *) VARDATA(DatumGetPointer(value)),
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
*lostr = (unsigned char *) VARDATA(DatumGetPointer(lobound)),
*histr = (unsigned char *) VARDATA(DatumGetPointer(hibound));
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
/*
* Assume bytea data is uniformly distributed across all byte values.
*/
rangelo = 0;
rangehi = 255;
/*
* Now strip any common prefix of the three strings.
*/
minlen = Min(Min(valuelen, loboundlen), hiboundlen);
for (i = 0; i < minlen; i++)
{
if (*lostr != *histr || *lostr != *valstr)
break;
lostr++, histr++, valstr++;
loboundlen--, hiboundlen--, valuelen--;
}
/*
* Now we can do the conversions.
*/
*scaledvalue = convert_one_bytea_to_scalar(valstr, valuelen, rangelo, rangehi);
*scaledlobound = convert_one_bytea_to_scalar(lostr, loboundlen, rangelo, rangehi);
*scaledhibound = convert_one_bytea_to_scalar(histr, hiboundlen, rangelo, rangehi);
}
static double
convert_one_bytea_to_scalar(unsigned char *value, int valuelen,
int rangelo, int rangehi)
{
double num,
denom,
base;
if (valuelen <= 0)
return 0.0; /* empty string has scalar value 0 */
/*
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
* Since base is 256, need not consider more than about 10 chars (even
* this many seems like overkill)
Add comparison operators and btree indexing support for type bytea. From Joe Conway.
2001-08-13 20:45:36 +02:00
*/
if (valuelen > 10)
valuelen = 10;
/* Convert initial characters to fraction */
base = rangehi - rangelo + 1;
num = 0.0;
denom = base;
while (valuelen-- > 0)
{
int ch = *value++;
if (ch < rangelo)
ch = rangelo - 1;
else if (ch > rangehi)
ch = rangehi + 1;
num += ((double) (ch - rangelo)) / denom;
denom *= base;
}
return num;
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* Do convert_to_scalar()'s work for any timevalue data type.
*/
static double
convert_timevalue_to_scalar(Datum value, Oid typid)
{
switch (typid)
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
{
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
case TIMESTAMPOID:
Another round of updates for new fmgr, mostly in the datetime code.
2000-06-09 03:11:16 +02:00
return DatumGetTimestamp(value);
Add missing TIMESTAMPTZ case.
2001-10-03 20:25:59 +02:00
case TIMESTAMPTZOID:
return DatumGetTimestampTz(value);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case ABSTIMEOID:
Another round of updates for new fmgr, mostly in the datetime code.
2000-06-09 03:11:16 +02:00
return DatumGetTimestamp(DirectFunctionCall1(abstime_timestamp,
value));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case DATEOID:
Avoid unexpected conversion overflow in planner for distant date values. The "date" type supports a wider range of dates than int64 timestamps do. However, there is pre-int64-timestamp code in the planner that assumes that all date values can be converted to timestamp with impunity. Fortunately, what we really need out of the conversion is always a double (float8) value; so even when the date is out of timestamp's range it's possible to produce a sane answer. All we need is a code path that doesn't try to force the result into int64. Per trouble report from David Rericha. Back-patch to all supported versions. Although this is surely a corner case, there's not much point in advertising a date range wider than timestamp's if we will choke on such values in unexpected places.
2010-12-29 04:49:57 +01:00
return date2timestamp_no_overflow(DatumGetDateADT(value));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case INTERVALOID:
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
{
Interval *interval = DatumGetIntervalP(value);
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Convert the month part of Interval to days using assumed
* average month length of 365.25/12.0 days. Not too
* accurate, but plenty good enough for our purposes.
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
*/
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#ifdef HAVE_INT64_TIMESTAMP
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
return interval->time + interval->day * (double) USECS_PER_DAY +
interval->month * ((DAYS_PER_YEAR / (double) MONTHS_PER_YEAR) * USECS_PER_DAY);
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#else
Add 'day' field to INTERVAL so 1 day interval can be distinguished from 24 hours. This is very helpful for daylight savings time: select '2005-05-03 00:00:00 EST'::timestamp with time zone + '24 hours'; ?column? ---------------------- 2005-05-04 01:00:00-04 select '2005-05-03 00:00:00 EST'::timestamp with time zone + '1 day'; ?column? ---------------------- 2005-05-04 01:00:00-04 Michael Glaesemann
2005-07-20 18:42:32 +02:00
return interval->time + interval->day * SECS_PER_DAY +
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
interval->month * ((DAYS_PER_YEAR / (double) MONTHS_PER_YEAR) * (double) SECS_PER_DAY);
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#endif
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case RELTIMEOID:
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#ifdef HAVE_INT64_TIMESTAMP
return (DatumGetRelativeTime(value) * 1000000.0);
#else
Another round of updates for new fmgr, mostly in the datetime code.
2000-06-09 03:11:16 +02:00
return DatumGetRelativeTime(value);
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#endif
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case TINTERVALOID:
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
{
Remove unnecessary parentheses in assignments. Add spaces where needed. Reference time interval variables as tinterval.
2005-07-21 06:41:43 +02:00
TimeInterval tinterval = DatumGetTimeInterval(value);
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#ifdef HAVE_INT64_TIMESTAMP
Remove unnecessary parentheses in assignments. Add spaces where needed. Reference time interval variables as tinterval.
2005-07-21 06:41:43 +02:00
if (tinterval->status != 0)
return ((tinterval->data[1] - tinterval->data[0]) * 1000000.0);
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#else
Remove unnecessary parentheses in assignments. Add spaces where needed. Reference time interval variables as tinterval.
2005-07-21 06:41:43 +02:00
if (tinterval->status != 0)
return tinterval->data[1] - tinterval->data[0];
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#endif
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
return 0; /* for lack of a better idea */
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
case TIMEOID:
Another round of updates for new fmgr, mostly in the datetime code.
2000-06-09 03:11:16 +02:00
return DatumGetTimeADT(value);
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
case TIMETZOID:
{
TimeTzADT *timetz = DatumGetTimeTzADTP(value);
/* use GMT-equivalent time */
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#ifdef HAVE_INT64_TIMESTAMP
return (double) (timetz->time + (timetz->zone * 1000000.0));
#else
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
return (double) (timetz->time + timetz->zone);
Support alternate storage scheme of 64-bit integer for date/time types. Use "--enable-integer-datetimes" in configuration to use this rather than the original float8 storage. I would recommend the integer-based storage for any platform on which it is available. We perhaps should make this the default for the production release. Change timezone(timestamptz) results to return timestamp rather than a character string. Formerly, we didn't have a way to represent timestamps with an explicit time zone other than freezing the info into a string. Now, we can reasonably omit the explicit time zone from the result and return a timestamp with values appropriate for the specified time zone. Much cleaner, and if you need the time zone in the result you can put it into a character string pretty easily anyway. Allow fractional seconds in date/time types even for dates prior to 1BC. Limit timestamp data types to 6 decimal places of precision. Just right for a micro-second storage of int8 date/time types, and reduces the number of places ad-hoc rounding was occuring for the float8-based types. Use lookup tables for precision/rounding calculations for timestamp and interval types. Formerly used pow() to calculate the desired value but with a more limited range there is no reason to not type in a lookup table. Should be *much* better performance, though formerly there were some optimizations to help minimize the number of times pow() was called. Define a HAVE_INT64_TIMESTAMP variable. Based on the configure option "--enable-integer-datetimes" and the existing internal INT64_IS_BUSTED. Add explicit date/interval operators and functions for addition and subtraction. Formerly relied on implicit type promotion from date to timestamp with time zone. Change timezone conversion functions for the timetz type from "timetz()" to "timezone()". This is consistant with other time zone coersion functions for other types. Bump the catalog version to 200204201. Fix up regression tests to reflect changes in fractional seconds representation for date/times in BC eras. All regression tests pass on my Linux box.
2002-04-21 21:52:18 +02:00
#endif
Teach convert_to_scalar about datatypes timetz, inet, cidr, macaddr.
2001-06-10 00:16:18 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Can't get here unless someone tries to use scalarltsel/scalargtsel on
* an operator with one timevalue and one non-timevalue operand.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unsupported type: %u", typid);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return 0;
Improve selectivity estimation involving string constants: pay attention to more than one character, and try to do the right thing in non-ASCII locales.
2000-03-23 01:55:42 +01:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* get_restriction_variable
* Examine the args of a restriction clause to see if it's of the
* form (variable op pseudoconstant) or (pseudoconstant op variable),
* where "variable" could be either a Var or an expression in vars of a
* single relation. If so, extract information about the variable,
* and also indicate which side it was on and the other argument.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* Inputs:
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
* root: the planner info
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* args: clause argument list
* varRelid: see specs for restriction selectivity functions
Guard against roundoff errors in new selectivity-estimation code, per bug report from Laurette Cisneros.
2002-01-03 05:02:34 +01:00
*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* Outputs: (these are valid only if TRUE is returned)
* *vardata: gets information about variable (see examine_variable)
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
* *other: gets other clause argument, aggressively reduced to a constant
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* *varonleft: set TRUE if variable is on the left, FALSE if on the right
*
* Returns TRUE if a variable is identified, otherwise FALSE.
*
* Note: if there are Vars on both sides of the clause, we must fail, because
* callers are expecting that the other side will act like a pseudoconstant.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Move ltree parentsel() selectivity function into /contrib/ltree.
2006-04-27 00:33:36 +02:00
bool
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
get_restriction_variable(PlannerInfo *root, List *args, int varRelid,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData *vardata, Node **other,
bool *varonleft)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Node *left,
*right;
VariableStatData rdata;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Fail if not a binary opclause (probably shouldn't happen) */
Use the new List API function names throughout the backend, and disable the list compatibility API by default. While doing this, I decided to keep the llast() macro around and introduce llast_int() and llast_oid() variants.
2004-05-31 01:40:41 +02:00
if (list_length(args) != 2)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return false;
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
left = (Node *) linitial(args);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
right = (Node *) lsecond(args);
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Examine both sides. Note that when varRelid is nonzero, Vars of other
* relations will be treated as pseudoconstants.
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
examine_variable(root, left, varRelid, vardata);
examine_variable(root, right, varRelid, &rdata);
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
/*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* If one side is a variable and the other not, we win.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (vardata->rel && rdata.rel == NULL)
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*varonleft = true;
Get rid of some old and crufty global variables in the planner. When this code was last gone over, there wasn't really any alternative to globals because we didn't have the PlannerInfo struct being passed all through the planner code. Now that we do, we can restructure things to avoid non-reentrancy. I'm fooling with this because otherwise I'd have had to add another global variable for the planned compact range table list.
2007-02-19 08:03:34 +01:00
*other = estimate_expression_value(root, rdata.var);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Assume we need no ReleaseVariableStats(rdata) here */
return true;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (vardata->rel == NULL && rdata.rel)
Rewrite of planner statistics-gathering code. ANALYZE is now available as a separate statement (though it can still be invoked as part of VACUUM, too). pg_statistic redesigned to be more flexible about what statistics are stored. ANALYZE now collects a list of several of the most common values, not just one, plus a histogram (not just the min and max values). Random sampling is used to make the process reasonably fast even on very large tables. The number of values and histogram bins collected is now user-settable via an ALTER TABLE command. There is more still to do; the new stats are not being used everywhere they could be in the planner. But the remaining changes for this project should be localized, and the behavior is already better than before. A not-very-related change is that sorting now makes use of btree comparison routines if it can find one, rather than invoking '<' twice.
2001-05-07 02:43:27 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*varonleft = false;
Get rid of some old and crufty global variables in the planner. When this code was last gone over, there wasn't really any alternative to globals because we didn't have the PlannerInfo struct being passed all through the planner code. Now that we do, we can restructure things to avoid non-reentrancy. I'm fooling with this because otherwise I'd have had to add another global variable for the planned compact range table list.
2007-02-19 08:03:34 +01:00
*other = estimate_expression_value(root, vardata->var);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Assume we need no ReleaseVariableStats(*vardata) here */
*vardata = rdata;
return true;
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
Change SearchSysCache coding conventions so that a reference count is maintained for each cache entry. A cache entry will not be freed until the matching ReleaseSysCache call has been executed. This eliminates worries about cache entries getting dropped while still in use. See my posting to pg-hackers of even date for more info.
2000-11-16 23:30:52 +01:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Ooops, clause has wrong structure (probably var op var) */
ReleaseVariableStats(*vardata);
ReleaseVariableStats(rdata);
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return false;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
/*
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* get_join_variables
* Apply examine_variable() to each side of a join clause.
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
* Also, attempt to identify whether the join clause has the same
* or reversed sense compared to the SpecialJoinInfo.
*
* We consider the join clause "normal" if it is "lhs_var OP rhs_var",
* or "reversed" if it is "rhs_var OP lhs_var". In complicated cases
* where we can't tell for sure, we default to assuming it's normal.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too.
2006-04-27 02:46:59 +02:00
void
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
get_join_variables(PlannerInfo *root, List *args, SpecialJoinInfo *sjinfo,
VariableStatData *vardata1, VariableStatData *vardata2,
bool *join_is_reversed)
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Node *left,
*right;
Use the new List API function names throughout the backend, and disable the list compatibility API by default. While doing this, I decided to keep the llast() macro around and introduce llast_int() and llast_oid() variants.
2004-05-31 01:40:41 +02:00
if (list_length(args) != 2)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
elog(ERROR, "join operator should take two arguments");
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
left = (Node *) linitial(args);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
right = (Node *) lsecond(args);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
examine_variable(root, left, 0, vardata1);
examine_variable(root, right, 0, vardata2);
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
if (vardata1->rel &&
bms_is_subset(vardata1->rel->relids, sjinfo->syn_righthand))
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
*join_is_reversed = true; /* var1 is on RHS */
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
else if (vardata2->rel &&
bms_is_subset(vardata2->rel->relids, sjinfo->syn_lefthand))
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
*join_is_reversed = true; /* var2 is on LHS */
Clean up the loose ends in selectivity estimation left by my patch for semi and anti joins. To do this, pass the SpecialJoinInfo struct for the current join as an additional optional argument to operator join selectivity estimation functions. This allows the estimator to tell not only what kind of join is being formed, but which variable is on which side of the join; a requirement long recognized but not dealt with till now. This also leaves the door open for future improvements in the estimators, such as accounting for the null-insertion effects of lower outer joins. I didn't do anything about that in the current patch but the information is in principle deducible from what's passed. The patch also clarifies the definition of join selectivity for semi/anti joins: it's the fraction of the left input that has (at least one) match in the right input. This allows getting rid of some very fuzzy thinking that I had committed in the original 7.4-era IN-optimization patch. There's probably room to estimate this better than the present patch does, but at least we know what to estimate. Since I had to touch CREATE OPERATOR anyway to allow a variant signature for join estimator functions, I took the opportunity to add a couple of additional checks that were missing, per my recent message to -hackers: * Check that estimator functions return float8; * Require execute permission at the time of CREATE OPERATOR on the operator's function as well as the estimator functions; * Require ownership of any pre-existing operator that's modified by the command. I also moved the lookup of the functions out of OperatorCreate() and into operatorcmds.c, since that seemed more consistent with most of the other catalog object creation processes, eg CREATE TYPE.
2008-08-16 02:01:38 +02:00
else
*join_is_reversed = false;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
/*
* examine_variable
* Try to look up statistical data about an expression.
* Fill in a VariableStatData struct to describe the expression.
*
* Inputs:
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
* root: the planner info
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* node: the expression tree to examine
* varRelid: see specs for restriction selectivity functions
*
* Outputs: *vardata is filled as follows:
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
* var: the input expression (with any binary relabeling stripped, if
* it is or contains a variable; but otherwise the type is preserved)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* rel: RelOptInfo for relation containing variable; NULL if expression
* contains no Vars (NOTE this could point to a RelOptInfo of a
* subquery, not one in the current query).
* statsTuple: the pg_statistic entry for the variable, if one exists;
* otherwise NULL.
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
* freefunc: pointer to a function to release statsTuple with.
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
* vartype: exposed type of the expression; this should always match
* the declared input type of the operator we are estimating for.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* atttype, atttypmod: type data to pass to get_attstatsslot(). This is
* commonly the same as the exposed type of the variable argument,
* but can be different in binary-compatible-type cases.
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
* isunique: TRUE if we were able to match the var to a unique index,
Don't trust deferred-unique indexes for join removal. The uniqueness condition might fail to hold intra-transaction, and assuming it does can give incorrect query results. Per report from Marti Raudsepp, though this is not his proposed patch. Back-patch to 9.0, where both these features were introduced. In the released branches, add the new IndexOptInfo field to the end of the struct, to try to minimize ABI breakage for third-party code that may be examining that struct.
2011-10-23 06:43:39 +02:00
* implying its values are unique for this query. (Caution: this
* should be trusted for statistical purposes only, since we do not
* check indimmediate.)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*
* Caller is responsible for doing ReleaseVariableStats() before exiting.
*/
If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too.
2006-04-27 02:46:59 +02:00
void
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
examine_variable(PlannerInfo *root, Node *node, int varRelid,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
VariableStatData *vardata)
{
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
Node *basenode;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Relids varnos;
RelOptInfo *onerel;
/* Make sure we don't return dangling pointers in vardata */
MemSet(vardata, 0, sizeof(VariableStatData));
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
/* Save the exposed type of the expression */
vardata->vartype = exprType(node);
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
/* Look inside any binary-compatible relabeling */
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (IsA(node, RelabelType))
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
basenode = (Node *) ((RelabelType *) node)->arg;
else
basenode = node;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Fast path for a simple Var */
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
if (IsA(basenode, Var) &&
(varRelid == 0 || varRelid == ((Var *) basenode)->varno))
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
Var *var = (Var *) basenode;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
/* Set up result fields other than the stats tuple */
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
vardata->var = basenode; /* return Var without relabeling */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->rel = find_base_rel(root, var->varno);
vardata->atttype = var->vartype;
vardata->atttypmod = var->vartypmod;
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
vardata->isunique = has_unique_index(vardata->rel, var->varattno);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
/* Try to locate some stats */
examine_simple_variable(root, var, vardata);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* Okay, it's a more complicated expression. Determine variable
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* membership. Note that when varRelid isn't zero, only vars of that
* relation are considered "real" vars.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
varnos = pull_varnos(basenode);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
onerel = NULL;
switch (bms_membership(varnos))
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
case BMS_EMPTY_SET:
/* No Vars at all ... must be pseudo-constant clause */
break;
case BMS_SINGLETON:
if (varRelid == 0 || bms_is_member(varRelid, varnos))
{
onerel = find_base_rel(root,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(varRelid ? varRelid : bms_singleton_member(varnos)));
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->rel = onerel;
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
node = basenode; /* strip any relabeling */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
/* else treat it as a constant */
break;
case BMS_MULTIPLE:
if (varRelid == 0)
{
/* treat it as a variable of a join relation */
vardata->rel = find_join_rel(root, varnos);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
node = basenode; /* strip any relabeling */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
else if (bms_is_member(varRelid, varnos))
{
/* ignore the vars belonging to other relations */
vardata->rel = find_base_rel(root, varRelid);
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
node = basenode; /* strip any relabeling */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* note: no point in expressional-index search here */
}
/* else treat it as a constant */
break;
}
bms_free(varnos);
Fix a pair of related issues with estimation of inequalities that involve binary-compatible relabeling of one or both operands. examine_variable should avoid stripping RelabelType from non-variable expressions, so that they will continue to have the correct type; and convert_to_scalar should just use that type and ignore the other input type. This isn't perfect but it beats failing entirely. Per example from Michael Fuhr.
2005-03-26 21:55:39 +01:00
vardata->var = node;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->atttype = exprType(node);
vardata->atttypmod = exprTypmod(node);
if (onerel)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We have an expression in vars of a single relation. Try to match
* it to expressional index columns, in hopes of finding some
* statistics.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* XXX it's conceivable that there are multiple matches with different
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* index opfamilies; if so, we need to pick one that matches the
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* operator we are estimating for. FIXME later.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *ilist;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
foreach(ilist, onerel->indexlist)
{
IndexOptInfo *index = (IndexOptInfo *) lfirst(ilist);
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
ListCell *indexpr_item;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
int pos;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexpr_item = list_head(index->indexprs);
if (indexpr_item == NULL)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
continue; /* no expressions here... */
for (pos = 0; pos < index->ncolumns; pos++)
{
if (index->indexkeys[pos] == 0)
{
Node *indexkey;
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
if (indexpr_item == NULL)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
elog(ERROR, "too few entries in indexprs list");
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexkey = (Node *) lfirst(indexpr_item);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (indexkey && IsA(indexkey, RelabelType))
indexkey = (Node *) ((RelabelType *) indexkey)->arg;
if (equal(node, indexkey))
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Found a match ... is it a unique index? Tests here
* should match has_unique_index().
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (index->unique &&
index->ncolumns == 1 &&
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
(index->indpred == NIL || index->predOK))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
vardata->isunique = true;
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
/*
* Has it got stats? We only consider stats for
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* non-partial indexes, since partial indexes probably
* don't reflect whole-relation statistics; the above
* check for uniqueness is the only info we take from
* a partial index.
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
*
* An index stats hook, however, must make its own
* decisions about what to do with partial indexes.
*/
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
if (get_index_stats_hook &&
(*get_index_stats_hook) (root, index->indexoid,
pos + 1, vardata))
{
/*
* The hook took control of acquiring a stats
* tuple. If it did supply a tuple, it'd better
* have supplied a freefunc.
*/
if (HeapTupleIsValid(vardata->statsTuple) &&
!vardata->freefunc)
elog(ERROR, "no function provided to release variable stats with");
}
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
else if (index->indpred == NIL)
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
{
vardata->statsTuple =
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
SearchSysCache3(STATRELATTINH,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ObjectIdGetDatum(index->indexoid),
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
Int16GetDatum(pos + 1),
BoolGetDatum(false));
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
vardata->freefunc = ReleaseSysCache;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (vardata->statsTuple)
break;
}
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
indexpr_item = lnext(indexpr_item);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
}
if (vardata->statsTuple)
break;
}
}
}
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
/*
* examine_simple_variable
* Handle a simple Var for examine_variable
*
* This is split out as a subroutine so that we can recurse to deal with
* Vars referencing subqueries.
*
* We already filled in all the fields of *vardata except for the stats tuple.
*/
static void
examine_simple_variable(PlannerInfo *root, Var *var,
VariableStatData *vardata)
{
RangeTblEntry *rte = root->simple_rte_array[var->varno];
Assert(IsA(rte, RangeTblEntry));
if (get_relation_stats_hook &&
(*get_relation_stats_hook) (root, rte, var->varattno, vardata))
{
/*
* The hook took control of acquiring a stats tuple. If it did supply
* a tuple, it'd better have supplied a freefunc.
*/
if (HeapTupleIsValid(vardata->statsTuple) &&
!vardata->freefunc)
elog(ERROR, "no function provided to release variable stats with");
}
else if (rte->rtekind == RTE_RELATION)
{
/*
* Plain table or parent of an inheritance appendrel, so look up the
* column in pg_statistic
*/
vardata->statsTuple = SearchSysCache3(STATRELATTINH,
ObjectIdGetDatum(rte->relid),
Int16GetDatum(var->varattno),
BoolGetDatum(rte->inh));
vardata->freefunc = ReleaseSysCache;
}
else if (rte->rtekind == RTE_SUBQUERY && !rte->inh)
{
/*
* Plain subquery (not one that was converted to an appendrel).
*
* Punt if subquery uses set operations, GROUP BY, or DISTINCT --- any
* of these will mash underlying columns' stats beyond recognition.
* (Set ops are particularly nasty; if we forged ahead, we would
* return stats relevant to only the leftmost subselect...)
*/
Query *subquery = rte->subquery;
RelOptInfo *rel;
TargetEntry *ste;
if (subquery->setOperations ||
subquery->groupClause ||
subquery->distinctClause)
return;
Add a security_barrier option for views. When a view is marked as a security barrier, it will not be pulled up into the containing query, and no quals will be pushed down into it, so that no function or operator chosen by the user can be applied to rows not exposed by the view. Views not configured with this option cannot provide robust row-level security, but will perform far better. Patch by KaiGai Kohei; original problem report by Heikki Linnakangas (in October 2009!). Review (in earlier versions) by Noah Misch and others. Design advice by Tom Lane and myself. Further review and cleanup by me.
2011-12-22 22:15:57 +01:00
/*
* If the sub-query originated from a view with the security_barrier
* attribute, we treat it as a black-box from outside of the view.
* This is probably a harsher restriction than necessary; it's
* certainly OK for the selectivity estimator (which is a C function,
* and therefore omnipotent anyway) to look at the statistics. But
* many selectivity estimators will happily *invoke the operator
* function* to try to work out a good estimate - and that's not OK.
* So for now, we do this.
*/
if (rte->security_barrier)
return;
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
/*
* OK, fetch RelOptInfo for subquery. Note that we don't change the
* rel returned in vardata, since caller expects it to be a rel of the
* caller's query level. Because we might already be recursing, we
* can't use that rel pointer either, but have to look up the Var's
* rel afresh.
*/
rel = find_base_rel(root, var->varno);
/* Subquery should have been planned already */
Assert(rel->subroot && IsA(rel->subroot, PlannerInfo));
Fix recursion into previously planned sub-query in examine_simple_variable. This code was looking at the sub-Query tree as seen in the parent query's RangeTblEntry; but that's the pristine parser output, and what we need to look at is the tree as it stands at the completion of planning. Otherwise we might pick up a Var that references a subquery that got flattened and hence has no RelOptInfo in the subroot. Per report from Peter Geoghegan.
2011-09-30 00:12:34 +02:00
/*
* Switch our attention to the subquery as mangled by the planner.
* It was okay to look at the pre-planning version for the tests
* above, but now we need a Var that will refer to the subroot's
* live RelOptInfos. For instance, if any subquery pullup happened
* during planning, Vars in the targetlist might have gotten replaced,
* and we need to see the replacement expressions.
*/
subquery = rel->subroot->parse;
Assert(IsA(subquery, Query));
Dig down into sub-selects to look for column statistics. If a sub-select's output column is a simple Var, recursively look for statistics applying to that Var, and use them if available. The need for this was foreseen ages ago, but we didn't have enough infrastructure to do it with reasonable speed until just now. We punt and stick with default estimates if the subquery uses set operations, GROUP BY, or DISTINCT, since those operations would change the underlying column statistics (particularly, the relative frequencies of different values) beyond recognition. This means that the types of sub-selects for which this improvement applies are fairly limited, since most subqueries satisfying those restrictions would have gotten flattened into the parent query anyway. But it does help for some cases, such as subqueries with ORDER BY or LIMIT.
2011-09-04 21:13:46 +02:00
/* Get the subquery output expression referenced by the upper Var */
ste = get_tle_by_resno(subquery->targetList, var->varattno);
if (ste == NULL || ste->resjunk)
elog(ERROR, "subquery %s does not have attribute %d",
rte->eref->aliasname, var->varattno);
var = (Var *) ste->expr;
/* Can only handle a simple Var of subquery's query level */
if (var && IsA(var, Var) &&
var->varlevelsup == 0)
{
/*
* OK, recurse into the subquery. Note that the original setting
* of vardata->isunique (which will surely be false) is left
* unchanged in this situation. That's what we want, since even
* if the underlying column is unique, the subquery may have
* joined to other tables in a way that creates duplicates.
*/
examine_simple_variable(rel->subroot, var, vardata);
}
}
else
{
/*
* Otherwise, the Var comes from a FUNCTION, VALUES, or CTE RTE. (We
* won't see RTE_JOIN here because join alias Vars have already been
* flattened.) There's not much we can do with function outputs, but
* maybe someday try to be smarter about VALUES and/or CTEs.
*/
}
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* get_variable_numdistinct
* Estimate the number of distinct values of a variable.
*
* vardata: results of examine_variable
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
* *isdefault: set to TRUE if the result is a default rather than based on
* anything meaningful.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*
* NB: be careful to produce an integral result, since callers may compare
* the result to exact integer counts.
*/
If we're going to expose VariableStatData for contrib modules to use, then we should export a reasonable set of the supporting routines too.
2006-04-27 02:46:59 +02:00
double
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
get_variable_numdistinct(VariableStatData *vardata, bool *isdefault)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
double stadistinct;
double ntuples;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
*isdefault = false;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Determine the stadistinct value to use. There are cases where we can
* get an estimate even without a pg_statistic entry, or can get a better
* value than is in pg_statistic.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (HeapTupleIsValid(vardata->statsTuple))
{
/* Use the pg_statistic entry */
Form_pg_statistic stats;
stats = (Form_pg_statistic) GETSTRUCT(vardata->statsTuple);
stadistinct = stats->stadistinct;
}
Second try at making examine_variable and friends behave sanely in cases with binary-compatible relabeling. My first try was implicitly assuming that all operators scalarineqsel is used for have binary- compatible datatypes on both sides ... which is very wrong of course. Per report from Michael Fuhr.
2005-04-01 22:31:50 +02:00
else if (vardata->vartype == BOOLOID)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
/*
* Special-case boolean columns: presumably, two distinct values.
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Are there any other datatypes we should wire in special estimates
* for?
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
stadistinct = 2.0;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
else
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We don't keep statistics for system columns, but in some cases we
* can infer distinctness anyway.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (vardata->var && IsA(vardata->var, Var))
{
switch (((Var *) vardata->var)->varattno)
{
case ObjectIdAttributeNumber:
case SelfItemPointerAttributeNumber:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
stadistinct = -1.0; /* unique */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
break;
case TableOidAttributeNumber:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
stadistinct = 1.0; /* only 1 value */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
break;
default:
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
stadistinct = 0.0; /* means "unknown" */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
break;
}
}
else
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
stadistinct = 0.0; /* means "unknown" */
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* XXX consider using estimate_num_groups on expressions?
*/
}
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* If there is a unique index for the variable, assume it is unique no
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* matter what pg_statistic says; the statistics could be out of date, or
* we might have found a partial unique index that proves the var is
* unique for this query.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
Teach the planner to treat a partial unique index as proving a variable is unique for a particular query, if the index predicate is satisfied. This requires a bit of reordering of operations so that we check the predicates before doing any selectivity estimates, but shouldn't really cause any noticeable slowdown. Per a comment from Michal Politowski.
2009-02-15 21:16:21 +01:00
if (vardata->isunique)
stadistinct = -1.0;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* If we had an absolute estimate, use that.
*/
if (stadistinct > 0.0)
return stadistinct;
/*
* Otherwise we need to get the relation size; punt if not available.
*/
if (vardata->rel == NULL)
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
{
*isdefault = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return DEFAULT_NUM_DISTINCT;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
ntuples = vardata->rel->tuples;
if (ntuples <= 0.0)
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
{
*isdefault = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return DEFAULT_NUM_DISTINCT;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
* If we had a relative estimate, use that.
*/
if (stadistinct < 0.0)
return floor((-stadistinct * ntuples) + 0.5);
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* With no data, estimate ndistinct = ntuples if the table is small, else
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
* use default. We use DEFAULT_NUM_DISTINCT as the cutoff for "small"
* so that the behavior isn't discontinuous.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (ntuples < DEFAULT_NUM_DISTINCT)
return ntuples;
Change get_variable_numdistinct's API to flag default estimates explicitly. Formerly, callers tested for DEFAULT_NUM_DISTINCT, which had the problem that a perfectly solid estimate might be mistaken for a content-free default.
2011-09-04 21:41:49 +02:00
*isdefault = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
return DEFAULT_NUM_DISTINCT;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
/*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* get_variable_range
* Estimate the minimum and maximum value of the specified variable.
* If successful, store values in *min and *max, and return TRUE.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* If no data available, return FALSE.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* sortop is the "<" comparison operator to use. This should generally
* be "<" not ">", as only the former is likely to be found in pg_statistic.
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
*/
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
static bool
Make all comparisons done for/with statistics use the default collation. While this will give wrong answers when estimating selectivity for a comparison operator that's using a non-default collation, the estimation error probably won't be large; and anyway the former approach created estimation errors of its own by trying to use a histogram that might have been computed with some other collation. So we'll adopt this simplified approach for now and perhaps improve it sometime in the future. This patch incorporates changes from Andres Freund to make sure that selfuncs.c passes a valid collation OID to any datatype-specific function it calls, in case that function wants collation information. Said OID will now always be DEFAULT_COLLATION_OID, but at least we won't get errors.
2011-03-12 22:30:36 +01:00
get_variable_range(PlannerInfo *root, VariableStatData *vardata, Oid sortop,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
Datum *min, Datum *max)
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
Datum tmin = 0;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Datum tmax = 0;
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
bool have_data = false;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
int16 typLen;
bool typByVal;
Datum *values;
int nvalues;
int i;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* XXX It's very tempting to try to use the actual column min and max, if
* we can get them relatively-cheaply with an index probe. However, since
* this function is called many times during join planning, that could
* have unpleasant effects on planning speed. Need more investigation
* before enabling this.
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
*/
#ifdef NOT_USED
if (get_actual_variable_range(root, vardata, sortop, min, max))
return true;
#endif
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (!HeapTupleIsValid(vardata->statsTuple))
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* no stats available, so default result */
return false;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
get_typlenbyval(vardata->atttype, &typLen, &typByVal);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* If there is a histogram, grab the first and last values.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* If there is a histogram that is sorted with some other operator than
* the one we want, fail --- this suggests that there is data we can't
* use.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_HISTOGRAM, sortop,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
&values, &nvalues,
NULL, NULL))
{
if (nvalues > 0)
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
tmin = datumCopy(values[0], typByVal, typLen);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
tmax = datumCopy(values[nvalues - 1], typByVal, typLen);
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
have_data = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
free_attstatsslot(vardata->atttype, values, nvalues, NULL, 0);
}
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
else if (get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_HISTOGRAM, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
&values, &nvalues,
NULL, NULL))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
free_attstatsslot(vardata->atttype, values, nvalues, NULL, 0);
return false;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
/*
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
* If we have most-common-values info, look for extreme MCVs. This is
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* needed even if we also have a histogram, since the histogram excludes
* the MCVs. However, usually the MCVs will not be the extreme values, so
* avoid unnecessary data copying.
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*/
if (get_attstatsslot(vardata->statsTuple,
vardata->atttype, vardata->atttypmod,
STATISTIC_KIND_MCV, InvalidOid,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
&values, &nvalues,
NULL, NULL))
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
bool tmin_is_mcv = false;
bool tmax_is_mcv = false;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
FmgrInfo opproc;
fmgr_info(get_opcode(sortop), &opproc);
for (i = 0; i < nvalues; i++)
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
if (!have_data)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
tmin = tmax = values[i];
tmin_is_mcv = tmax_is_mcv = have_data = true;
continue;
}
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (DatumGetBool(FunctionCall2Coll(&opproc,
DEFAULT_COLLATION_OID,
values[i], tmin)))
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
{
tmin = values[i];
tmin_is_mcv = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (DatumGetBool(FunctionCall2Coll(&opproc,
DEFAULT_COLLATION_OID,
tmax, values[i])))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
tmax = values[i];
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
tmax_is_mcv = true;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
}
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
if (tmin_is_mcv)
tmin = datumCopy(tmin, typByVal, typLen);
if (tmax_is_mcv)
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
tmax = datumCopy(tmax, typByVal, typLen);
free_attstatsslot(vardata->atttype, values, nvalues, NULL, 0);
}
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
*min = tmin;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
*max = tmax;
Fix mergejoin cost estimation so that we consider the statistical ranges of the two join variables at both ends: not only trailing rows that need not be scanned because there cannot be a match on the other side, but initial rows that will be scanned without possibly having a match. This allows a more realistic estimate of startup cost to be made, per recent pgsql-performance discussion. In passing, fix a couple of bugs that had crept into mergejoinscansel: it was not quite up to speed for the task of estimating descending-order scans, which is a new requirement in 8.3.
2007-12-08 22:05:11 +01:00
return have_data;
First step in fixing selectivity-estimation code. eqsel and neqsel now behave as per my suggestions in pghackers a few days ago. selectivity for < > <= >= should work OK for integral types as well, but still need work for nonintegral types. Since these routines have never actually executed before :-(, this may result in some significant changes in the optimizer's choices of execution plans. Let me know if you see any serious misbehavior. CAUTION: THESE CHANGES REQUIRE INITDB. pg_statistic table has changed.
1999-08-01 06:54:25 +02:00
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/*
* get_actual_variable_range
* Attempt to identify the current *actual* minimum and/or maximum
* of the specified variable, by looking for a suitable btree index
* and fetching its low and/or high values.
* If successful, store values in *min and *max, and return TRUE.
* (Either pointer can be NULL if that endpoint isn't needed.)
* If no data available, return FALSE.
*
* sortop is the "<" comparison operator to use.
*/
static bool
get_actual_variable_range(PlannerInfo *root, VariableStatData *vardata,
Oid sortop,
Datum *min, Datum *max)
{
bool have_data = false;
RelOptInfo *rel = vardata->rel;
RangeTblEntry *rte;
ListCell *lc;
/* No hope if no relation or it doesn't have indexes */
if (rel == NULL || rel->indexlist == NIL)
return false;
/* If it has indexes it must be a plain relation */
rte = root->simple_rte_array[rel->relid];
Assert(rte->rtekind == RTE_RELATION);
/* Search through the indexes to see if any match our problem */
foreach(lc, rel->indexlist)
{
IndexOptInfo *index = (IndexOptInfo *) lfirst(lc);
ScanDirection indexscandir;
/* Ignore non-btree indexes */
if (index->relam != BTREE_AM_OID)
continue;
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Ignore partial indexes --- we only want stats that cover the entire
* relation.
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
*/
if (index->indpred != NIL)
continue;
/*
Fix bogus test for hypothetical indexes in get_actual_variable_range(). That function was supposing that indexoid == 0 for a hypothetical index, but that is not likely to be true in any non-toy implementation of an index adviser, since assigning a fake OID is the only way to know at EXPLAIN time which hypothetical index got selected. Fix by adding a flag to IndexOptInfo to mark hypothetical indexes. Back-patch to 9.0 where get_actual_variable_range() was added. Gurjeet Singh
2011-02-17 01:24:45 +01:00
* The index list might include hypothetical indexes inserted by a
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
* get_relation_info hook --- don't try to access them.
*/
Fix bogus test for hypothetical indexes in get_actual_variable_range(). That function was supposing that indexoid == 0 for a hypothetical index, but that is not likely to be true in any non-toy implementation of an index adviser, since assigning a fake OID is the only way to know at EXPLAIN time which hypothetical index got selected. Fix by adding a flag to IndexOptInfo to mark hypothetical indexes. Back-patch to 9.0 where get_actual_variable_range() was added. Gurjeet Singh
2011-02-17 01:24:45 +01:00
if (index->hypothetical)
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
continue;
/*
* The first index column must match the desired variable and sort
* operator --- but we can use a descending-order index.
*/
if (!match_index_to_operand(vardata->var, 0, index))
continue;
Simplify and speed up mapping of index opfamilies to pathkeys. Formerly we looked up the operators associated with each index (caching them in relcache) and then the planner looked up the btree opfamily containing such operators in order to build the btree-centric pathkey representation that describes the index's sort order. This is quite pointless for btree indexes: we might as well just use the index's opfamily information directly. That saves syscache lookup cycles during planning, and furthermore allows us to eliminate the relcache's caching of operators altogether, which may help in reducing backend startup time. I added code to plancat.c to perform the same type of double lookup on-the-fly if it's ever faced with a non-btree amcanorder index AM. If such a thing actually becomes interesting for production, we should replace that logic with some more-direct method for identifying the corresponding btree opfamily; but it's not worth spending effort on now. There is considerably more to do pursuant to my recent proposal to get rid of sort-operator-based representations of sort orderings, but this patch grabs some of the low-hanging fruit. I'll look at the remainder of that work after the current commitfest.
2010-11-29 18:29:42 +01:00
switch (get_op_opfamily_strategy(sortop, index->sortopfamily[0]))
{
case BTLessStrategyNumber:
if (index->reverse_sort[0])
indexscandir = BackwardScanDirection;
else
indexscandir = ForwardScanDirection;
break;
case BTGreaterStrategyNumber:
if (index->reverse_sort[0])
indexscandir = ForwardScanDirection;
else
indexscandir = BackwardScanDirection;
break;
default:
/* index doesn't match the sortop */
continue;
}
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Found a suitable index to extract data from. We'll need an EState
* and a bunch of other infrastructure.
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
*/
{
EState *estate;
ExprContext *econtext;
MemoryContext tmpcontext;
MemoryContext oldcontext;
Relation heapRel;
Relation indexRel;
IndexInfo *indexInfo;
TupleTableSlot *slot;
int16 typLen;
bool typByVal;
ScanKeyData scankeys[1];
IndexScanDesc index_scan;
HeapTuple tup;
Datum values[INDEX_MAX_KEYS];
bool isnull[INDEX_MAX_KEYS];
estate = CreateExecutorState();
econtext = GetPerTupleExprContext(estate);
/* Make sure any cruft is generated in the econtext's memory */
tmpcontext = econtext->ecxt_per_tuple_memory;
oldcontext = MemoryContextSwitchTo(tmpcontext);
/*
* Open the table and index so we can read from them. We should
* already have at least AccessShareLock on the table, but not
* necessarily on the index.
*/
heapRel = heap_open(rte->relid, NoLock);
indexRel = index_open(index->indexoid, AccessShareLock);
/* extract index key information from the index's pg_index info */
indexInfo = BuildIndexInfo(indexRel);
/* some other stuff */
slot = MakeSingleTupleTableSlot(RelationGetDescr(heapRel));
econtext->ecxt_scantuple = slot;
get_typlenbyval(vardata->atttype, &typLen, &typByVal);
/* set up an IS NOT NULL scan key so that we ignore nulls */
ScanKeyEntryInitialize(&scankeys[0],
SK_ISNULL | SK_SEARCHNOTNULL,
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
1, /* index col to scan */
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
InvalidStrategy, /* no strategy */
InvalidOid, /* no strategy subtype */
Clean up cruft around collation initialization for tupdescs and scankeys. I found actual bugs in GiST and plpgsql; the rest of this is cosmetic but meant to decrease the odds of future bugs of omission.
2011-03-26 23:28:40 +01:00
InvalidOid, /* no collation */
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
InvalidOid, /* no reg proc for this */
(Datum) 0); /* constant */
have_data = true;
/* If min is requested ... */
if (min)
{
index_scan = index_beginscan(heapRel, indexRel, SnapshotNow,
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
1, 0);
index_rescan(index_scan, scankeys, 1, NULL, 0);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/* Fetch first tuple in sortop's direction */
if ((tup = index_getnext(index_scan,
indexscandir)) != NULL)
{
/* Extract the index column values from the heap tuple */
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
ExecStoreTuple(tup, slot, InvalidBuffer, false);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
FormIndexDatum(indexInfo, slot, estate,
values, isnull);
/* Shouldn't have got a null, but be careful */
if (isnull[0])
elog(ERROR, "found unexpected null value in index \"%s\"",
RelationGetRelationName(indexRel));
/* Copy the index column value out to caller's context */
MemoryContextSwitchTo(oldcontext);
*min = datumCopy(values[0], typByVal, typLen);
MemoryContextSwitchTo(tmpcontext);
}
else
have_data = false;
index_endscan(index_scan);
}
/* If max is requested, and we didn't find the index is empty */
if (max && have_data)
{
index_scan = index_beginscan(heapRel, indexRel, SnapshotNow,
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
1, 0);
index_rescan(index_scan, scankeys, 1, NULL, 0);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
/* Fetch first tuple in reverse direction */
if ((tup = index_getnext(index_scan,
-indexscandir)) != NULL)
{
/* Extract the index column values from the heap tuple */
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
ExecStoreTuple(tup, slot, InvalidBuffer, false);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
FormIndexDatum(indexInfo, slot, estate,
values, isnull);
/* Shouldn't have got a null, but be careful */
if (isnull[0])
elog(ERROR, "found unexpected null value in index \"%s\"",
RelationGetRelationName(indexRel));
/* Copy the index column value out to caller's context */
MemoryContextSwitchTo(oldcontext);
*max = datumCopy(values[0], typByVal, typLen);
MemoryContextSwitchTo(tmpcontext);
}
else
have_data = false;
index_endscan(index_scan);
}
/* Clean everything up */
ExecDropSingleTupleTableSlot(slot);
index_close(indexRel, AccessShareLock);
heap_close(heapRel, NoLock);
MemoryContextSwitchTo(oldcontext);
FreeExecutorState(estate);
/* And we're done */
break;
}
}
return have_data;
}
Improve eqjoinsel's ndistinct clamping to work for multiple levels of join. This patch fixes an oversight in my commit 7f3eba30c9d622d1981b1368f2d79ba0999cdff2 of 2008-10-23. That patch accounted for baserel restriction clauses that reduced the number of rows coming out of a table (and hence the number of possibly-distinct values of a join variable), but not for join restriction clauses that might have been applied at a lower level of join. To account for the latter, look up the sizes of the min_lefthand and min_righthand inputs of the current join, and clamp with those in the same way as for the base relations. Noted while investigating a complaint from Ben Chobot, although this in itself doesn't seem to explain his report. Back-patch to 8.4; previous versions used different estimation methods for which this heuristic isn't relevant.
2011-08-31 22:04:48 +02:00
/*
* find_join_input_rel
* Look up the input relation for a join.
*
* We assume that the input relation's RelOptInfo must have been constructed
* already.
*/
static RelOptInfo *
find_join_input_rel(PlannerInfo *root, Relids relids)
{
RelOptInfo *rel = NULL;
switch (bms_membership(relids))
{
case BMS_EMPTY_SET:
/* should not happen */
break;
case BMS_SINGLETON:
rel = find_base_rel(root, bms_singleton_member(relids));
break;
case BMS_MULTIPLE:
rel = find_join_rel(root, relids);
break;
}
if (rel == NULL)
elog(ERROR, "could not find RelOptInfo for given relids");
return rel;
}
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*-------------------------------------------------------------------------
*
* Pattern analysis functions
*
* These routines support analysis of LIKE and regular-expression patterns
* by the planner/optimizer. It's important that they agree with the
* regular-expression code in backend/regex/ and the LIKE code in
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
* backend/utils/adt/like.c. Also, the computation of the fixed prefix
* must be conservative: if we report a string longer than the true fixed
* prefix, the query may produce actually wrong answers, rather than just
* getting a bad selectivity estimate!
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*
* Note that the prefix-analysis functions are called from
* backend/optimizer/path/indxpath.c as well as from routines in this file.
*
*-------------------------------------------------------------------------
*/
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
/*
* Check whether char is a letter (and, hence, subject to case-folding)
*
* In multibyte character sets, we can't use isalpha, and it does not seem
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
* worth trying to convert to wchar_t to use iswalpha. Instead, just assume
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
* any multibyte char is potentially case-varying.
*/
static int
pattern_char_isalpha(char c, bool is_multibyte,
pg_locale_t locale, bool locale_is_c)
{
if (locale_is_c)
return (c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z');
else if (is_multibyte && IS_HIGHBIT_SET(c))
return true;
#ifdef HAVE_LOCALE_T
else if (locale)
return isalpha_l((unsigned char) c, locale);
#endif
else
return isalpha((unsigned char) c);
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* Extract the fixed prefix, if any, for a pattern.
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*
* *prefix is set to a palloc'd prefix string (in the form of a Const node),
* or to NULL if no fixed prefix exists for the pattern.
* *rest is set to a palloc'd Const representing the remainder of the pattern
* after the portion describing the fixed prefix.
* Each of these has the same type (TEXT or BYTEA) as the given pattern Const.
*
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
* The return value distinguishes no fixed prefix, a partial prefix,
* or an exact-match-only pattern.
*/
static Pattern_Prefix_Status
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
like_fixed_prefix(Const *patt_const, bool case_insensitive, Oid collation,
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const **prefix_const, Const **rest_const)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
char *match;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
char *patt;
int pattlen;
char *rest;
Oid typeid = patt_const->consttype;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
int pos,
match_pos;
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
bool is_multibyte = (pg_database_encoding_max_length() > 1);
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
pg_locale_t locale = 0;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
bool locale_is_c = false;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text or bytea */
Assert(typeid == BYTEAOID || typeid == TEXTOID);
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
if (case_insensitive)
{
if (typeid == BYTEAOID)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
errmsg("case insensitive matching not supported on type bytea")));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
/* If case-insensitive, we need locale info */
if (lc_ctype_is_c(collation))
locale_is_c = true;
else if (collation != DEFAULT_COLLATION_OID)
{
if (!OidIsValid(collation))
{
/*
* This typically means that the parser could not resolve a
* conflict of implicit collations, so report it that way.
*/
ereport(ERROR,
(errcode(ERRCODE_INDETERMINATE_COLLATION),
errmsg("could not determine which collation to use for ILIKE"),
errhint("Use the COLLATE clause to set the collation explicitly.")));
}
locale = pg_newlocale_from_collation(collation);
}
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (typeid != BYTEAOID)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
patt = TextDatumGetCString(patt_const->constvalue);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pattlen = strlen(patt);
}
else
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
bytea *bstr = DatumGetByteaP(patt_const->constvalue);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
pattlen = VARSIZE(bstr) - VARHDRSZ;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
patt = (char *) palloc(pattlen);
memcpy(patt, VARDATA(bstr), pattlen);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
if ((Pointer) bstr != DatumGetPointer(patt_const->constvalue))
pfree(bstr);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
pgindent run.
2002-09-04 22:31:48 +02:00
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
match = palloc(pattlen + 1);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
match_pos = 0;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
for (pos = 0; pos < pattlen; pos++)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
/* % and _ are wildcard characters in LIKE */
if (patt[pos] == '%' ||
patt[pos] == '_')
break;
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
/* Backslash escapes the next character */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (patt[pos] == '\\')
{
pos++;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
if (pos >= pattlen)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
/* Stop if case-varying character (it's sort of a wildcard) */
if (case_insensitive &&
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
pattern_char_isalpha(patt[pos], is_multibyte, locale, locale_is_c))
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
break;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
match[match_pos++] = patt[pos];
}
match[match_pos] = '\0';
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
rest = &patt[pos];
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
if (typeid != BYTEAOID)
{
*prefix_const = string_to_const(match, typeid);
*rest_const = string_to_const(rest, typeid);
}
else
{
*prefix_const = string_to_bytea_const(match, match_pos);
*rest_const = string_to_bytea_const(rest, pattlen - pos);
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
pfree(patt);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pfree(match);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/* in LIKE, an empty pattern is an exact match! */
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (pos == pattlen)
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
return Pattern_Prefix_Exact; /* reached end of pattern, so exact */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (match_pos > 0)
return Pattern_Prefix_Partial;
return Pattern_Prefix_None;
}
static Pattern_Prefix_Status
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
regex_fixed_prefix(Const *patt_const, bool case_insensitive, Oid collation,
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const **prefix_const, Const **rest_const)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
char *match;
int pos,
match_pos,
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
prev_pos,
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
prev_match_pos;
bool have_leading_paren;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
char *patt;
char *rest;
Oid typeid = patt_const->consttype;
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
bool is_multibyte = (pg_database_encoding_max_length() > 1);
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
pg_locale_t locale = 0;
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
bool locale_is_c = false;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Should be unnecessary, there are no bytea regex operators defined. As
* such, it should be noted that the rest of this function has *not* been
* made safe for binary (possibly NULL containing) strings.
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
*/
if (typeid == BYTEAOID)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("regular-expression matching not supported on type bytea")));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
if (case_insensitive)
{
/* If case-insensitive, we need locale info */
if (lc_ctype_is_c(collation))
locale_is_c = true;
else if (collation != DEFAULT_COLLATION_OID)
{
if (!OidIsValid(collation))
{
/*
* This typically means that the parser could not resolve a
* conflict of implicit collations, so report it that way.
*/
ereport(ERROR,
(errcode(ERRCODE_INDETERMINATE_COLLATION),
errmsg("could not determine which collation to use for regular expression"),
errhint("Use the COLLATE clause to set the collation explicitly.")));
}
locale = pg_newlocale_from_collation(collation);
}
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
patt = TextDatumGetCString(patt_const->constvalue);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
/*
* Check for ARE director prefix. It's worth our trouble to recognize
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* this because similar_escape() used to use it, and some other code might
* still use it, to force ARE mode.
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
*/
pos = 0;
if (strncmp(patt, "***:", 4) == 0)
pos = 4;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/* Pattern must be anchored left */
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
if (patt[pos] != '^')
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
rest = patt;
pgindent run.
2002-09-04 22:31:48 +02:00
*prefix_const = NULL;
*rest_const = string_to_const(rest, typeid);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return Pattern_Prefix_None;
}
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
pos++;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
/*
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
* If '|' is present in pattern, then there may be multiple alternatives
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* for the start of the string. (There are cases where this isn't so, for
* instance if the '|' is inside parens, but detecting that reliably is
* too hard.)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
if (strchr(patt + pos, '|') != NULL)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
rest = patt;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
*prefix_const = NULL;
*rest_const = string_to_const(rest, typeid);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
return Pattern_Prefix_None;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/* OK, allocate space for pattern */
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
match = palloc(strlen(patt) + 1);
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
prev_match_pos = match_pos = 0;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
/*
* We special-case the syntax '^(...)$' because psql uses it. But beware:
* sequences beginning "(?" are not what they seem, unless they're "(?:".
Remove regex_flavor GUC, so that regular expressions are always "advanced" style by default. Per discussion, there seems to be hardly anything that really relies on being able to change the regex flavor, so the ability to select it via embedded options ought to be enough for any stragglers. Also, if we didn't remove the GUC, we'd really be morally obligated to mark the regex functions non-immutable, which'd possibly create performance issues.
2009-10-21 22:38:58 +02:00
* (We must recognize that because of similar_escape().)
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
*/
have_leading_paren = false;
Remove regex_flavor GUC, so that regular expressions are always "advanced" style by default. Per discussion, there seems to be hardly anything that really relies on being able to change the regex flavor, so the ability to select it via embedded options ought to be enough for any stragglers. Also, if we didn't remove the GUC, we'd really be morally obligated to mark the regex functions non-immutable, which'd possibly create performance issues.
2009-10-21 22:38:58 +02:00
if (patt[pos] == '(' &&
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
(patt[pos + 1] != '?' || patt[pos + 2] == ':'))
{
have_leading_paren = true;
pos += (patt[pos + 1] != '?' ? 1 : 3);
}
/* Scan remainder of pattern */
prev_pos = pos;
while (patt[pos])
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
int len;
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Check for characters that indicate multiple possible matches here.
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
* Also, drop out at ')' or '$' so the termination test works right.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
if (patt[pos] == '.' ||
patt[pos] == '(' ||
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
patt[pos] == ')' ||
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
patt[pos] == '[' ||
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
patt[pos] == '^' ||
patt[pos] == '$')
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
/* Stop if case-varying character (it's sort of a wildcard) */
if (case_insensitive &&
Pgindent run before 9.1 beta2.
2011-06-09 20:32:50 +02:00
pattern_char_isalpha(patt[pos], is_multibyte, locale, locale_is_c))
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
break;
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
* Check for quantifiers. Except for +, this means the preceding
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* character is optional, so we must remove it from the prefix too!
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
if (patt[pos] == '*' ||
patt[pos] == '?' ||
Remove regex_flavor GUC, so that regular expressions are always "advanced" style by default. Per discussion, there seems to be hardly anything that really relies on being able to change the regex flavor, so the ability to select it via embedded options ought to be enough for any stragglers. Also, if we didn't remove the GUC, we'd really be morally obligated to mark the regex functions non-immutable, which'd possibly create performance issues.
2009-10-21 22:38:58 +02:00
patt[pos] == '{')
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
match_pos = prev_match_pos;
pos = prev_pos;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
}
if (patt[pos] == '+')
{
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
pos = prev_pos;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
}
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
/*
* Normally, backslash quotes the next character. But in AREs,
* backslash followed by alphanumeric is an escape, not a quoted
Remove regex_flavor GUC, so that regular expressions are always "advanced" style by default. Per discussion, there seems to be hardly anything that really relies on being able to change the regex flavor, so the ability to select it via embedded options ought to be enough for any stragglers. Also, if we didn't remove the GUC, we'd really be morally obligated to mark the regex functions non-immutable, which'd possibly create performance issues.
2009-10-21 22:38:58 +02:00
* character. Must treat it as having multiple possible matches.
* Note: since only ASCII alphanumerics are escapes, we don't have to
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
* be paranoid about multibyte or collations here.
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
*/
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (patt[pos] == '\\')
{
Remove regex_flavor GUC, so that regular expressions are always "advanced" style by default. Per discussion, there seems to be hardly anything that really relies on being able to change the regex flavor, so the ability to select it via embedded options ought to be enough for any stragglers. Also, if we didn't remove the GUC, we'd really be morally obligated to mark the regex functions non-immutable, which'd possibly create performance issues.
2009-10-21 22:38:58 +02:00
if (isalnum((unsigned char) patt[pos + 1]))
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
break;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
pos++;
if (patt[pos] == '\0')
break;
}
Teach regex_fixed_prefix() the correct handling of advanced regex escapes --- they aren't simply quoted characters. Problem noted by Antti Salmela. Also fix problem with incorrect handling of multibyte characters when followed by a quantifier.
2004-12-02 03:45:07 +01:00
/* save position in case we need to back up on next loop cycle */
prev_match_pos = match_pos;
prev_pos = pos;
/* must use encoding-aware processing here */
len = pg_mblen(&patt[pos]);
memcpy(&match[match_pos], &patt[pos], len);
match_pos += len;
pos += len;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
match[match_pos] = '\0';
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
rest = &patt[pos];
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Fix regex_fixed_prefix() to cope reasonably well with regex patterns of the form '^(foo)$'. Before, these could never be optimized into indexscans. The recent changes to make psql and pg_dump generate such patterns (for \d commands and -t and related switches, respectively) therefore represented a big performance hit for people with large pg_class catalogs, as seen in recent gripe from Erik Jones. While at it, be more paranoid about case-sensitivity checking in multibyte encodings, and fix some other corner cases in which a regex might be interpreted too liberally.
2007-01-03 23:39:26 +01:00
if (have_leading_paren && patt[pos] == ')')
pos++;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (patt[pos] == '$' && patt[pos + 1] == '\0')
{
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
rest = &patt[pos + 1];
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*prefix_const = string_to_const(match, typeid);
pgindent run.
2002-09-04 22:31:48 +02:00
*rest_const = string_to_const(rest, typeid);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
pfree(patt);
pfree(match);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return Pattern_Prefix_Exact; /* pattern specifies exact match */
}
Cause CHAR(n) to TEXT or VARCHAR conversion to automatically strip trailing blanks, in hopes of reducing the surprise factor for newbies. Remove redundant operators for VARCHAR (it depends wholly on TEXT operations now). Clean up resolution of ambiguous operators/functions to avoid surprising choices for domains: domains are treated as equivalent to their base types and binary-coercibility is no longer considered a preference item when choosing among multiple operators/functions. IsBinaryCoercible now correctly reflects the notion that you need *only* relabel the type to get from type A to type B: that is, a domain is binary-coercible to its base type, but not vice versa. Various marginal cleanup, including merging the essentially duplicate resolution code in parse_func.c and parse_oper.c. Improve opr_sanity regression test to understand about binary compatibility (using pg_cast), and fix a couple of small errors in the catalogs revealed thereby. Restructure "special operator" handling to fetch operators via index opclasses rather than hardwiring assumptions about names (cleans up the pattern_ops stuff a little).
2003-05-26 02:11:29 +02:00
*prefix_const = string_to_const(match, typeid);
pgindent run.
2002-09-04 22:31:48 +02:00
*rest_const = string_to_const(rest, typeid);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pfree(patt);
pfree(match);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (match_pos > 0)
return Pattern_Prefix_Partial;
return Pattern_Prefix_None;
}
Pattern_Prefix_Status
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
pattern_fixed_prefix(Const *patt, Pattern_Type ptype, Oid collation,
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const **prefix, Const **rest)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Pattern_Prefix_Status result;
switch (ptype)
{
case Pattern_Type_Like:
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
result = like_fixed_prefix(patt, false, collation, prefix, rest);
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
break;
case Pattern_Type_Like_IC:
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
result = like_fixed_prefix(patt, true, collation, prefix, rest);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
case Pattern_Type_Regex:
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
result = regex_fixed_prefix(patt, false, collation, prefix, rest);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
case Pattern_Type_Regex_IC:
Teach pattern_fixed_prefix() about collations. This is necessary, not optional, now that ILIKE and regexes are collation aware --- else we might derive a wrong comparison constant for index optimized pattern matches.
2011-04-11 18:28:28 +02:00
result = regex_fixed_prefix(patt, true, collation, prefix, rest);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized ptype: %d", (int) ptype);
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
result = Pattern_Prefix_None; /* keep compiler quiet */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
}
return result;
}
/*
* Estimate the selectivity of a fixed prefix for a pattern match.
*
* A fixed prefix "foo" is estimated as the selectivity of the expression
Remove support for OR'd indexscans internal to a single IndexScan plan node, as this behavior is now better done as a bitmap OR indexscan. This allows considerable simplification in nodeIndexscan.c itself as well as several planner modules concerned with indexscan plan generation. Also we can improve the sharing of code between regular and bitmap indexscans, since they are now working with nigh-identical Plan nodes.
2005-04-25 03:30:14 +02:00
* "variable >= 'foo' AND variable < 'fop'" (see also indxpath.c).
Store current LC_COLLATE and LC_CTYPE settings in pg_control during initdb; re-adopt these settings at every postmaster or standalone-backend startup. This should fix problems with indexes becoming corrupt due to failure to provide consistent locale environment for postmaster at all times. Also, refuse to start up a non-locale-enabled compilation in a database originally initdb'd with a non-C locale. Suppress LIKE index optimization if locale is not "C" or "POSIX" (are there any other locales where it's safe?). Issue NOTICE during initdb if selected locale disables LIKE optimization.
2000-11-25 21:33:54 +01:00
*
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
* The selectivity estimate is with respect to the portion of the column
* population represented by the histogram --- the caller must fold this
* together with info about MCVs and NULLs.
*
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
* We use the >= and < operators from the specified btree opfamily to do the
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
* estimation. The given variable and Const must be of the associated
* datatype.
Tweak selectivity and related routines to cope with domains. Per report from Andreas Pflug.
2003-03-23 02:49:02 +01:00
*
Store current LC_COLLATE and LC_CTYPE settings in pg_control during initdb; re-adopt these settings at every postmaster or standalone-backend startup. This should fix problems with indexes becoming corrupt due to failure to provide consistent locale environment for postmaster at all times. Also, refuse to start up a non-locale-enabled compilation in a database originally initdb'd with a non-C locale. Suppress LIKE index optimization if locale is not "C" or "POSIX" (are there any other locales where it's safe?). Issue NOTICE during initdb if selected locale disables LIKE optimization.
2000-11-25 21:33:54 +01:00
* XXX Note: we make use of the upper bound to estimate operator selectivity
* even if the locale is such that we cannot rely on the upper-bound string.
* The selectivity only needs to be approximately right anyway, so it seems
* more useful to use the upper-bound code than not.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
static Selectivity
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
prefix_selectivity(PlannerInfo *root, VariableStatData *vardata,
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
Oid vartype, Oid opfamily, Const *prefixcon)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Selectivity prefixsel;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Oid cmpopr;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
FmgrInfo opproc;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *greaterstrcon;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Selectivity eq_sel;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
cmpopr = get_opfamily_member(opfamily, vartype, vartype,
BTGreaterEqualStrategyNumber);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (cmpopr == InvalidOid)
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
elog(ERROR, "no >= operator for opfamily %u", opfamily);
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
fmgr_info(get_opcode(cmpopr), &opproc);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
prefixsel = ineq_histogram_selectivity(root, vardata, &opproc, true,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
prefixcon->constvalue,
prefixcon->consttype);
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
if (prefixsel < 0.0)
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
{
/* No histogram is present ... return a suitable default estimate */
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
return DEFAULT_MATCH_SEL;
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Fix comments that were mis-wrapped, for Tom Lane.
2001-03-23 05:49:58 +01:00
/*-------
* If we can create a string larger than the prefix, say
* "x < greaterstr".
*-------
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Revert thinko introduced into prefix_selectivity() by my recent patch: make_greater_string needs the < procedure not the >= one. Spotted by Peter.
2008-03-17 18:13:54 +01:00
cmpopr = get_opfamily_member(opfamily, vartype, vartype,
BTLessStrategyNumber);
if (cmpopr == InvalidOid)
elog(ERROR, "no < operator for opfamily %u", opfamily);
fmgr_info(get_opcode(cmpopr), &opproc);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
greaterstrcon = make_greater_string(prefixcon, &opproc,
DEFAULT_COLLATION_OID);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (greaterstrcon)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Selectivity topsel;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
topsel = ineq_histogram_selectivity(root, vardata, &opproc, false,
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
greaterstrcon->constvalue,
greaterstrcon->consttype);
/* ineq_histogram_selectivity worked before, it shouldn't fail now */
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
Assert(topsel >= 0.0);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Merge the two selectivities in the same way as for a range query
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* (see clauselist_selectivity()). Note that we don't need to worry
Improve patternsel() by applying the operator itself to each value listed in the column's most-common-values statistics entry. This gives us an exact selectivity result for the portion of the column population represented by the MCV list, which can be a big leg up in accuracy if that's a large fraction of the population. The heuristics involving pattern contents and prefix are applied only to the part of the population not included in the MCV list.
2006-01-10 18:35:52 +01:00
* about double-exclusion of nulls, since ineq_histogram_selectivity
* doesn't count those anyway.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
prefixsel = topsel + prefixsel - 1.0;
}
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
/*
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* If the prefix is long then the two bounding values might be too close
* together for the histogram to distinguish them usefully, resulting in a
* zero estimate (plus or minus roundoff error). To avoid returning a
* ridiculously small estimate, compute the estimated selectivity for
* "variable = 'foo'", and clamp to that. (Obviously, the resultant
* estimate should be at least that.)
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*
* We apply this even if we couldn't make a greater string. That case
* suggests that the prefix is near the maximum possible, and thus
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* probably off the end of the histogram, and thus we probably got a very
* small estimate from the >= condition; so we still need to clamp.
Modify prefix_selectivity() so that it will never estimate the selectivity of the generated range condition var >= 'foo' AND var < 'fop' as being less than what eqsel() would estimate for var = 'foo'. This is intuitively reasonable and it gets rid of the need for some entirely ad-hoc coding we formerly used to reject bogus estimates. The basic problem here is that if the prefix is more than a few characters long, the two boundary values are too close together to be distinguishable by comparison to the column histogram, resulting in a selectivity estimate of zero, which is often not very sane. Change motivated by an example from Peter Eisentraut. Arguably this is a bug fix, but I'll refrain from back-patching it for the moment.
2008-03-08 23:41:38 +01:00
*/
cmpopr = get_opfamily_member(opfamily, vartype, vartype,
BTEqualStrategyNumber);
if (cmpopr == InvalidOid)
elog(ERROR, "no = operator for opfamily %u", opfamily);
eq_sel = var_eq_const(vardata, cmpopr, prefixcon->constvalue,
false, true);
prefixsel = Max(prefixsel, eq_sel);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return prefixsel;
}
/*
* Estimate the selectivity of a pattern of the specified type.
* Note that any fixed prefix of the pattern will have been removed already.
*
* For now, we use a very simplistic approach: fixed characters reduce the
* selectivity a good deal, character ranges reduce it a little,
* wildcards (such as % for LIKE or .* for regex) increase it.
*/
TODO item done: * Change FIXED_CHAR_SEL to 0.20 from 0.04 to give better selectivity (Bruce)
2002-03-08 05:29:01 +01:00
#define FIXED_CHAR_SEL 0.20 /* about 1/5 */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
#define CHAR_RANGE_SEL 0.25
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
#define ANY_CHAR_SEL 0.9 /* not 1, since it won't match end-of-string */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
#define FULL_WILDCARD_SEL 5.0
#define PARTIAL_WILDCARD_SEL 2.0
static Selectivity
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
like_selectivity(Const *patt_const, bool case_insensitive)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Selectivity sel = 1.0;
int pos;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Oid typeid = patt_const->consttype;
char *patt;
int pattlen;
/* the right-hand const is type text or bytea */
Assert(typeid == BYTEAOID || typeid == TEXTOID);
if (typeid == BYTEAOID && case_insensitive)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("case insensitive matching not supported on type bytea")));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (typeid != BYTEAOID)
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
patt = TextDatumGetCString(patt_const->constvalue);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pattlen = strlen(patt);
}
else
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
bytea *bstr = DatumGetByteaP(patt_const->constvalue);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
pattlen = VARSIZE(bstr) - VARHDRSZ;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
patt = (char *) palloc(pattlen);
memcpy(patt, VARDATA(bstr), pattlen);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
if ((Pointer) bstr != DatumGetPointer(patt_const->constvalue))
pfree(bstr);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
/* Skip any leading wildcard; it's already factored into initial sel */
for (pos = 0; pos < pattlen; pos++)
{
if (patt[pos] != '%' && patt[pos] != '_')
break;
}
for (; pos < pattlen; pos++)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
/* % and _ are wildcard characters in LIKE */
if (patt[pos] == '%')
sel *= FULL_WILDCARD_SEL;
else if (patt[pos] == '_')
sel *= ANY_CHAR_SEL;
else if (patt[pos] == '\\')
{
/* Backslash quotes the next character */
pos++;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
if (pos >= pattlen)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
sel *= FIXED_CHAR_SEL;
}
else
sel *= FIXED_CHAR_SEL;
}
/* Could get sel > 1 if multiple wildcards */
if (sel > 1.0)
sel = 1.0;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
pfree(patt);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
return sel;
}
static Selectivity
regex_selectivity_sub(char *patt, int pattlen, bool case_insensitive)
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Selectivity sel = 1.0;
int paren_depth = 0;
int paren_pos = 0; /* dummy init to keep compiler quiet */
int pos;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
for (pos = 0; pos < pattlen; pos++)
{
if (patt[pos] == '(')
{
if (paren_depth == 0)
Another pgindent run. Fixes enum indenting, and improves #endif spacing. Also adds space for one-line comments.
2001-10-28 07:26:15 +01:00
paren_pos = pos; /* remember start of parenthesized item */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
paren_depth++;
}
else if (patt[pos] == ')' && paren_depth > 0)
{
paren_depth--;
if (paren_depth == 0)
sel *= regex_selectivity_sub(patt + (paren_pos + 1),
pos - (paren_pos + 1),
case_insensitive);
}
else if (patt[pos] == '|' && paren_depth == 0)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If unquoted | is present at paren level 0 in pattern, we have
* multiple alternatives; sum their probabilities.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
sel += regex_selectivity_sub(patt + (pos + 1),
pattlen - (pos + 1),
case_insensitive);
break; /* rest of pattern is now processed */
}
else if (patt[pos] == '[')
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
bool negclass = false;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
if (patt[++pos] == '^')
{
negclass = true;
pos++;
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
if (patt[pos] == ']') /* ']' at start of class is not
* special */
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
pos++;
while (pos < pattlen && patt[pos] != ']')
pos++;
if (paren_depth == 0)
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
sel *= (negclass ? (1.0 - CHAR_RANGE_SEL) : CHAR_RANGE_SEL);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
else if (patt[pos] == '.')
{
if (paren_depth == 0)
sel *= ANY_CHAR_SEL;
}
else if (patt[pos] == '*' ||
patt[pos] == '?' ||
patt[pos] == '+')
{
/* Ought to be smarter about quantifiers... */
if (paren_depth == 0)
sel *= PARTIAL_WILDCARD_SEL;
}
else if (patt[pos] == '{')
{
while (pos < pattlen && patt[pos] != '}')
pos++;
if (paren_depth == 0)
sel *= PARTIAL_WILDCARD_SEL;
}
else if (patt[pos] == '\\')
{
/* backslash quotes the next character */
pos++;
if (pos >= pattlen)
break;
if (paren_depth == 0)
sel *= FIXED_CHAR_SEL;
}
else
{
if (paren_depth == 0)
sel *= FIXED_CHAR_SEL;
}
}
/* Could get sel > 1 if multiple wildcards */
if (sel > 1.0)
sel = 1.0;
return sel;
}
static Selectivity
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
regex_selectivity(Const *patt_const, bool case_insensitive)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Selectivity sel;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
char *patt;
int pattlen;
Oid typeid = patt_const->consttype;
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Should be unnecessary, there are no bytea regex operators defined. As
* such, it should be noted that the rest of this function has *not* been
* made safe for binary (possibly NULL containing) strings.
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
*/
if (typeid == BYTEAOID)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("regular-expression matching not supported on type bytea")));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
/* the right-hand const is type text for all of these */
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
patt = TextDatumGetCString(patt_const->constvalue);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pattlen = strlen(patt);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/* If patt doesn't end with $, consider it to have a trailing wildcard */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
if (pattlen > 0 && patt[pattlen - 1] == '$' &&
(pattlen == 1 || patt[pattlen - 2] != '\\'))
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
/* has trailing $ */
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
sel = regex_selectivity_sub(patt, pattlen - 1, case_insensitive);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
else
{
/* no trailing $ */
sel = regex_selectivity_sub(patt, pattlen, case_insensitive);
sel *= FULL_WILDCARD_SEL;
if (sel > 1.0)
sel = 1.0;
}
return sel;
}
static Selectivity
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
pattern_selectivity(Const *patt, Pattern_Type ptype)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Selectivity result;
switch (ptype)
{
case Pattern_Type_Like:
Reimplement LIKE/ESCAPE as operators so that indexscan optimization can still work, per recent discussion on pghackers. Correct some bugs in ILIKE implementation.
2000-09-15 20:45:31 +02:00
result = like_selectivity(patt, false);
break;
case Pattern_Type_Like_IC:
result = like_selectivity(patt, true);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
break;
case Pattern_Type_Regex:
result = regex_selectivity(patt, false);
break;
case Pattern_Type_Regex_IC:
result = regex_selectivity(patt, true);
break;
default:
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
elog(ERROR, "unrecognized ptype: %d", (int) ptype);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
result = 1.0; /* keep compiler quiet */
break;
}
return result;
}
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
/*
* For bytea, the increment function need only increment the current byte
* (there are no multibyte characters to worry about).
*/
static bool
byte_increment(unsigned char *ptr, int len)
{
if (*ptr >= 255)
return false;
(*ptr)++;
return true;
}
Store current LC_COLLATE and LC_CTYPE settings in pg_control during initdb; re-adopt these settings at every postmaster or standalone-backend startup. This should fix problems with indexes becoming corrupt due to failure to provide consistent locale environment for postmaster at all times. Also, refuse to start up a non-locale-enabled compilation in a database originally initdb'd with a non-C locale. Suppress LIKE index optimization if locale is not "C" or "POSIX" (are there any other locales where it's safe?). Issue NOTICE during initdb if selected locale disables LIKE optimization.
2000-11-25 21:33:54 +01:00
/*
Indexing support for pattern matching operations via separate operator class when lc_collate is not C.
2003-05-15 17:50:21 +02:00
* Try to generate a string greater than the given string or any
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
* string it is a prefix of. If successful, return a palloc'd string
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
* in the form of a Const node; else return NULL.
*
* The caller must provide the appropriate "less than" comparison function
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
* for testing the strings, along with the collation to use.
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
*
* The key requirement here is that given a prefix string, say "foo",
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
* we must be able to generate another string "fop" that is greater than
* all strings "foobar" starting with "foo". We can test that we have
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
* generated a string greater than the prefix string, but in non-C collations
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
* that is not a bulletproof guarantee that an extension of the string might
* not sort after it; an example is that "foo " is less than "foo!", but it
* is not clear that a "dictionary" sort ordering will consider "foo!" less
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* than "foo bar". CAUTION: Therefore, this function should be used only for
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
* estimation purposes when working in a non-C collation.
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
*
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
* To try to catch most cases where an extended string might otherwise sort
* before the result value, we determine which of the strings "Z", "z", "y",
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
* and "9" is seen as largest by the collation, and append that to the given
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
* prefix before trying to find a string that compares as larger.
Locale support is on by default. The choice of locale is done in initdb and/or with GUC variables.
2002-04-03 07:39:33 +02:00
*
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
* To search for a greater string, we repeatedly "increment" the rightmost
* character, using an encoding-specific character incrementer function.
* When it's no longer possible to increment the last character, we truncate
* off that character and start incrementing the next-to-rightmost.
* For example, if "z" were the last character in the sort order, then we
* could produce "foo" as a string greater than "fonz".
Make initdb print a message about which locale it is about to use. Re-add warning if the locale prevents LIKE-optimization. Done within initdb now.
2002-05-09 15:30:24 +02:00
*
Indexing support for pattern matching operations via separate operator class when lc_collate is not C.
2003-05-15 17:50:21 +02:00
* This could be rather slow in the worst case, but in most cases we
* won't have to try more than one or two strings before succeeding.
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
*
* Note that it's important for the character incrementer not to be too anal
* about producing every possible character code, since in some cases the only
* way to get a larger string is to increment a previous character position.
* So we don't want to spend too much time trying every possible character
* code at the last position. A good rule of thumb is to be sure that we
* don't try more than 256*K values for a K-byte character (and definitely
* not 256^K, which is what an exhaustive search would approach).
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Const *
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
make_greater_string(const Const *str_const, FmgrInfo *ltproc, Oid collation)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Oid datatype = str_const->consttype;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
char *workstr;
int len;
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
Datum cmpstr;
text *cmptxt = NULL;
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
mbcharacter_incrementer charinc;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Get a modifiable copy of the prefix string in C-string format, and set
* up the string we will compare to as a Datum. In C locale this can just
* be the given prefix string, otherwise we need to add a suffix. Types
* NAME and BYTEA sort bytewise so they don't need a suffix either.
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
*/
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
if (datatype == NAMEOID)
{
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
workstr = DatumGetCString(DirectFunctionCall1(nameout,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
str_const->constvalue));
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
len = strlen(workstr);
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
cmpstr = str_const->constvalue;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
else if (datatype == BYTEAOID)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
bytea *bstr = DatumGetByteaP(str_const->constvalue);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
len = VARSIZE(bstr) - VARHDRSZ;
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
workstr = (char *) palloc(len);
memcpy(workstr, VARDATA(bstr), len);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
if ((Pointer) bstr != DatumGetPointer(str_const->constvalue))
pfree(bstr);
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
cmpstr = str_const->constvalue;
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
else
{
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
workstr = TextDatumGetCString(str_const->constvalue);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
len = strlen(workstr);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (lc_collate_is_c(collation) || len == 0)
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
cmpstr = str_const->constvalue;
else
{
/* If first time through, determine the suffix to use */
static char suffixchar = 0;
Fix make_greater_string to not have an undocumented collation assumption. The previous coding worked only if ltproc->fn_collation was always either DEFAULT_COLLATION_OID or a C-compatible locale. While that's true at the moment, it wasn't documented (and in fact wasn't true when this code was committed...). But it only takes a couple more lines to make its internal caching behavior locale-aware, so let's do that.
2011-04-08 23:39:59 +02:00
static Oid suffixcollation = 0;
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (!suffixchar || suffixcollation != collation)
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
char *best;
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
best = "Z";
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (varstr_cmp(best, 1, "z", 1, collation) < 0)
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
best = "z";
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (varstr_cmp(best, 1, "y", 1, collation) < 0)
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
best = "y";
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (varstr_cmp(best, 1, "9", 1, collation) < 0)
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
best = "9";
suffixchar = *best;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
suffixcollation = collation;
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
}
/* And build the string to compare to */
cmptxt = (text *) palloc(VARHDRSZ + len + 1);
SET_VARSIZE(cmptxt, VARHDRSZ + len + 1);
memcpy(VARDATA(cmptxt), workstr, len);
*(VARDATA(cmptxt) + len) = suffixchar;
cmpstr = PointerGetDatum(cmptxt);
}
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
}
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
/* Select appropriate character-incrementer function */
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
if (datatype == BYTEAOID)
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
charinc = byte_increment;
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
else
charinc = pg_database_encoding_character_incrementer();
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
/* And search ... */
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
while (len > 0)
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
int charlen;
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
unsigned char *lastchar;
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
/* Identify the last character --- for bytea, just the last byte */
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
if (datatype == BYTEAOID)
charlen = 1;
else
charlen = len - pg_mbcliplen(workstr, len, len - 1);
lastchar = (unsigned char *) (workstr + len - charlen);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
* Try to generate a larger string by incrementing the last character
* (for BYTEA, we treat each byte as a character).
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
*
* Note: the incrementer function is expected to return true if it's
* generated a valid-per-the-encoding new character, otherwise false.
* The contents of the character on false return are unspecified.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
while (charinc(lastchar, charlen))
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
{
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
Const *workstr_const;
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
if (datatype == BYTEAOID)
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
workstr_const = string_to_bytea_const(workstr, len);
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
else
workstr_const = string_to_const(workstr, datatype);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
if (DatumGetBool(FunctionCall2Coll(ltproc,
collation,
cmpstr,
workstr_const->constvalue)))
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
{
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
/* Successfully made a string larger than cmpstr */
if (cmptxt)
pfree(cmptxt);
Improve the performance of LIKE/regex estimation in non-C locales, by making make_greater_string() try harder to generate a string that's actually greater than its input string. Before we just assumed that making a string that was memcmp-greater was enough, but it is easy to generate examples where this is not so when the locale is not C. Instead, loop until the relevant comparison function agrees that the generated string is greater than the input. Unfortunately this is probably not enough to guarantee that the generated string is greater than all extensions of the input, so we cannot relax the restriction to C locale for the LIKE/regex index optimization. But it should at least improve the odds of getting a useful selectivity estimate in prefix_selectivity(). Per example from Guillaume Smet. Backpatch to 8.1, mainly because that's what the complainant is using...
2007-11-07 23:37:24 +01:00
pfree(workstr);
return workstr_const;
}
/* No good, release unusable value and try again */
pfree(DatumGetPointer(workstr_const->constvalue));
pfree(workstr_const);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
Further improvement of make_greater_string. Make sure that it considers all the possibilities that the old code did, instead of trying only one possibility per character position. To keep the runtime in bounds, instead tweak the character incrementers to not try every possible multibyte character code. Remove unnecessary logic to restore the old character value on failure. Additional comment and formatting cleanup.
2011-10-30 17:21:28 +01:00
* No luck here, so truncate off the last character and try to
* increment the next one.
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
Improve make_greater_string() with encoding-specific incrementers. This infrastructure doesn't in any way guarantee that the character we produce will sort before the one we incremented; but it does at least make it much more likely that we'll end up with something that is a valid character, which improves our chances. Kyotaro Horiguchi, with various adjustments by me.
2011-10-29 20:22:20 +02:00
len -= charlen;
workstr[len] = '\0';
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/* Failed... */
Second pass at improving LIKE/regex estimation in non-C locales. It turns out that it's actually quite likely that a string that is an extension of the given prefix will sort as larger than the "greater" string our previous code created. To provide some defense against that, do the comparisons against a modified string instead of just the bare prefix. We tack on "Z", "z", "y", or "9", whichever is seen as largest in the current locale. Testing suggests that this is sufficient at least for cases involving ASCII data.
2007-11-09 21:10:02 +01:00
if (cmptxt)
pfree(cmptxt);
Eliminate some no-longer-needed workarounds for palloc's old behavior of rejecting palloc(0). Also, tweak like_selectivity() to avoid assuming the presented pattern is nonempty; although that assumption is valid, it doesn't really help much, and the new coding is more correct anyway since it properly handles redundant wildcards. In combination these changes should eliminate a Coverity warning noted by Martijn.
2006-04-20 19:50:18 +02:00
pfree(workstr);
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
return NULL;
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
/*
* Generate a Datum of the appropriate type from a C string.
* Note that all of the supported types are pass-by-ref, so the
* returned value should be pfree'd if no longer needed.
*/
static Datum
string_to_datum(const char *str, Oid datatype)
{
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
Assert(str != NULL);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
/*
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
* We cheat a little by assuming that CStringGetTextDatum() will do for
* bpchar and varchar constants too...
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
*/
if (datatype == NAMEOID)
More functions updated to new fmgr style --- money, name, tid datatypes. We're reaching the mopup stage here (good thing too, this is getting tedious).
2000-08-03 18:35:08 +02:00
return DirectFunctionCall1(namein, CStringGetDatum(str));
> Okay. When you get back to the original issue, the gold is hidden in > src/backend/optimizer/path/indxpath.c; see the "special indexable > operators" stuff near the bottom of that file. (It's a bit of a crock > that this code is hardwired there, and not somehow accessed through a > system catalog, but it's what we've got at the moment.) The attached patch re-enables a bytea right hand argument (as compared to a text right hand argument), and enables index usage, for bytea LIKE Joe Conway
2002-09-02 08:22:20 +02:00
else if (datatype == BYTEAOID)
return DirectFunctionCall1(byteain, CStringGetDatum(str));
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
else
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
return CStringGetTextDatum(str);
Add new selectivity estimation functions for pattern-matching operators (LIKE and regexp matches). These are not yet referenced in pg_operator, so by default the system will continue to use eqsel/neqsel. Also, tweak convert_to_scalar() logic so that common prefixes of strings are stripped off, allowing better accuracy when all strings in a table share a common prefix.
2000-04-16 06:41:03 +02:00
}
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
/*
* Generate a Const node of the appropriate type from a C string.
*/
static Const *
string_to_const(const char *str, Oid datatype)
{
Datum conval = string_to_datum(str, datatype);
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
Oid collation;
int constlen;
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
/*
* We only need to support a few datatypes here, so hard-wire properties
* instead of incurring the expense of catalog lookups.
*/
switch (datatype)
{
case TEXTOID:
case VARCHAROID:
case BPCHAROID:
collation = DEFAULT_COLLATION_OID;
constlen = -1;
break;
case NAMEOID:
collation = InvalidOid;
constlen = NAMEDATALEN;
break;
case BYTEAOID:
collation = InvalidOid;
constlen = -1;
break;
default:
elog(ERROR, "unexpected datatype in string_to_const: %u",
datatype);
return NULL;
}
return makeConst(datatype, -1, collation, constlen,
Remove unused constisset and constiscast fields of Const nodes. Clean up code and documentation associated with Param nodes.
2002-11-25 22:29:42 +01:00
conval, false, false);
Modify optimizer data structures so that IndexOptInfo lists built for create_index_paths are not immediately discarded, but are available for subsequent planner work. This allows avoiding redundant syscache lookups in several places. Change interface to operator selectivity estimation procedures to allow faster and more flexible estimation. Initdb forced due to change of pg_proc entries for selectivity functions!
2001-05-20 22:28:20 +02:00
}
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
/*
* Generate a Const node of bytea type from a binary C string and a length.
*/
static Const *
string_to_bytea_const(const char *str, size_t str_len)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
bytea *bstr = palloc(VARHDRSZ + str_len);
Datum conval;
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
memcpy(VARDATA(bstr), str, str_len);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(bstr, VARHDRSZ + str_len);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
conval = PointerGetDatum(bstr);
Pass collation to makeConst() instead of looking it up internally. In nearly all cases, the caller already knows the correct collation, and in a number of places, the value the caller has handy is more correct than the default for the type would be. (In particular, this patch makes it significantly less likely that eval_const_expressions will result in changing the exposed collation of an expression.) So an internal lookup is both expensive and wrong.
2011-03-26 01:10:42 +01:00
return makeConst(BYTEAOID, -1, InvalidOid, -1, conval, false, false);
Repair indexed bytea like operations, and related selectivity functionality. Per bug report by Alvar Freude: http://archives.postgresql.org/pgsql-bugs/2003-12/msg00022.php
2003-12-07 05:14:10 +01:00
}
Revise handling of index-type-specific indexscan cost estimation, per pghackers discussion of 5-Jan-2000. The amopselect and amopnpages estimators are gone, and in their place is a per-AM amcostestimate procedure (linked to from pg_am, not pg_amop).
2000-01-23 00:50:30 +01:00
/*-------------------------------------------------------------------------
*
* Index cost estimation functions
*
* genericcostestimate is a general-purpose estimator for use when we
* don't have any better idea about how to estimate. Index-type-specific
* knowledge can be incorporated in the type-specific routines.
*
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
* One bit of index-type-specific knowledge we can relatively easily use
* in genericcostestimate is the estimate of the number of index tuples
* visited. If numIndexTuples is not 0 then it is used as the estimate,
* otherwise we compute a generic estimate.
*
Revise handling of index-type-specific indexscan cost estimation, per pghackers discussion of 5-Jan-2000. The amopselect and amopnpages estimators are gone, and in their place is a per-AM amcostestimate procedure (linked to from pg_am, not pg_amop).
2000-01-23 00:50:30 +01:00
*-------------------------------------------------------------------------
*/
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
static void
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
genericcostestimate(PlannerInfo *root,
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path,
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
RelOptInfo *outer_rel,
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
double numIndexTuples,
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Cost *indexStartupCost,
Cost *indexTotalCost,
Selectivity *indexSelectivity,
double *indexCorrelation)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexOptInfo *index = path->indexinfo;
List *indexQuals = path->indexquals;
List *indexOrderBys = path->indexorderbys;
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
double numIndexPages;
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
double num_sa_scans;
double num_outer_scans;
double num_scans;
Revise cost_qual_eval() to compute both startup (one-time) and per-tuple costs for expression evaluation, not only per-tuple cost as before. This extension is needed in order to deal realistically with hashed or materialized sub-selects.
2003-01-12 23:35:29 +01:00
QualCost index_qual_cost;
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
double qual_op_cost;
double qual_arg_cost;
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
double spc_random_page_cost;
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
List *selectivityQuals;
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
ListCell *l;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
/*----------
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If the index is partial, AND the index predicate with the explicitly
* given indexquals to produce a more accurate idea of the index
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* selectivity. However, we need to be careful not to insert redundant
* clauses, because clauselist_selectivity() is easily fooled into
* computing a too-low selectivity estimate. Our approach is to add
* only the index predicate clause(s) that cannot be proven to be implied
* by the given indexquals. This successfully handles cases such as a
* qual "x = 42" used with a partial index "WHERE x >= 40 AND x < 50".
* There are many other cases where we won't detect redundancy, leading
* to a too-low selectivity estimate, which will bias the system in favor
* of using partial indexes where possible. That is not necessarily bad
* though.
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
*
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
* Note that indexQuals contains RestrictInfo nodes while the indpred
* does not. This is OK for both predicate_implied_by() and
* clauselist_selectivity().
*----------
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
*/
if (index->indpred != NIL)
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
{
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
List *predExtraQuals = NIL;
foreach(l, index->indpred)
{
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Node *predQual = (Node *) lfirst(l);
List *oneQual = list_make1(predQual);
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
Fix some problems with selectivity estimation for partial indexes. First, genericcostestimate() was being way too liberal about including partial-index conditions in its selectivity estimate, resulting in substantial underestimates for situations such as an indexqual "x = 42" used with an index on x "WHERE x >= 40 AND x < 50". While the code is intentionally set up to favor selecting partial indexes when available, this was too much... Second, choose_bitmap_and() was likewise easily fooled by cases of this type, since it would similarly think that the partial index had selectivity independent of the indexqual. Fixed by using predicate_implied_by() rather than simple equality checks to determine redundancy. This is a good deal more expensive but I don't see much alternative. At least the extra cost is only paid when there's actually a partial index under consideration. Per report from Jeff Davis. I'm not going to risk back-patching this, though.
2007-03-21 23:18:12 +01:00
if (!predicate_implied_by(oneQual, indexQuals))
predExtraQuals = list_concat(predExtraQuals, oneQual);
}
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
/* list_concat avoids modifying the passed-in indexQuals list */
Use the new List API function names throughout the backend, and disable the list compatibility API by default. While doing this, I decided to keep the llast() macro around and introduce llast_int() and llast_oid() variants.
2004-05-31 01:40:41 +02:00
selectivityQuals = list_concat(predExtraQuals, indexQuals);
Adjust indexscan planning logic to keep RestrictInfo nodes associated with index qual clauses in the Path representation. This saves a little work during createplan and (probably more importantly) allows reuse of cached selectivity estimates during indexscan planning. Also fix latent bug: wrong plan would have been generated for a 'special operator' used in a nestloop-inner-indexscan join qual, because the special operator would not have gotten into the list of quals to recheck. This bug is only latent because at present the special-operator code could never trigger on a join qual, but sooner or later someone will want to do it.
2004-01-06 00:39:54 +01:00
}
else
selectivityQuals = indexQuals;
Generate a more specific error message when an operator used in an index doesn't have a restriction selectivity estimator.
1999-05-31 21:32:47 +02:00
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Check for ScalarArrayOpExpr index quals, and estimate the number of
* index scans that will be performed.
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
*/
num_sa_scans = 1;
foreach(l, indexQuals)
{
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
if (IsA(rinfo->clause, ScalarArrayOpExpr))
{
ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) rinfo->clause;
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
int alength = estimate_array_length(lsecond(saop->args));
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
if (alength > 1)
num_sa_scans *= alength;
}
}
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/* Estimate the fraction of main-table tuples that will be visited */
*indexSelectivity = clauselist_selectivity(root, selectivityQuals,
First steps towards index scans with heap access decoupled from index access: define new index access method functions 'amgetmulti' that can fetch multiple TIDs per call. (The functions exist but are totally untested as yet.) Since I was modifying pg_am anyway, remove the no-longer-needed 'rel' parameter from amcostestimate functions, and also remove the vestigial amowner column that was creating useless work for Alvaro's shared-object-dependencies project. Initdb forced due to changes in pg_am.
2005-03-28 01:53:05 +02:00
index->rel->relid,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
JOIN_INNER,
NULL);
Further tweaking of indexscan cost estimates.
2000-04-09 06:31:37 +02:00
/*
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
* If caller didn't give us an estimate, estimate the number of index
* tuples that will be visited. We do it in this rather peculiar-looking
* way in order to get the right answer for partial indexes.
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
*/
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
if (numIndexTuples <= 0.0)
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
{
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
numIndexTuples = *indexSelectivity * index->rel->tuples;
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
/*
* The above calculation counts all the tuples visited across all
* scans induced by ScalarArrayOpExpr nodes. We want to consider the
* average per-indexscan number, so adjust. This is a handy place to
* round to integer, too. (If caller supplied tuple estimate, it's
* responsible for handling these considerations.)
*/
numIndexTuples = rint(numIndexTuples / num_sa_scans);
}
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* We can bound the number of tuples by the index size in any case. Also,
* always estimate at least one tuple is touched, even when
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
* indexSelectivity estimate is tiny.
Further tweaking of indexscan cost estimates.
2000-04-09 06:31:37 +02:00
*/
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
if (numIndexTuples > index->tuples)
numIndexTuples = index->tuples;
Further tweaking of indexscan cost estimates.
2000-04-09 06:31:37 +02:00
if (numIndexTuples < 1.0)
numIndexTuples = 1.0;
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
/*
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
* Estimate the number of index pages that will be retrieved.
*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* We use the simplistic method of taking a pro-rata fraction of the total
* number of index pages. In effect, this counts only leaf pages and not
* any overhead such as index metapage or upper tree levels. In practice
* this seems a better approximation than charging for access to the upper
* levels, perhaps because those tend to stay in cache under load.
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
*/
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
if (index->pages > 1 && index->tuples > 1)
numIndexPages = ceil(numIndexTuples * index->pages / index->tuples);
Partial indexes work again, courtesy of Martijn van Oosterhout. Note: I didn't force an initdb, figuring that one today was enough. However, there is a new function in pg_proc.h, and pg_dump won't be able to dump partial indexes until you add that function.
2001-07-16 07:07:00 +02:00
else
Further tweaking of indexscan cost estimates.
2000-04-09 06:31:37 +02:00
numIndexPages = 1.0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
/* fetch estimated page cost for schema containing index */
get_tablespace_page_costs(index->reltablespace,
&spc_random_page_cost,
NULL);
New cost model for planning, incorporating a penalty for random page accesses versus sequential accesses, a (very crude) estimate of the effects of caching on random page accesses, and cost to evaluate WHERE- clause expressions. Export critical parameters for this model as SET variables. Also, create SET variables for the planner's enable flags (enable_seqscan, enable_indexscan, etc) so that these can be controlled more conveniently than via PGOPTIONS. Planner now estimates both startup cost (cost before retrieving first tuple) and total cost of each path, so it can optimize queries with LIMIT on a reasonable basis by interpolating between these costs. Same facility is a win for EXISTS(...) subqueries and some other cases. Redesign pathkey representation to achieve a major speedup in planning (I saw as much as 5X on a 10-way join); also minor changes in planner to reduce memory consumption by recycling discarded Path nodes and not constructing unnecessary lists. Minor cleanups to display more-plausible costs in some cases in EXPLAIN output. Initdb forced by change in interface to index cost estimation functions.
2000-02-15 21:49:31 +01:00
/*
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
* Now compute the disk access costs.
New cost model for planning, incorporating a penalty for random page accesses versus sequential accesses, a (very crude) estimate of the effects of caching on random page accesses, and cost to evaluate WHERE- clause expressions. Export critical parameters for this model as SET variables. Also, create SET variables for the planner's enable flags (enable_seqscan, enable_indexscan, etc) so that these can be controlled more conveniently than via PGOPTIONS. Planner now estimates both startup cost (cost before retrieving first tuple) and total cost of each path, so it can optimize queries with LIMIT on a reasonable basis by interpolating between these costs. Same facility is a win for EXISTS(...) subqueries and some other cases. Redesign pathkey representation to achieve a major speedup in planning (I saw as much as 5X on a 10-way join); also minor changes in planner to reduce memory consumption by recycling discarded Path nodes and not constructing unnecessary lists. Minor cleanups to display more-plausible costs in some cases in EXPLAIN output. Initdb forced by change in interface to index cost estimation functions.
2000-02-15 21:49:31 +01:00
*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* The above calculations are all per-index-scan. However, if we are in a
* nestloop inner scan, we can expect the scan to be repeated (with
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
* different search keys) for each row of the outer relation. Likewise,
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* ScalarArrayOpExpr quals result in multiple index scans. This creates
* the potential for cache effects to reduce the number of disk page
* fetches needed. We want to estimate the average per-scan I/O cost in
* the presence of caching.
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
*
* We use the Mackert-Lohman formula (see costsize.c for details) to
* estimate the total number of page fetches that occur. While this
* wasn't what it was designed for, it seems a reasonable model anyway.
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Note that we are counting pages not tuples anymore, so we take N = T =
* index size, as if there were one "tuple" per page.
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
*/
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
if (outer_rel != NULL && outer_rel->rows > 1)
{
Fix oversight in planning for multiple indexscans driven by ScalarArrayOpExpr index quals: we were estimating the right total number of rows returned, but treating the index-access part of the cost as if a single scan were fetching that many consecutive index tuples. Actually we should treat it as a multiple indexscan, and if there are enough of 'em the Mackert-Lohman discount should kick in.
2006-07-02 00:07:23 +02:00
num_outer_scans = outer_rel->rows;
num_scans = num_sa_scans * num_outer_scans;
}
else
{
num_outer_scans = 1;
num_scans = num_sa_scans;
}
if (num_scans > 1)
{
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
double pages_fetched;
/* total page fetches ignoring cache effects */
pages_fetched = numIndexPages * num_scans;
/* use Mackert and Lohman formula to adjust for cache effects */
pages_fetched = index_pages_fetched(pages_fetched,
index->pages,
Improve usage of effective_cache_size parameter by assuming that all the tables in the query compete for cache space, not just the one we are currently costing an indexscan for. This seems more realistic, and it definitely will help in examples recently exhibited by Stefan Kaltenbrunner. To get the total size of all the tables involved, we must tweak the handling of 'append relations' a bit --- formerly we looked up information about the child tables on-the-fly during set_append_rel_pathlist, but it needs to be done before we start doing any cost estimation, so push it into the add_base_rels_to_query scan.
2006-09-20 00:49:53 +02:00
(double) index->pages,
root);
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* Now compute the total disk access cost, and then report a pro-rated
* share for each outer scan. (Don't pro-rate for ScalarArrayOpExpr,
* since that's internal to the indexscan.)
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
*/
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
*indexTotalCost = (pages_fetched * spc_random_page_cost)
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
/ num_outer_scans;
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
}
else
{
/*
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
* For a single index scan, we just charge spc_random_page_cost per
* page touched.
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
*/
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
*indexTotalCost = numIndexPages * spc_random_page_cost;
Make the planner estimate costs for nestloop inner indexscans on the basis that the Mackert-Lohmann formula applies across all the repetitions of the nestloop, not just each scan independently. We use the M-L formula to estimate the number of pages fetched from the index as well as from the table; that isn't what it was designed for, but it seems reasonably applicable anyway. This makes large numbers of repetitions look much cheaper than before, which accords with many reports we've received of overestimation of the cost of a nestloop. Also, change the index access cost model to charge random_page_cost per index leaf page touched, while explicitly not counting anything for access to metapage or upper tree pages. This may all need tweaking after we get some field experience, but in simple tests it seems to be giving saner results than before. The main thing is to get the infrastructure in place to let cost_index() and amcostestimate functions take repeated scans into account at all. Per my recent proposal. Note: this patch changes pg_proc.h, but I did not force initdb because the changes are basically cosmetic --- the system does not look into pg_proc to decide how to call an index amcostestimate function, and there's no way to call such a function from SQL at all.
2006-06-06 19:59:58 +02:00
}
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
Add a fudge factor to genericcostestimate() to prevent the planner from thinking that indexes of different sizes are equally attractive. Per gripe from Jim Nasby. (I remain unconvinced that there's such a problem in existing releases, but CVS HEAD definitely has got a problem because of its new count-only-leaf-pages approach to indexscan costing.)
2006-07-24 03:19:48 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* A difficulty with the leaf-pages-only cost approach is that for small
* selectivities (eg, single index tuple fetched) all indexes will look
* equally attractive because we will estimate exactly 1 leaf page to be
* fetched. All else being equal, we should prefer physically smaller
* indexes over larger ones. (An index might be smaller because it is
* partial or because it contains fewer columns; presumably the other
* columns in the larger index aren't useful to the query, or the larger
* index would have better selectivity.)
Add a fudge factor to genericcostestimate() to prevent the planner from thinking that indexes of different sizes are equally attractive. Per gripe from Jim Nasby. (I remain unconvinced that there's such a problem in existing releases, but CVS HEAD definitely has got a problem because of its new count-only-leaf-pages approach to indexscan costing.)
2006-07-24 03:19:48 +02:00
*
* We can deal with this by adding a very small "fudge factor" that
* depends on the index size. The fudge factor used here is one
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
* spc_random_page_cost per 100000 index pages, which should be small
* enough to not alter index-vs-seqscan decisions, but will prevent
* indexes of different sizes from looking exactly equally attractive.
Add a fudge factor to genericcostestimate() to prevent the planner from thinking that indexes of different sizes are equally attractive. Per gripe from Jim Nasby. (I remain unconvinced that there's such a problem in existing releases, but CVS HEAD definitely has got a problem because of its new count-only-leaf-pages approach to indexscan costing.)
2006-07-24 03:19:48 +02:00
*/
Support ALTER TABLESPACE name SET/RESET ( tablespace_options ). This patch only supports seq_page_cost and random_page_cost as parameters, but it provides the infrastructure to scalably support many more. In particular, we may want to add support for effective_io_concurrency, but I'm leaving that as future work for now. Thanks to Tom Lane for design help and Alvaro Herrera for the review.
2010-01-05 22:54:00 +01:00
*indexTotalCost += index->pages * spc_random_page_cost / 100000.0;
Add a fudge factor to genericcostestimate() to prevent the planner from thinking that indexes of different sizes are equally attractive. Per gripe from Jim Nasby. (I remain unconvinced that there's such a problem in existing releases, but CVS HEAD definitely has got a problem because of its new count-only-leaf-pages approach to indexscan costing.)
2006-07-24 03:19:48 +02:00
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* CPU cost: any complex expressions in the indexquals will need to be
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* evaluated once at the start of the scan to reduce them to runtime keys
* to pass to the index AM (see nodeIndexscan.c). We model the per-tuple
* CPU costs as cpu_index_tuple_cost plus one cpu_operator_cost per
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* indexqual operator. Because we have numIndexTuples as a per-scan
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
* number, we have to multiply by num_sa_scans to get the correct result
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
* for ScalarArrayOpExpr cases. Similarly add in costs for any index
* ORDER BY expressions.
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Note: this neglects the possible costs of rechecking lossy operators
* and OR-clause expressions. Detecting that that might be needed seems
* more expensive than it's worth, though, considering all the other
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* inaccuracies here ...
Ye-old pgindent run. Same 4-space tabs.
2000-04-12 19:17:23 +02:00
*/
Turn the rangetable used by the executor into a flat list, and avoid storing useless substructure for its RangeTblEntry nodes. (I chose to keep using the same struct node type and just zero out the link fields for unneeded info, rather than making a separate ExecRangeTblEntry type --- it seemed too fragile to have two different rangetable representations.) Along the way, put subplans into a list in the toplevel PlannedStmt node, and have SubPlan nodes refer to them by list index instead of direct pointers. Vadim wanted to do that years ago, but I never understood what he was on about until now. It makes things a *whole* lot more robust, because we can stop worrying about duplicate processing of subplans during expression tree traversals. That's been a constant source of bugs, and it's finally gone. There are some consequent simplifications yet to be made, like not using a separate EState for subplans in the executor, but I'll tackle that later.
2007-02-22 23:00:26 +01:00
cost_qual_eval(&index_qual_cost, indexQuals, root);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
qual_arg_cost = index_qual_cost.startup + index_qual_cost.per_tuple;
cost_qual_eval(&index_qual_cost, indexOrderBys, root);
qual_arg_cost += index_qual_cost.startup + index_qual_cost.per_tuple;
qual_op_cost = cpu_operator_cost *
(list_length(indexQuals) + list_length(indexOrderBys));
qual_arg_cost -= qual_op_cost;
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
if (qual_arg_cost < 0) /* just in case... */
qual_arg_cost = 0;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
Repair misestimation of indexscan CPU costs. When an indexqual contains a run-time key (that is, a nonconstant expression compared to the index variable), the key is evaluated just once per scan, but we were charging costs as though it were evaluated once per visited index entry.
2004-01-17 21:09:35 +01:00
*indexStartupCost = qual_arg_cost;
genericcostestimate() neglected to include qual startup cost in indexTotalCost. I think this may not make any real difference in 7.4, but it definitely is a problem with CVS tip's new equation.
2004-02-27 22:44:34 +01:00
*indexTotalCost += qual_arg_cost;
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
*indexTotalCost += numIndexTuples * num_sa_scans * (cpu_index_tuple_cost + qual_op_cost);
/*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* We also add a CPU-cost component to represent the general costs of
* starting an indexscan, such as analysis of btree index keys and initial
* tree descent. This is estimated at 100x cpu_operator_cost, which is a
* bit arbitrary but seems the right order of magnitude. (As noted above,
* we don't charge any I/O for touching upper tree levels, but charging
* nothing at all has been found too optimistic.)
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
*
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* Although this is startup cost with respect to any one scan, we add it
* to the "total" cost component because it's only very interesting in the
* many-ScalarArrayOpExpr-scan case, and there it will be paid over the
* life of the scan node.
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
*/
*indexTotalCost += num_sa_scans * 100.0 * cpu_operator_cost;
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
/*
* Generic assumption about index correlation: there isn't any.
*/
*indexCorrelation = 0.0;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
btcostestimate(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path = (IndexPath *) PG_GETARG_POINTER(1);
RelOptInfo *outer_rel = (RelOptInfo *) PG_GETARG_POINTER(2);
Cost *indexStartupCost = (Cost *) PG_GETARG_POINTER(3);
Cost *indexTotalCost = (Cost *) PG_GETARG_POINTER(4);
Selectivity *indexSelectivity = (Selectivity *) PG_GETARG_POINTER(5);
double *indexCorrelation = (double *) PG_GETARG_POINTER(6);
IndexOptInfo *index = path->indexinfo;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Oid relid;
AttrNumber colnum;
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
VariableStatData vardata;
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
double numIndexTuples;
List *indexBoundQuals;
int indexcol;
bool eqQualHere;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
bool found_saop;
Support "x IS NOT NULL" clauses as indexscan conditions. This turns out to be just a minor extension of the previous patch that made "x IS NULL" indexable, because we can treat the IS NOT NULL condition as if it were "x < NULL" or "x > NULL" (depending on the index's NULLS FIRST/LAST option), just like IS NULL is treated like "x = NULL". Aside from any possible usefulness in its own right, this is an important improvement for index-optimized MAX/MIN aggregates: it is now reliably possible to get a column's min or max value cheaply, even when there are a lot of nulls cluttering the interesting end of the index.
2010-01-01 22:53:49 +01:00
bool found_is_null_op;
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
double num_sa_scans;
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
ListCell *lcc,
*lci;
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* For a btree scan, only leading '=' quals plus inequality quals for the
* immediately next attribute contribute to index selectivity (these are
* the "boundary quals" that determine the starting and stopping points of
* the index scan). Additional quals can suppress visits to the heap, so
* it's OK to count them in indexSelectivity, but they should not count
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
* for estimating numIndexTuples. So we must examine the given indexquals
* to find out which ones count as boundary quals. We rely on the
* knowledge that they are given in index column order.
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
*
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
* For a RowCompareExpr, we consider only the first column, just as
* rowcomparesel() does.
*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* If there's a ScalarArrayOpExpr in the quals, we'll actually perform N
* index scans not one, but the ScalarArrayOpExpr's operator can be
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
* considered to act the same as it normally does.
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
*/
indexBoundQuals = NIL;
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
indexcol = 0;
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
eqQualHere = false;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
found_saop = false;
Support "x IS NOT NULL" clauses as indexscan conditions. This turns out to be just a minor extension of the previous patch that made "x IS NULL" indexable, because we can treat the IS NOT NULL condition as if it were "x < NULL" or "x > NULL" (depending on the index's NULLS FIRST/LAST option), just like IS NULL is treated like "x = NULL". Aside from any possible usefulness in its own right, this is an important improvement for index-optimized MAX/MIN aggregates: it is now reliably possible to get a column's min or max value cheaply, even when there are a lot of nulls cluttering the interesting end of the index.
2010-01-01 22:53:49 +01:00
found_is_null_op = false;
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
num_sa_scans = 1;
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
forboth(lcc, path->indexquals, lci, path->indexqualcols)
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(lcc);
Expr *clause;
Node *leftop,
*rightop;
Oid clause_op;
int op_strategy;
bool is_null_op = false;
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
if (indexcol != lfirst_int(lci))
{
/* Beginning of a new column's quals */
if (!eqQualHere)
break; /* done if no '=' qual for indexcol */
eqQualHere = false;
indexcol++;
if (indexcol != lfirst_int(lci))
break; /* no quals at all for indexcol */
}
Assert(IsA(rinfo, RestrictInfo));
clause = rinfo->clause;
Allow row comparisons to be used as indexscan qualifications. This completes the project to upgrade our handling of row comparisons.
2006-01-25 21:29:24 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
if (IsA(clause, OpExpr))
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
leftop = get_leftop(clause);
rightop = get_rightop(clause);
clause_op = ((OpExpr *) clause)->opno;
}
else if (IsA(clause, RowCompareExpr))
{
RowCompareExpr *rc = (RowCompareExpr *) clause;
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
leftop = (Node *) linitial(rc->largs);
rightop = (Node *) linitial(rc->rargs);
clause_op = linitial_oid(rc->opnos);
}
else if (IsA(clause, ScalarArrayOpExpr))
{
ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) clause;
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
leftop = (Node *) linitial(saop->args);
rightop = (Node *) lsecond(saop->args);
clause_op = saop->opno;
found_saop = true;
}
else if (IsA(clause, NullTest))
{
NullTest *nt = (NullTest *) clause;
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
leftop = (Node *) nt->arg;
rightop = NULL;
clause_op = InvalidOid;
if (nt->nulltesttype == IS_NULL)
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
{
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
found_is_null_op = true;
is_null_op = true;
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
}
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
}
else
{
elog(ERROR, "unsupported indexqual type: %d",
(int) nodeTag(clause));
continue; /* keep compiler quiet */
}
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
if (match_index_to_operand(leftop, indexcol, index))
{
/* clause_op is correct */
}
else
{
Assert(match_index_to_operand(rightop, indexcol, index));
/* Must flip operator to get the opfamily member */
clause_op = get_commutator(clause_op);
}
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
/* check for equality operator */
if (OidIsValid(clause_op))
{
op_strategy = get_op_opfamily_strategy(clause_op,
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
index->opfamily[indexcol]);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
Assert(op_strategy != 0); /* not a member of opfamily?? */
if (op_strategy == BTEqualStrategyNumber)
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
eqQualHere = true;
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
}
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
else if (is_null_op)
{
/* IS NULL is like = for purposes of selectivity determination */
eqQualHere = true;
}
/* count up number of SA scans induced by indexBoundQuals only */
if (IsA(clause, ScalarArrayOpExpr))
{
ScalarArrayOpExpr *saop = (ScalarArrayOpExpr *) clause;
int alength = estimate_array_length(lsecond(saop->args));
Improve planner's handling of duplicated index column expressions. It's potentially useful for an index to repeat the same indexable column or expression in multiple index columns, if the columns have different opclasses. (If they share opclasses too, the duplicate column is pretty useless, but nonetheless we've allowed such cases since 9.0.) However, the planner failed to cope with this, because createplan.c was relying on simple equal() matching to figure out which index column each index qual is intended for. We do have that information available upstream in indxpath.c, though, so the fix is to not flatten the multi-level indexquals list when putting it into an IndexPath. Then we can rely on the sublist structure to identify target index columns in createplan.c. There's a similar issue for index ORDER BYs (the KNNGIST feature), so introduce a multi-level-list representation for that too. This adds a bit more representational overhead, but we might more or less buy that back by not having to search for matching index columns anymore in createplan.c; likewise btcostestimate saves some cycles. Per bug #6351 from Christian Rudolph. Likely symptoms include the "btree index keys must be ordered by attribute" failure shown there, as well as "operator MMMM is not a member of opfamily NNNN". Although this is a pre-existing problem that can be demonstrated in 9.0 and 9.1, I'm not going to back-patch it, because the API changes in the planner seem likely to break things such as index plugins. The corner cases where this matters seem too narrow to justify possibly breaking things in a minor release.
2011-12-24 00:44:21 +01:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
if (alength > 1)
num_sa_scans *= alength;
}
indexBoundQuals = lappend(indexBoundQuals, rinfo);
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
}
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If index is unique and we found an '=' clause for each column, we can
* just assume numIndexTuples = 1 and skip the expensive
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
* clauselist_selectivity calculations. However, a ScalarArrayOp or
* NullTest invalidates that theory, even though it sets eqQualHere.
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
*/
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
if (index->unique &&
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
indexcol == index->ncolumns - 1 &&
Teach planner and executor to handle ScalarArrayOpExpr as an indexable qualification when the underlying operator is indexable and useOr is true. That is, indexkey op ANY (ARRAY[...]) is effectively translated into an OR combination of one indexscan for each array element. This only works for bitmap index scans, of course, since regular indexscans no longer support OR'ing of scans. There are still some loose ends to clean up before changing 'x IN (list)' to translate as a ScalarArrayOpExpr; for instance predtest.c ought to be taught about it. But this gets the basic functionality in place.
2005-11-25 20:47:50 +01:00
eqQualHere &&
Make 'col IS NULL' clauses be indexable conditions. Teodor Sigaev, with some kibitzing from Tom Lane.
2007-04-07 00:33:43 +02:00
!found_saop &&
Support "x IS NOT NULL" clauses as indexscan conditions. This turns out to be just a minor extension of the previous patch that made "x IS NULL" indexable, because we can treat the IS NOT NULL condition as if it were "x < NULL" or "x > NULL" (depending on the index's NULLS FIRST/LAST option), just like IS NULL is treated like "x = NULL". Aside from any possible usefulness in its own right, this is an important improvement for index-optimized MAX/MIN aggregates: it is now reliably possible to get a column's min or max value cheaply, even when there are a lot of nulls cluttering the interesting end of the index.
2010-01-01 22:53:49 +01:00
!found_is_null_op)
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
numIndexTuples = 1.0;
else
{
Selectivity btreeSelectivity;
btreeSelectivity = clauselist_selectivity(root, indexBoundQuals,
index->rel->relid,
Implement SEMI and ANTI joins in the planner and executor. (Semijoins replace the old JOIN_IN code, but antijoins are new functionality.) Teach the planner to convert appropriate EXISTS and NOT EXISTS subqueries into semi and anti joins respectively. Also, LEFT JOINs with suitable upper-level IS NULL filters are recognized as being anti joins. Unify the InClauseInfo and OuterJoinInfo infrastructure into "SpecialJoinInfo". With that change, it becomes possible to associate a SpecialJoinInfo with every join attempt, which permits some cleanup of join selectivity estimation. That needs to be taken much further than this patch does, but the next step is to change the API for oprjoin selectivity functions, which seems like material for a separate patch. So for the moment the output size estimates for semi and especially anti joins are quite bogus.
2008-08-14 20:48:00 +02:00
JOIN_INNER,
NULL);
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
numIndexTuples = btreeSelectivity * index->rel->tuples;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
/*
* As in genericcostestimate(), we have to adjust for any
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
* ScalarArrayOpExpr quals included in indexBoundQuals, and then round
* to integer.
Fix some planner bugs exposed by reports from Arjen van der Meijden. These are all in new-in-8.2 logic associated with indexability of ScalarArrayOpExpr (IN-clauses) or amortization of indexscan costs across repeated indexscans on the inside of a nestloop. In particular: Fix some logic errors in the estimation for multiple scans induced by a ScalarArrayOpExpr indexqual. Include a small cost component in bitmap index scans to reflect the costs of manipulating the bitmap itself; this is mainly to prevent a bitmap scan from appearing to have the same cost as a plain indexscan for fetching a single tuple. Also add a per-index-scan-startup CPU cost component; while prior releases were clearly too pessimistic about the cost of repeated indexscans, the original 8.2 coding allowed the cost of an indexscan to effectively go to zero if repeated often enough, which is overly optimistic. Pay some attention to index correlation when estimating costs for a nestloop inner indexscan: this is significant when the plan fetches multiple heap tuples per iteration, since high correlation means those tuples are probably on the same or adjacent heap pages.
2006-12-15 19:42:26 +01:00
*/
numIndexTuples = rint(numIndexTuples / num_sa_scans);
Change the planner to allow indexscan qualification clauses to use nonconsecutive columns of a multicolumn index, as per discussion around mid-May (pghackers thread "Best way to scan on-disk bitmaps"). This turns out to require only minimal changes in btree, and so far as I can see none at all in GiST. btcostestimate did need some work, but its original assumption that index selectivity == heap selectivity was quite bogus even before this.
2005-06-14 01:14:49 +02:00
}
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
genericcostestimate(root, path, outer_rel,
numIndexTuples,
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
indexStartupCost, indexTotalCost,
indexSelectivity, indexCorrelation);
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If we can get an estimate of the first column's ordering correlation C
* from pg_statistic, estimate the index correlation as C for a
* single-column index, or C * 0.75 for multiple columns. (The idea here
* is that multiple columns dilute the importance of the first column's
* ordering, but don't negate it entirely. Before 8.0 we divided the
* correlation by the number of columns, but that seems too strong.)
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
*/
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
MemSet(&vardata, 0, sizeof(vardata));
Replace functional-index facility with expressional indexes. Any column of an index can now be a computed expression instead of a simple variable. Restrictions on expressions are the same as for predicates (only immutable functions, no sub-selects). This fixes problems recently introduced with inlining SQL functions, because the inlining transformation is applied to both expression trees so the planner can still match them up. Along the way, improve efficiency of handling index predicates (both predicates and index expressions are now cached by the relcache) and fix 7.3 oversight that didn't record dependencies of predicate expressions.
2003-05-28 18:04:02 +02:00
if (index->indexkeys[0] != 0)
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
/* Simple variable --- look to stats for the underlying table */
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
RangeTblEntry *rte = planner_rt_fetch(index->rel->relid, root);
Assert(rte->rtekind == RTE_RELATION);
relid = rte->relid;
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Assert(relid != InvalidOid);
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
colnum = index->indexkeys[0];
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
if (get_relation_stats_hook &&
(*get_relation_stats_hook) (root, rte, colnum, &vardata))
{
/*
* The hook took control of acquiring a stats tuple. If it did
* supply a tuple, it'd better have supplied a freefunc.
*/
if (HeapTupleIsValid(vardata.statsTuple) &&
!vardata.freefunc)
elog(ERROR, "no function provided to release variable stats with");
}
else
{
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
vardata.statsTuple = SearchSysCache3(STATRELATTINH,
ObjectIdGetDatum(relid),
Int16GetDatum(colnum),
BoolGetDatum(rte->inh));
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
vardata.freefunc = ReleaseSysCache;
}
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
}
else
{
/* Expression --- maybe there are stats for the index itself */
relid = index->indexoid;
colnum = 1;
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
if (get_index_stats_hook &&
(*get_index_stats_hook) (root, relid, colnum, &vardata))
{
/*
* The hook took control of acquiring a stats tuple. If it did
* supply a tuple, it'd better have supplied a freefunc.
*/
if (HeapTupleIsValid(vardata.statsTuple) &&
!vardata.freefunc)
elog(ERROR, "no function provided to release variable stats with");
}
else
{
Wrap calls to SearchSysCache and related functions using macros. The purpose of this change is to eliminate the need for every caller of SearchSysCache, SearchSysCacheCopy, SearchSysCacheExists, GetSysCacheOid, and SearchSysCacheList to know the maximum number of allowable keys for a syscache entry (currently 4). This will make it far easier to increase the maximum number of keys in a future release should we choose to do so, and it makes the code shorter, too. Design and review by Tom Lane.
2010-02-14 19:42:19 +01:00
vardata.statsTuple = SearchSysCache3(STATRELATTINH,
ObjectIdGetDatum(relid),
Int16GetDatum(colnum),
BoolGetDatum(false));
Dept of second thoughts: let's make sure that get_index_stats_hook is only applied to expression indexes, not to plain relations. The original coding in btcostestimate conflated the two cases, but it's not hard to use get_relation_stats_hook instead when we're looking to the underlying relation.
2008-09-28 22:42:12 +02:00
vardata.freefunc = ReleaseSysCache;
}
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
}
if (HeapTupleIsValid(vardata.statsTuple))
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
{
Simplify and speed up mapping of index opfamilies to pathkeys. Formerly we looked up the operators associated with each index (caching them in relcache) and then the planner looked up the btree opfamily containing such operators in order to build the btree-centric pathkey representation that describes the index's sort order. This is quite pointless for btree indexes: we might as well just use the index's opfamily information directly. That saves syscache lookup cycles during planning, and furthermore allows us to eliminate the relcache's caching of operators altogether, which may help in reducing backend startup time. I added code to plancat.c to perform the same type of double lookup on-the-fly if it's ever faced with a non-btree amcanorder index AM. If such a thing actually becomes interesting for production, we should replace that logic with some more-direct method for identifying the corresponding btree opfamily; but it's not worth spending effort on now. There is considerably more to do pursuant to my recent proposal to get rid of sort-operator-based representations of sort orderings, but this patch grabs some of the low-hanging fruit. I'll look at the remainder of that work after the current commitfest.
2010-11-29 18:29:42 +01:00
Oid sortop;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
float4 *numbers;
int nnumbers;
Simplify and speed up mapping of index opfamilies to pathkeys. Formerly we looked up the operators associated with each index (caching them in relcache) and then the planner looked up the btree opfamily containing such operators in order to build the btree-centric pathkey representation that describes the index's sort order. This is quite pointless for btree indexes: we might as well just use the index's opfamily information directly. That saves syscache lookup cycles during planning, and furthermore allows us to eliminate the relcache's caching of operators altogether, which may help in reducing backend startup time. I added code to plancat.c to perform the same type of double lookup on-the-fly if it's ever faced with a non-btree amcanorder index AM. If such a thing actually becomes interesting for production, we should replace that logic with some more-direct method for identifying the corresponding btree opfamily; but it's not worth spending effort on now. There is considerably more to do pursuant to my recent proposal to get rid of sort-operator-based representations of sort orderings, but this patch grabs some of the low-hanging fruit. I'll look at the remainder of that work after the current commitfest.
2010-11-29 18:29:42 +01:00
sortop = get_opfamily_member(index->opfamily[0],
index->opcintype[0],
index->opcintype[0],
BTLessStrategyNumber);
if (OidIsValid(sortop) &&
get_attstatsslot(vardata.statsTuple, InvalidOid, 0,
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
STATISTIC_KIND_CORRELATION,
Simplify and speed up mapping of index opfamilies to pathkeys. Formerly we looked up the operators associated with each index (caching them in relcache) and then the planner looked up the btree opfamily containing such operators in order to build the btree-centric pathkey representation that describes the index's sort order. This is quite pointless for btree indexes: we might as well just use the index's opfamily information directly. That saves syscache lookup cycles during planning, and furthermore allows us to eliminate the relcache's caching of operators altogether, which may help in reducing backend startup time. I added code to plancat.c to perform the same type of double lookup on-the-fly if it's ever faced with a non-btree amcanorder index AM. If such a thing actually becomes interesting for production, we should replace that logic with some more-direct method for identifying the corresponding btree opfamily; but it's not worth spending effort on now. There is considerably more to do pursuant to my recent proposal to get rid of sort-operator-based representations of sort orderings, but this patch grabs some of the low-hanging fruit. I'll look at the remainder of that work after the current commitfest.
2010-11-29 18:29:42 +01:00
sortop,
When estimating the selectivity of an inequality "column > constant" or "column < constant", and the comparison value is in the first or last histogram bin or outside the histogram entirely, try to fetch the actual column min or max value using an index scan (if there is an index on the column). If successful, replace the lower or upper histogram bound with that value before carrying on with the estimate. This limits the estimation error caused by moving min/max values when the comparison value is close to the min or max. Per a complaint from Josh Berkus. It is tempting to consider using this mechanism for mergejoinscansel as well, but that would inject index fetches into main-line join estimation not just endpoint cases. I'm refraining from that until we can get a better handle on the costs of doing this type of lookup.
2010-01-04 03:44:40 +01:00
NULL,
NULL, NULL,
&numbers, &nnumbers))
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
{
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
double varCorrelation;
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
Assert(nnumbers == 1);
varCorrelation = numbers[0];
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Simplify and speed up mapping of index opfamilies to pathkeys. Formerly we looked up the operators associated with each index (caching them in relcache) and then the planner looked up the btree opfamily containing such operators in order to build the btree-centric pathkey representation that describes the index's sort order. This is quite pointless for btree indexes: we might as well just use the index's opfamily information directly. That saves syscache lookup cycles during planning, and furthermore allows us to eliminate the relcache's caching of operators altogether, which may help in reducing backend startup time. I added code to plancat.c to perform the same type of double lookup on-the-fly if it's ever faced with a non-btree amcanorder index AM. If such a thing actually becomes interesting for production, we should replace that logic with some more-direct method for identifying the corresponding btree opfamily; but it's not worth spending effort on now. There is considerably more to do pursuant to my recent proposal to get rid of sort-operator-based representations of sort orderings, but this patch grabs some of the low-hanging fruit. I'll look at the remainder of that work after the current commitfest.
2010-11-29 18:29:42 +01:00
if (index->reverse_sort[0])
varCorrelation = -varCorrelation;
Make use of statistics on index expressions. There are still some corner cases that could stand improvement, but it does all the basic stuff. A byproduct is that the selectivity routines are no longer constrained to working on simple Vars; we might in future be able to improve the behavior for subexpressions that don't match indexes.
2004-02-17 01:52:53 +01:00
if (index->ncolumns > 1)
*indexCorrelation = varCorrelation * 0.75;
else
*indexCorrelation = varCorrelation;
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
Document that get_attstatsslot/free_attstatsslot only need to be passed valid type information if they are asked to fetch the values part of a pg_statistic slot; these arguments are unneeded if fetching only the numbers part. Use this to save a catcache lookup in btcostestimate, which is looking like a bit of a hotspot in recent profiling. Not a big savings, but since it's essentially free, might as well do it.
2005-10-11 19:27:14 +02:00
free_attstatsslot(InvalidOid, NULL, 0, numbers, nnumbers);
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
}
}
Add hooks to let plugins override the planner's lookups in pg_statistic. Simon Riggs, with some editorialization by me.
2008-09-28 21:51:40 +02:00
ReleaseVariableStats(vardata);
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
PG_RETURN_VOID();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
hashcostestimate(PG_FUNCTION_ARGS)
There, now we support GiST...now what? :)
1996-08-26 08:32:06 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path = (IndexPath *) PG_GETARG_POINTER(1);
RelOptInfo *outer_rel = (RelOptInfo *) PG_GETARG_POINTER(2);
Cost *indexStartupCost = (Cost *) PG_GETARG_POINTER(3);
Cost *indexTotalCost = (Cost *) PG_GETARG_POINTER(4);
Selectivity *indexSelectivity = (Selectivity *) PG_GETARG_POINTER(5);
double *indexCorrelation = (double *) PG_GETARG_POINTER(6);
genericcostestimate(root, path, outer_rel, 0.0,
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
indexStartupCost, indexTotalCost,
indexSelectivity, indexCorrelation);
PG_RETURN_VOID();
There, now we support GiST...now what? :)
1996-08-26 08:32:06 +02:00
}
Latest round of fmgr updates. All functions with bool,char, or int2 inputs have been converted to newstyle. This should go a long way towards fixing our portability problems with platforms where char and short parameters are passed differently from int-width parameters. Still more to do for the Alpha port however.
2000-06-05 09:29:25 +02:00
Datum
gistcostestimate(PG_FUNCTION_ARGS)
There, now we support GiST...now what? :)
1996-08-26 08:32:06 +02:00
{
Remove planner's private fields from Query struct, and put them into a new PlannerInfo struct, which is passed around instead of the bare Query in all the planning code. This commit is essentially just a code-beautification exercise, but it does open the door to making larger changes to the planner data structures without having to muck with the widely-known Query struct.
2005-06-06 00:32:58 +02:00
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path = (IndexPath *) PG_GETARG_POINTER(1);
RelOptInfo *outer_rel = (RelOptInfo *) PG_GETARG_POINTER(2);
Cost *indexStartupCost = (Cost *) PG_GETARG_POINTER(3);
Cost *indexTotalCost = (Cost *) PG_GETARG_POINTER(4);
Selectivity *indexSelectivity = (Selectivity *) PG_GETARG_POINTER(5);
double *indexCorrelation = (double *) PG_GETARG_POINTER(6);
genericcostestimate(root, path, outer_rel, 0.0,
First cut at making indexscan cost estimates depend on correlation between index order and table order.
2001-05-10 01:13:37 +02:00
indexStartupCost, indexTotalCost,
indexSelectivity, indexCorrelation);
PG_RETURN_VOID();
There, now we support GiST...now what? :)
1996-08-26 08:32:06 +02:00
}
GIN: Generalized Inverted iNdex. text[], int4[], Tsearch2 support for GIN.
2006-05-02 13:28:56 +02:00
Add SP-GiST (space-partitioned GiST) index access method. SP-GiST is comparable to GiST in flexibility, but supports non-balanced partitioned search structures rather than balanced trees. As described at PGCon 2011, this new indexing structure can beat GiST in both index build time and query speed for search problems that it is well matched to. There are a number of areas that could still use improvement, but at this point the code seems committable. Teodor Sigaev and Oleg Bartunov, with considerable revisions by Tom Lane
2011-12-17 22:41:16 +01:00
Datum
spgcostestimate(PG_FUNCTION_ARGS)
{
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path = (IndexPath *) PG_GETARG_POINTER(1);
RelOptInfo *outer_rel = (RelOptInfo *) PG_GETARG_POINTER(2);
Cost *indexStartupCost = (Cost *) PG_GETARG_POINTER(3);
Cost *indexTotalCost = (Cost *) PG_GETARG_POINTER(4);
Selectivity *indexSelectivity = (Selectivity *) PG_GETARG_POINTER(5);
double *indexCorrelation = (double *) PG_GETARG_POINTER(6);
genericcostestimate(root, path, outer_rel, 0.0,
Add SP-GiST (space-partitioned GiST) index access method. SP-GiST is comparable to GiST in flexibility, but supports non-balanced partitioned search structures rather than balanced trees. As described at PGCon 2011, this new indexing structure can beat GiST in both index build time and query speed for search problems that it is well matched to. There are a number of areas that could still use improvement, but at this point the code seems committable. Teodor Sigaev and Oleg Bartunov, with considerable revisions by Tom Lane
2011-12-17 22:41:16 +01:00
indexStartupCost, indexTotalCost,
indexSelectivity, indexCorrelation);
PG_RETURN_VOID();
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/*
* Support routines for gincostestimate
*/
typedef struct
{
bool haveFullScan;
double partialEntries;
double exactEntries;
double searchEntries;
double arrayScans;
} GinQualCounts;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/* Find the index column matching "op"; return its index, or -1 if no match */
static int
find_index_column(Node *op, IndexOptInfo *index)
{
int i;
for (i = 0; i < index->ncolumns; i++)
{
if (match_index_to_operand(op, i, index))
return i;
}
return -1;
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/*
* Estimate the number of index terms that need to be searched for while
* testing the given GIN query, and increment the counts in *counts
* appropriately. If the query is unsatisfiable, return false.
*/
static bool
gincost_pattern(IndexOptInfo *index, int indexcol,
Oid clause_op, Datum query,
GinQualCounts *counts)
{
Oid extractProcOid;
int strategy_op;
Oid lefttype,
righttype;
int32 nentries = 0;
bool *partial_matches = NULL;
Pointer *extra_data = NULL;
bool *nullFlags = NULL;
int32 searchMode = GIN_SEARCH_MODE_DEFAULT;
int32 i;
/*
* Get the operator's strategy number and declared input data types
* within the index opfamily. (We don't need the latter, but we use
* get_op_opfamily_properties because it will throw error if it fails
* to find a matching pg_amop entry.)
*/
get_op_opfamily_properties(clause_op, index->opfamily[indexcol], false,
&strategy_op, &lefttype, &righttype);
/*
* GIN always uses the "default" support functions, which are those
* with lefttype == righttype == the opclass' opcintype (see
* IndexSupportInitialize in relcache.c).
*/
extractProcOid = get_opfamily_proc(index->opfamily[indexcol],
index->opcintype[indexcol],
index->opcintype[indexcol],
GIN_EXTRACTQUERY_PROC);
if (!OidIsValid(extractProcOid))
{
/* should not happen; throw same error as index_getprocinfo */
elog(ERROR, "missing support function %d for attribute %d of index \"%s\"",
GIN_EXTRACTQUERY_PROC, indexcol + 1,
get_rel_name(index->indexoid));
}
OidFunctionCall7(extractProcOid,
query,
PointerGetDatum(&nentries),
UInt16GetDatum(strategy_op),
PointerGetDatum(&partial_matches),
PointerGetDatum(&extra_data),
PointerGetDatum(&nullFlags),
PointerGetDatum(&searchMode));
if (nentries <= 0 && searchMode == GIN_SEARCH_MODE_DEFAULT)
{
/* No match is possible */
return false;
}
for (i = 0; i < nentries; i++)
{
/*
* For partial match we haven't any information to estimate number of
* matched entries in index, so, we just estimate it as 100
*/
if (partial_matches && partial_matches[i])
counts->partialEntries += 100;
else
counts->exactEntries++;
counts->searchEntries++;
}
if (searchMode == GIN_SEARCH_MODE_INCLUDE_EMPTY)
{
/* Treat "include empty" like an exact-match item */
counts->exactEntries++;
counts->searchEntries++;
}
else if (searchMode != GIN_SEARCH_MODE_DEFAULT)
{
/* It's GIN_SEARCH_MODE_ALL */
counts->haveFullScan = true;
}
return true;
}
/*
* Estimate the number of index terms that need to be searched for while
* testing the given GIN index clause, and increment the counts in *counts
* appropriately. If the query is unsatisfiable, return false.
*/
static bool
gincost_opexpr(IndexOptInfo *index, OpExpr *clause, GinQualCounts *counts)
{
Node *leftop = get_leftop((Expr *) clause);
Node *rightop = get_rightop((Expr *) clause);
Oid clause_op = clause->opno;
int indexcol;
Node *operand;
/* Locate the operand being compared to the index column */
if ((indexcol = find_index_column(leftop, index)) >= 0)
{
operand = rightop;
}
else if ((indexcol = find_index_column(rightop, index)) >= 0)
{
operand = leftop;
clause_op = get_commutator(clause_op);
}
else
{
elog(ERROR, "could not match index to operand");
operand = NULL; /* keep compiler quiet */
}
if (IsA(operand, RelabelType))
operand = (Node *) ((RelabelType *) operand)->arg;
/*
* It's impossible to call extractQuery method for unknown operand. So
* unless operand is a Const we can't do much; just assume there will
* be one ordinary search entry from the operand at runtime.
*/
if (!IsA(operand, Const))
{
counts->exactEntries++;
counts->searchEntries++;
return true;
}
/* If Const is null, there can be no matches */
if (((Const *) operand)->constisnull)
return false;
/* Otherwise, apply extractQuery and get the actual term counts */
return gincost_pattern(index, indexcol, clause_op,
((Const *) operand)->constvalue,
counts);
}
/*
* Estimate the number of index terms that need to be searched for while
* testing the given GIN index clause, and increment the counts in *counts
* appropriately. If the query is unsatisfiable, return false.
*
* A ScalarArrayOpExpr will give rise to N separate indexscans at runtime,
* each of which involves one value from the RHS array, plus all the
* non-array quals (if any). To model this, we average the counts across
* the RHS elements, and add the averages to the counts in *counts (which
* correspond to per-indexscan costs). We also multiply counts->arrayScans
* by N, causing gincostestimate to scale up its estimates accordingly.
*/
static bool
gincost_scalararrayopexpr(IndexOptInfo *index, ScalarArrayOpExpr *clause,
double numIndexEntries,
GinQualCounts *counts)
{
Node *leftop = (Node *) linitial(clause->args);
Node *rightop = (Node *) lsecond(clause->args);
Oid clause_op = clause->opno;
int indexcol;
ArrayType *arrayval;
int16 elmlen;
bool elmbyval;
char elmalign;
int numElems;
Datum *elemValues;
bool *elemNulls;
GinQualCounts arraycounts;
int numPossible = 0;
int i;
Assert(clause->useOr);
/* index column must be on the left */
if ((indexcol = find_index_column(leftop, index)) < 0)
elog(ERROR, "could not match index to operand");
if (IsA(rightop, RelabelType))
rightop = (Node *) ((RelabelType *) rightop)->arg;
/*
* It's impossible to call extractQuery method for unknown operand. So
* unless operand is a Const we can't do much; just assume there will
* be one ordinary search entry from each array entry at runtime, and
* fall back on a probably-bad estimate of the number of array entries.
*/
if (!IsA(rightop, Const))
{
counts->exactEntries++;
counts->searchEntries++;
counts->arrayScans *= estimate_array_length(rightop);
return true;
}
/* If Const is null, there can be no matches */
if (((Const *) rightop)->constisnull)
return false;
/* Otherwise, extract the array elements and iterate over them */
arrayval = DatumGetArrayTypeP(((Const *) rightop)->constvalue);
get_typlenbyvalalign(ARR_ELEMTYPE(arrayval),
&elmlen, &elmbyval, &elmalign);
deconstruct_array(arrayval,
ARR_ELEMTYPE(arrayval),
elmlen, elmbyval, elmalign,
&elemValues, &elemNulls, &numElems);
memset(&arraycounts, 0, sizeof(arraycounts));
for (i = 0; i < numElems; i++)
{
GinQualCounts elemcounts;
/* NULL can't match anything, so ignore, as the executor will */
if (elemNulls[i])
continue;
/* Otherwise, apply extractQuery and get the actual term counts */
memset(&elemcounts, 0, sizeof(elemcounts));
if (gincost_pattern(index, indexcol, clause_op, elemValues[i],
&elemcounts))
{
/* We ignore array elements that are unsatisfiable patterns */
numPossible++;
if (elemcounts.haveFullScan)
{
/*
* Full index scan will be required. We treat this as if
* every key in the index had been listed in the query; is
* that reasonable?
*/
elemcounts.partialEntries = 0;
elemcounts.exactEntries = numIndexEntries;
elemcounts.searchEntries = numIndexEntries;
}
arraycounts.partialEntries += elemcounts.partialEntries;
arraycounts.exactEntries += elemcounts.exactEntries;
arraycounts.searchEntries += elemcounts.searchEntries;
}
}
if (numPossible == 0)
{
/* No satisfiable patterns in the array */
return false;
}
/*
* Now add the averages to the global counts. This will give us an
* estimate of the average number of terms searched for in each indexscan,
* including contributions from both array and non-array quals.
*/
counts->partialEntries += arraycounts.partialEntries / numPossible;
counts->exactEntries += arraycounts.exactEntries / numPossible;
counts->searchEntries += arraycounts.searchEntries / numPossible;
counts->arrayScans *= numPossible;
return true;
}
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
* GIN has search behavior completely different from other index types
*/
GIN: Generalized Inverted iNdex. text[], int4[], Tsearch2 support for GIN.
2006-05-02 13:28:56 +02:00
Datum
gincostestimate(PG_FUNCTION_ARGS)
{
PlannerInfo *root = (PlannerInfo *) PG_GETARG_POINTER(0);
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
IndexPath *path = (IndexPath *) PG_GETARG_POINTER(1);
RelOptInfo *outer_rel = (RelOptInfo *) PG_GETARG_POINTER(2);
Cost *indexStartupCost = (Cost *) PG_GETARG_POINTER(3);
Cost *indexTotalCost = (Cost *) PG_GETARG_POINTER(4);
Selectivity *indexSelectivity = (Selectivity *) PG_GETARG_POINTER(5);
double *indexCorrelation = (double *) PG_GETARG_POINTER(6);
IndexOptInfo *index = path->indexinfo;
List *indexQuals = path->indexquals;
List *indexOrderBys = path->indexorderbys;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
ListCell *l;
List *selectivityQuals;
double numPages = index->pages,
numTuples = index->tuples;
double numEntryPages,
numDataPages,
numPendingPages,
numEntries;
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
GinQualCounts counts;
bool matchPossible;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
double entryPagesFetched,
dataPagesFetched,
dataPagesFetchedBySel;
double qual_op_cost,
qual_arg_cost,
spc_random_page_cost,
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
outer_scans;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
QualCost index_qual_cost;
Relation indexRel;
GinStatsData ginStats;
pgindent run for 8.3.
2007-11-15 22:14:46 +01:00
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
* Obtain statistic information from the meta page
*/
indexRel = index_open(index->indexoid, AccessShareLock);
ginGetStats(indexRel, &ginStats);
index_close(indexRel, AccessShareLock);
numEntryPages = ginStats.nEntryPages;
numDataPages = ginStats.nDataPages;
numPendingPages = ginStats.nPendingPages;
numEntries = ginStats.nEntries;
/*
* nPendingPages can be trusted, but the other fields are as of the last
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* VACUUM. Scale them by the ratio numPages / nTotalPages to account for
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
* growth since then. If the fields are zero (implying no VACUUM at all,
* and an index created pre-9.1), assume all pages are entry pages.
*/
if (ginStats.nTotalPages == 0 || ginStats.nEntryPages == 0)
{
numEntryPages = numPages;
numDataPages = 0;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
numEntries = numTuples; /* bogus, but no other info available */
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
else
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
double scale = numPages / ginStats.nTotalPages;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
numEntryPages = ceil(numEntryPages * scale);
numDataPages = ceil(numDataPages * scale);
numEntries = ceil(numEntries * scale);
/* ensure we didn't round up too much */
numEntryPages = Min(numEntryPages, numPages);
numDataPages = Min(numDataPages, numPages - numEntryPages);
}
Avoid possible divide-by-zero in gincostestimate. Per report from Jeff Janes.
2011-04-22 01:28:36 +02:00
/* In an empty index, numEntries could be zero. Avoid divide-by-zero */
if (numEntries < 1)
numEntries = 1;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Include predicate in selectivityQuals (should match
* genericcostestimate)
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
if (index->indpred != NIL)
{
List *predExtraQuals = NIL;
foreach(l, index->indpred)
{
Node *predQual = (Node *) lfirst(l);
List *oneQual = list_make1(predQual);
if (!predicate_implied_by(oneQual, indexQuals))
predExtraQuals = list_concat(predExtraQuals, oneQual);
}
Rethink representation of index clauses' mapping to index columns. In commit e2c2c2e8b1df7dfdb01e7e6f6191a569ce3c3195 I made use of nested list structures to show which clauses went with which index columns, but on reflection that's a data structure that only an old-line Lisp hacker could love. Worse, it adds unnecessary complication to the many places that don't much care which clauses go with which index columns. Revert to the previous arrangement of flat lists of clauses, and instead add a parallel integer list of column numbers. The places that care about the pairing can chase both lists with forboth(), while the places that don't care just examine one list the same as before. The only real downside to this is that there are now two more lists that need to be passed to amcostestimate functions in case they care about column matching (which btcostestimate does, so not passing the info is not an option). Rather than deal with 11-argument amcostestimate functions, pass just the IndexPath and expect the functions to extract fields from it. That gets us down to 7 arguments which is better than 11, and it seems more future-proof against likely additions to the information we keep about an index path.
2011-12-25 01:03:21 +01:00
/* list_concat avoids modifying the passed-in indexQuals list */
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
selectivityQuals = list_concat(predExtraQuals, indexQuals);
}
else
selectivityQuals = indexQuals;
/* Estimate the fraction of main-table tuples that will be visited */
*indexSelectivity = clauselist_selectivity(root, selectivityQuals,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
index->rel->relid,
JOIN_INNER,
NULL);
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/* fetch estimated page cost for schema containing index */
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
get_tablespace_page_costs(index->reltablespace,
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
&spc_random_page_cost,
NULL);
/*
* Generic assumption about index correlation: there isn't any.
*/
*indexCorrelation = 0.0;
/*
* Examine quals to estimate number of search entries & partial matches
*/
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
memset(&counts, 0, sizeof(counts));
counts.arrayScans = 1;
matchPossible = true;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
foreach(l, indexQuals)
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
RestrictInfo *rinfo = (RestrictInfo *) lfirst(l);
Expr *clause;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
Assert(IsA(rinfo, RestrictInfo));
clause = rinfo->clause;
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
if (IsA(clause, OpExpr))
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
{
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
matchPossible = gincost_opexpr(index,
(OpExpr *) clause,
&counts);
if (!matchPossible)
break;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
else if (IsA(clause, ScalarArrayOpExpr))
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
{
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
matchPossible = gincost_scalararrayopexpr(index,
(ScalarArrayOpExpr *) clause,
numEntries,
&counts);
if (!matchPossible)
break;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
else
{
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/* shouldn't be anything else for a GIN index */
elog(ERROR, "unsupported GIN indexqual type: %d",
(int) nodeTag(clause));
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
}
Fix up gincostestimate for new extractQuery API. The only reason this wasn't crashing while testing the core anyarray operators was that it was disabled for those cases because of passing the wrong type information to get_opfamily_proc :-(. So fix that too, and make it insist on finding the support proc --- in hindsight, silently doing nothing is not as sane a coping mechanism as all that.
2011-01-09 02:13:33 +01:00
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/* Fall out if there were any provably-unsatisfiable quals */
if (!matchPossible)
{
*indexStartupCost = 0;
*indexTotalCost = 0;
*indexSelectivity = 0;
PG_RETURN_VOID();
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
if (counts.haveFullScan || indexQuals == NIL)
Fix up gincostestimate for new extractQuery API. The only reason this wasn't crashing while testing the core anyarray operators was that it was disabled for those cases because of passing the wrong type information to get_opfamily_proc :-(. So fix that too, and make it insist on finding the support proc --- in hindsight, silently doing nothing is not as sane a coping mechanism as all that.
2011-01-09 02:13:33 +01:00
{
/*
* Full index scan will be required. We treat this as if every key in
* the index had been listed in the query; is that reasonable?
*/
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
counts.partialEntries = 0;
counts.exactEntries = numEntries;
counts.searchEntries = numEntries;
Fix up gincostestimate for new extractQuery API. The only reason this wasn't crashing while testing the core anyarray operators was that it was disabled for those cases because of passing the wrong type information to get_opfamily_proc :-(. So fix that too, and make it insist on finding the support proc --- in hindsight, silently doing nothing is not as sane a coping mechanism as all that.
2011-01-09 02:13:33 +01:00
}
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/* Will we have more than one iteration of a nestloop scan? */
if (outer_rel != NULL && outer_rel->rows > 1)
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
outer_scans = outer_rel->rows;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
else
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
outer_scans = 1;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
* Compute cost to begin scan, first of all, pay attention to pending list.
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
entryPagesFetched = numPendingPages;
/*
* Estimate number of entry pages read. We need to do
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
* counts.searchEntries searches. Use a power function as it should be,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* but tuples on leaf pages usually is much greater. Here we include all
* searches in entry tree, including search of first entry in partial
* match algorithm
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
entryPagesFetched += ceil(counts.searchEntries * rint(pow(numEntryPages, 0.15)));
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Add an estimate of entry pages read by partial match algorithm. It's a
* scan over leaf pages in entry tree. We haven't any useful stats here,
* so estimate it as proportion.
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
entryPagesFetched += ceil(numEntryPages * counts.partialEntries / numEntries);
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Partial match algorithm reads all data pages before doing actual scan,
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
* so it's a startup cost. Again, we haven't any useful stats here, so,
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* estimate it as proportion
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
dataPagesFetched = ceil(numDataPages * counts.partialEntries / numEntries);
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/*
* Calculate cache effects if more than one scan due to nestloops or array
* quals. The result is pro-rated per nestloop scan, but the array qual
* factor shouldn't be pro-rated (compare genericcostestimate).
*/
if (outer_scans > 1 || counts.arrayScans > 1)
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
{
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
entryPagesFetched *= outer_scans * counts.arrayScans;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
entryPagesFetched = index_pages_fetched(entryPagesFetched,
(BlockNumber) numEntryPages,
numEntryPages, root);
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
entryPagesFetched /= outer_scans;
dataPagesFetched *= outer_scans * counts.arrayScans;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
dataPagesFetched = index_pages_fetched(dataPagesFetched,
(BlockNumber) numDataPages,
numDataPages, root);
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
dataPagesFetched /= outer_scans;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
}
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* Here we use random page cost because logically-close pages could be far
* apart on disk.
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
*indexStartupCost = (entryPagesFetched + dataPagesFetched) * spc_random_page_cost;
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/*
* Now we compute the number of data pages fetched while the scan proceeds.
*/
/* data pages scanned for each exact (non-partial) matched entry */
dataPagesFetched = ceil(numDataPages * counts.exactEntries / numEntries);
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
/*
* Estimate number of data pages read, using selectivity estimation and
* capacity of data page.
*/
dataPagesFetchedBySel = ceil(*indexSelectivity *
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
(numTuples / (BLCKSZ / SizeOfIptrData)));
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
if (dataPagesFetchedBySel > dataPagesFetched)
{
/*
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
* At least one of entries is very frequent and, unfortunately, we
* couldn't get statistic about entries (only tsvector has such
* statistics). So, we obviously have too small estimation of pages
* fetched from data tree. Re-estimate it from known capacity of data
* pages
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*/
dataPagesFetched = dataPagesFetchedBySel;
}
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
/* Account for cache effects, the same as above */
if (outer_scans > 1 || counts.arrayScans > 1)
{
dataPagesFetched *= outer_scans * counts.arrayScans;
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
dataPagesFetched = index_pages_fetched(dataPagesFetched,
(BlockNumber) numDataPages,
numDataPages, root);
Fix gincostestimate to handle ScalarArrayOpExpr reasonably. The original coding of this function overlooked the possibility that it could be passed anything except simple OpExpr indexquals. But ScalarArrayOpExpr is possible too, and the code would probably crash (and surely give ridiculous answers) in such a case. Add logic to try to estimate sanely for such cases. In passing, fix the treatment of inner-indexscan cost estimation: it was failing to scale up properly for multiple iterations of a nestloop. (I think somebody might've thought that index_pages_fetched() is linear, but of course it's not.) Report, diagnosis, and preliminary patch by Marti Raudsepp; I refactored it a bit and fixed the cost estimation. Back-patch into 9.1 where the bogus code was introduced.
2011-12-21 01:57:34 +01:00
dataPagesFetched /= outer_scans;
}
/* And apply random_page_cost as the cost per page */
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
*indexTotalCost = *indexStartupCost +
dataPagesFetched * spc_random_page_cost;
/*
* Add on index qual eval costs, much as in genericcostestimate
*/
cost_qual_eval(&index_qual_cost, indexQuals, root);
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
qual_arg_cost = index_qual_cost.startup + index_qual_cost.per_tuple;
cost_qual_eval(&index_qual_cost, indexOrderBys, root);
qual_arg_cost += index_qual_cost.startup + index_qual_cost.per_tuple;
qual_op_cost = cpu_operator_cost *
(list_length(indexQuals) + list_length(indexOrderBys));
qual_arg_cost -= qual_op_cost;
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
if (qual_arg_cost < 0) /* just in case... */
Improve GIN indexscan cost estimation. The better estimate requires more statistics than we previously stored: in particular, counts of "entry" versus "data" pages within the index, as well as knowledge of the number of distinct key values. We collect this information during initial index build and update it during VACUUM, storing the info in new fields on the index metapage. No initdb is required because these fields will read as zeroes in a pre-existing index, and the new gincostestimate code is coded to behave (reasonably) sanely if they are zeroes. Teodor Sigaev, reviewed by Jan Urbanski, Tom Lane, and Itagaki Takahiro.
2010-10-18 02:52:32 +02:00
qual_arg_cost = 0;
*indexStartupCost += qual_arg_cost;
*indexTotalCost += qual_arg_cost;
Create core infrastructure for KNNGIST. This is a heavily revised version of builtin_knngist_core-0.9. The ordering operators are no longer mixed in with actual quals, which would have confused not only humans but significant parts of the planner. Instead, ordering operators are carried separately throughout planning and execution. Since the API for ambeginscan and amrescan functions had to be changed anyway, this commit takes the opportunity to rationalize that a bit. RelationGetIndexScan no longer forces a premature index_rescan call; instead, callers of index_beginscan must call index_rescan too. Aside from making the AM-side initialization logic a bit less peculiar, this has the advantage that we do not make a useless extra am_rescan call when there are runtime key values. AMs formerly could not assume that the key values passed to amrescan were actually valid; now they can. Teodor Sigaev and Tom Lane
2010-12-03 02:50:48 +01:00
*indexTotalCost += (numTuples * *indexSelectivity) * (cpu_index_tuple_cost + qual_op_cost);
GIN: Generalized Inverted iNdex. text[], int4[], Tsearch2 support for GIN.
2006-05-02 13:28:56 +02:00
PG_RETURN_VOID();
}