2010-09-20 22:08:53 +02:00
<!-- doc/src/sgml/indices.sgml -->
2001-02-20 23:27:56 +01:00
2001-05-17 23:50:18 +02:00
<chapter id="indexes">
2010-04-03 09:23:02 +02:00
<title>Indexes</title>
2001-02-20 23:27:56 +01:00
2001-05-17 23:50:18 +02:00
<indexterm zone="indexes">
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-02-20 23:27:56 +01:00
<para>
2001-05-17 23:50:18 +02:00
Indexes are a common way to enhance database performance. An index
2001-02-20 23:27:56 +01:00
allows the database server to find and retrieve specific rows much
2001-05-17 23:50:18 +02:00
faster than it could do without an index. But indexes also add
2001-02-20 23:27:56 +01:00
overhead to the database system as a whole, so they should be used
sensibly.
</para>
2001-05-17 23:50:18 +02:00
<sect1 id="indexes-intro">
2001-02-20 23:27:56 +01:00
<title>Introduction</title>
1998-10-14 18:20:16 +02:00
1999-07-22 17:11:05 +02:00
<para>
2003-05-28 18:04:02 +02:00
Suppose we have a table similar to this:
2001-02-20 23:27:56 +01:00
<programlisting>
CREATE TABLE test1 (
id integer,
content varchar
);
</programlisting>
2009-04-27 18:27:36 +02:00
and the application issues many queries of the form:
2001-02-20 23:27:56 +01:00
<programlisting>
SELECT content FROM test1 WHERE id = <replaceable>constant</replaceable>;
</programlisting>
2003-05-28 18:04:02 +02:00
With no advance preparation, the system would have to scan the entire
<structname>test1</structname> table, row by row, to find all
2009-04-27 18:27:36 +02:00
matching entries. If there are many rows in
<structname>test1</structname> and only a few rows (perhaps zero
or one) that would be returned by such a query, this is clearly an
2009-06-17 23:58:49 +02:00
inefficient method. But if the system has been instructed to maintain an
2009-04-27 18:27:36 +02:00
index on the <structfield>id</structfield> column, it can use a more
2001-02-20 23:27:56 +01:00
efficient method for locating matching rows. For instance, it
might only have to walk a few levels deep into a search tree.
1999-07-22 17:11:05 +02:00
</para>
1998-10-14 18:20:16 +02:00
2000-03-26 21:47:17 +02:00
<para>
2009-04-27 18:27:36 +02:00
A similar approach is used in most non-fiction books: terms and
2001-02-20 23:27:56 +01:00
concepts that are frequently looked up by readers are collected in
an alphabetic index at the end of the book. The interested reader
can scan the index relatively quickly and flip to the appropriate
2003-05-28 18:04:02 +02:00
page(s), rather than having to read the entire book to find the
material of interest. Just as it is the task of the author to
2009-04-27 18:27:36 +02:00
anticipate the items that readers are likely to look up,
2001-02-20 23:27:56 +01:00
it is the task of the database programmer to foresee which indexes
2009-04-27 18:27:36 +02:00
will be useful.
2000-03-26 21:47:17 +02:00
</para>
1999-07-22 17:11:05 +02:00
<para>
2009-04-27 18:27:36 +02:00
The following command can be used to create an index on the
2001-02-20 23:27:56 +01:00
<structfield>id</structfield> column, as discussed:
<programlisting>
CREATE INDEX test1_id_index ON test1 (id);
</programlisting>
The name <structname>test1_id_index</structname> can be chosen
freely, but you should pick something that enables you to remember
later what the index was for.
</para>
1998-10-14 18:20:16 +02:00
2001-02-20 23:27:56 +01:00
<para>
To remove an index, use the <command>DROP INDEX</command> command.
2001-10-31 21:38:26 +01:00
Indexes can be added to and removed from tables at any time.
2001-02-20 23:27:56 +01:00
</para>
1999-07-22 17:11:05 +02:00
2001-02-20 23:27:56 +01:00
<para>
2004-12-24 00:07:38 +01:00
Once an index is created, no further intervention is required: the
2003-05-28 18:04:02 +02:00
system will update the index when the table is modified, and it will
2009-06-17 23:58:49 +02:00
use the index in queries when it thinks doing so would be more efficient
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
than a sequential table scan. But you might have to run the
2001-05-07 02:43:27 +02:00
<command>ANALYZE</command> command regularly to update
2001-02-20 23:27:56 +01:00
statistics to allow the query planner to make educated decisions.
2017-11-23 15:39:47 +01:00
See <xref linkend="performance-tips"/> for information about
2001-02-20 23:27:56 +01:00
how to find out whether an index is used and when and why the
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
planner might choose <emphasis>not</emphasis> to use an index.
1999-07-22 17:11:05 +02:00
</para>
<para>
2003-03-13 02:30:29 +01:00
Indexes can also benefit <command>UPDATE</command> and
2005-09-12 21:17:45 +02:00
<command>DELETE</command> commands with search conditions.
Indexes can moreover be used in join searches. Thus,
2001-02-20 23:27:56 +01:00
an index defined on a column that is part of a join condition can
2009-04-27 18:27:36 +02:00
also significantly speed up queries with joins.
1999-07-22 17:11:05 +02:00
</para>
2006-08-25 06:06:58 +02:00
<para>
Creating an index on a large table can take a long time. By default,
2010-08-17 06:37:21 +02:00
<productname>PostgreSQL</productname> allows reads (<command>SELECT</command> statements) to occur
on the table in parallel with index creation, but writes (<command>INSERT</command>,
<command>UPDATE</command>, <command>DELETE</command>) are blocked until the index build is finished.
2006-09-14 01:42:26 +02:00
In production environments this is often unacceptable.
2006-08-25 06:06:58 +02:00
It is possible to allow writes to occur in parallel with index
creation, but there are several caveats to be aware of —
Doc: fix "Unresolved ID reference" warnings, clean up man page cross-refs.
Use xreflabel attributes instead of endterm attributes to control the
appearance of links to subsections of SQL command reference pages.
This is simpler, it matches what we do elsewhere (e.g. for GUC variables),
and it doesn't draw "Unresolved ID reference" warnings from the PDF
toolchain.
Fix some places where the text was absolutely dependent on an <xref>
rendering exactly so, by using a <link> around the required text
instead. At least one of those spots had already been turned into
bad grammar by subsequent changes, and the whole idea is just too
fragile for my taste. <xref> does NOT have fixed output, don't write
as if it does.
Consistently include a page-level link in cross-man-page references,
because otherwise they are useless/nonsensical in man-page output.
Likewise, be consistent about mentioning "below" or "above" in same-page
references; we were doing that in about 90% of the cases, but now it's
100%.
Also get rid of another nonfunctional-in-PDF idea, of making
cross-references to functions by sticking ID tags on <row> constructs.
We can put the IDs on <indexterm>s instead --- which is probably not any
more sensible in abstract terms, but it works where the other doesn't.
(There is talk of attaching cross-reference IDs to most or all of
the docs' function descriptions, but for now I just fixed the two
that exist.)
Discussion: https://postgr.es/m/14480.1589154358@sss.pgh.pa.us
2020-05-11 20:15:49 +02:00
for more information see <xref linkend="sql-createindex-concurrently"/>.
2006-08-25 06:06:58 +02:00
</para>
1999-07-22 17:11:05 +02:00
<para>
2005-09-12 21:17:45 +02:00
After an index is created, the system has to keep it synchronized with the
2001-02-20 23:27:56 +01:00
table. This adds overhead to data manipulation operations.
2005-09-12 21:17:45 +02:00
Therefore indexes that are seldom or never used in queries
should be removed.
2001-02-20 23:27:56 +01:00
</para>
</sect1>
2001-05-17 23:50:18 +02:00
<sect1 id="indexes-types">
2001-02-20 23:27:56 +01:00
<title>Index Types</title>
<para>
2001-11-21 06:53:41 +01:00
<productname>PostgreSQL</productname> provides several index types:
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
B-tree, Hash, GiST, SP-GiST, GIN and BRIN.
Each index type uses a different
2003-11-06 23:21:47 +01:00
algorithm that is best suited to different types of queries.
2020-11-25 20:04:28 +01:00
By default, the <link linkend="sql-createindex"><command>CREATE
INDEX</command></link> command creates
2009-04-27 18:27:36 +02:00
B-tree indexes, which fit the most common situations.
2020-11-25 20:04:28 +01:00
The other index types are selected by writing the keyword
<literal>USING</literal> followed by the index type name.
For example, to create a Hash index:
<programlisting>
CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> USING HASH (<replaceable>column</replaceable>);
</programlisting>
2005-01-08 23:13:38 +01:00
</para>
2020-11-25 20:04:28 +01:00
<sect2 id="indexes-types-btree">
<title>B-Tree</title>
2001-05-13 00:51:36 +02:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2020-11-25 20:04:28 +01:00
<secondary>B-Tree</secondary>
2001-05-13 00:51:36 +02:00
</indexterm>
<indexterm>
2020-11-25 20:04:28 +01:00
<primary>B-Tree</primary>
2003-08-31 19:32:24 +02:00
<see>index</see>
2001-05-13 00:51:36 +02:00
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
2005-01-08 23:13:38 +01:00
B-trees can handle equality and range queries on data that can be sorted
into some ordering.
In particular, the <productname>PostgreSQL</productname> query planner
2001-02-20 23:27:56 +01:00
will consider using a B-tree index whenever an indexed column is
involved in a comparison using one of these operators:
1999-07-22 17:11:05 +02:00
2020-11-25 20:04:28 +01:00
<synopsis>
< <= = >= >
</synopsis>
2005-01-08 23:13:38 +01:00
Constructs equivalent to combinations of these operators, such as
2017-10-09 03:44:17 +02:00
<literal>BETWEEN</literal> and <literal>IN</literal>, can also be implemented with
a B-tree index search. Also, an <literal>IS NULL</literal> or <literal>IS NOT
NULL</literal> condition on an index column can be used with a B-tree index.
1999-07-22 17:11:05 +02:00
</para>
2003-05-15 17:50:21 +02:00
<para>
The optimizer can also use a B-tree index for queries involving the
2017-10-09 03:44:17 +02:00
pattern matching operators <literal>LIKE</literal> and <literal>~</literal>
2006-01-18 22:29:45 +01:00
<emphasis>if</emphasis> the pattern is a constant and is anchored to
the beginning of the string — for example, <literal>col LIKE
'foo%'</literal> or <literal>col ~ '^foo'</literal>, but not
2008-09-23 11:20:39 +02:00
<literal>col LIKE '%bar'</literal>. However, if your database does not
2006-01-18 22:29:45 +01:00
use the C locale you will need to create the index with a special
2009-04-27 18:27:36 +02:00
operator class to support indexing of pattern-matching queries; see
2017-11-23 15:39:47 +01:00
<xref linkend="indexes-opclass"/> below. It is also possible to use
2006-01-18 22:29:45 +01:00
B-tree indexes for <literal>ILIKE</literal> and
<literal>~*</literal>, but only if the pattern starts with
2009-04-27 18:27:36 +02:00
non-alphabetic characters, i.e., characters that are not affected by
2006-01-18 22:29:45 +01:00
upper/lower case conversion.
2003-05-15 17:50:21 +02:00
</para>
2010-12-04 05:49:06 +01:00
<para>
B-tree indexes can also be used to retrieve data in sorted order.
This is not always faster than a simple scan and sort, but it is
often helpful.
</para>
2020-11-25 20:04:28 +01:00
</sect2>
<sect2 id="indexes-types-hash">
<title>Hash</title>
2006-01-18 22:29:45 +01:00
2001-05-13 00:51:36 +02:00
<indexterm>
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2001-05-13 00:51:36 +02:00
<secondary>hash</secondary>
</indexterm>
<indexterm>
<primary>hash</primary>
2003-08-31 19:32:24 +02:00
<see>index</see>
2001-05-13 00:51:36 +02:00
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
Hash indexes store a 32-bit hash code derived from the
value of the indexed column. Hence,
such indexes can only handle simple equality comparisons.
2003-03-13 02:30:29 +01:00
The query planner will consider using a hash index whenever an
2001-02-20 23:27:56 +01:00
indexed column is involved in a comparison using the
2020-11-25 20:04:28 +01:00
equal operator:
2001-02-20 23:27:56 +01:00
<synopsis>
2020-11-25 20:04:28 +01:00
=
2001-02-20 23:27:56 +01:00
</synopsis>
</para>
2020-11-25 20:04:28 +01:00
</sect2>
<sect2 id="indexes-type-gist">
<title>GiST</title>
2001-02-20 23:27:56 +01:00
2005-11-07 18:36:47 +01:00
<indexterm>
<primary>index</primary>
<secondary>GiST</secondary>
</indexterm>
<indexterm>
<primary>GiST</primary>
<see>index</see>
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
2005-11-07 18:36:47 +01:00
GiST indexes are not a single kind of index, but rather an infrastructure
within which many different indexing strategies can be implemented.
Accordingly, the particular operators with which a GiST index can be
used vary depending on the indexing strategy (the <firstterm>operator
2017-10-09 03:44:17 +02:00
class</firstterm>). As an example, the standard distribution of
2005-11-07 18:36:47 +01:00
<productname>PostgreSQL</productname> includes GiST operator classes
for several two-dimensional geometric data types, which support indexed
queries using these operators:
2005-10-21 03:41:28 +02:00
2020-11-25 20:04:28 +01:00
<synopsis>
<< &< &> >> <<| &<| |&> |>> @> <@ ~= &&
</synopsis>
2005-11-07 18:36:47 +01:00
2017-11-23 15:39:47 +01:00
(See <xref linkend="functions-geometry"/> for the meaning of
2005-11-07 18:36:47 +01:00
these operators.)
Add an in-core GiST index opclass for inet/cidr types.
This operator class can accelerate subnet/supernet tests as well as
btree-equivalent ordered comparisons. It also handles a new network
operator inet && inet (overlaps, a/k/a "is supernet or subnet of"),
which is expected to be useful in exclusion constraints.
Ideally this opclass would be the default for GiST with inet/cidr data,
but we can't mark it that way until we figure out how to do a more or
less graceful transition from the current situation, in which the
really-completely-bogus inet/cidr opclasses in contrib/btree_gist are
marked as default. Having the opclass in core and not default is better
than not having it at all, though.
While at it, add new documentation sections to allow us to officially
document GiST/GIN/SP-GiST opclasses, something there was never a clear
place to do before. I filled these in with some simple tables listing
the existing opclasses and the operators they support, but there's
certainly scope to put more information there.
Emre Hasegeli, reviewed by Andreas Karlsson, further hacking by me
2014-04-08 21:46:14 +02:00
The GiST operator classes included in the standard distribution are
2017-11-23 15:39:47 +01:00
documented in <xref linkend="gist-builtin-opclasses-table"/>.
2005-11-07 18:36:47 +01:00
Many other GiST operator
2017-10-09 03:44:17 +02:00
classes are available in the <literal>contrib</literal> collection or as separate
2017-11-23 15:39:47 +01:00
projects. For more information see <xref linkend="gist"/>.
2005-11-07 18:36:47 +01:00
</para>
2010-12-04 05:49:06 +01:00
<para>
2017-10-09 03:44:17 +02:00
GiST indexes are also capable of optimizing <quote>nearest-neighbor</quote>
2010-12-04 05:49:06 +01:00
searches, such as
<programlisting><![CDATA[
SELECT * FROM places ORDER BY location <-> point '(101,456)' LIMIT 10;
]]>
</programlisting>
which finds the ten places closest to a given target point. The ability
to do this is again dependent on the particular operator class being used.
2017-11-23 15:39:47 +01:00
In <xref linkend="gist-builtin-opclasses-table"/>, operators that can be
2017-10-09 03:44:17 +02:00
used in this way are listed in the column <quote>Ordering Operators</quote>.
2010-12-04 05:49:06 +01:00
</para>
2020-11-25 20:04:28 +01:00
</sect2>
<sect2 id="indexes-type-spgist">
<title>SP-GiST</title>
2010-12-04 05:49:06 +01:00
2011-12-17 22:41:16 +01:00
<indexterm>
<primary>index</primary>
<secondary>SP-GiST</secondary>
</indexterm>
<indexterm>
<primary>SP-GiST</primary>
<see>index</see>
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
2011-12-17 22:41:16 +01:00
SP-GiST indexes, like GiST indexes, offer an infrastructure that supports
various kinds of searches. SP-GiST permits implementation of a wide range
of different non-balanced disk-based data structures, such as quadtrees,
2013-05-08 13:29:28 +02:00
k-d trees, and radix trees (tries). As an example, the standard distribution of
2011-12-17 22:41:16 +01:00
<productname>PostgreSQL</productname> includes SP-GiST operator classes
for two-dimensional points, which support indexed
queries using these operators:
2020-11-25 20:04:28 +01:00
<synopsis>
<< >> ~= <@ <<| |>>
</synopsis>
2011-12-17 22:41:16 +01:00
2017-11-23 15:39:47 +01:00
(See <xref linkend="functions-geometry"/> for the meaning of
2011-12-17 22:41:16 +01:00
these operators.)
Add an in-core GiST index opclass for inet/cidr types.
This operator class can accelerate subnet/supernet tests as well as
btree-equivalent ordered comparisons. It also handles a new network
operator inet && inet (overlaps, a/k/a "is supernet or subnet of"),
which is expected to be useful in exclusion constraints.
Ideally this opclass would be the default for GiST with inet/cidr data,
but we can't mark it that way until we figure out how to do a more or
less graceful transition from the current situation, in which the
really-completely-bogus inet/cidr opclasses in contrib/btree_gist are
marked as default. Having the opclass in core and not default is better
than not having it at all, though.
While at it, add new documentation sections to allow us to officially
document GiST/GIN/SP-GiST opclasses, something there was never a clear
place to do before. I filled these in with some simple tables listing
the existing opclasses and the operators they support, but there's
certainly scope to put more information there.
Emre Hasegeli, reviewed by Andreas Karlsson, further hacking by me
2014-04-08 21:46:14 +02:00
The SP-GiST operator classes included in the standard distribution are
2017-11-23 15:39:47 +01:00
documented in <xref linkend="spgist-builtin-opclasses-table"/>.
For more information see <xref linkend="spgist"/>.
2011-12-17 22:41:16 +01:00
</para>
2018-09-19 00:54:10 +02:00
<para>
Like GiST, SP-GiST supports <quote>nearest-neighbor</quote> searches.
2018-11-22 19:24:57 +01:00
For SP-GiST operator classes that support distance ordering, the
2020-11-25 20:04:28 +01:00
corresponding operator is listed in the <quote>Ordering Operators</quote>
2018-09-19 00:54:10 +02:00
column in <xref linkend="spgist-builtin-opclasses-table"/>.
</para>
2020-11-25 20:04:28 +01:00
</sect2>
<sect2 id="indexes-types-gin">
<title>GIN</title>
2018-09-19 00:54:10 +02:00
2006-09-14 13:16:27 +02:00
<indexterm>
<primary>index</primary>
<secondary>GIN</secondary>
</indexterm>
<indexterm>
<primary>GIN</primary>
<see>index</see>
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
2017-10-09 03:44:17 +02:00
GIN indexes are <quote>inverted indexes</quote> which are appropriate for
2016-05-08 22:36:19 +02:00
data values that contain multiple component values, such as arrays. An
inverted index contains a separate entry for each component value, and
can efficiently handle queries that test for the presence of specific
component values.
</para>
<para>
Like GiST and SP-GiST, GIN can support
many different user-defined indexing strategies, and the particular
2008-07-11 23:06:29 +02:00
operators with which a GIN index can be used vary depending on the
indexing strategy.
2006-09-14 13:16:27 +02:00
As an example, the standard distribution of
Replace the built-in GIN array opclasses with a single polymorphic opclass.
We had thirty different GIN array opclasses sharing the same operators and
support functions. That still didn't cover all the built-in types, nor
did it cover arrays of extension-added types. What we want is a single
polymorphic opclass for "anyarray". There were two missing features needed
to make this possible:
1. We have to be able to declare the index storage type as ANYELEMENT
when the opclass is declared to index ANYARRAY. This just takes a few
more lines in index_create(). Although this currently seems of use only
for GIN, there's no reason to make index_create() restrict it to that.
2. We have to be able to identify the proper GIN compare function for
the index storage type. This patch proceeds by making the compare function
optional in GIN opclass definitions, and specifying that the default btree
comparison function for the index storage type will be looked up when the
opclass omits it. Again, that seems pretty generically useful.
Since the comparison function lookup is done in initGinState(), making
use of the second feature adds an additional cache lookup to GIN index
access setup. It seems unlikely that that would be very noticeable given
the other costs involved, but maybe at some point we should consider
making GinState data persist longer than it now does --- we could keep it
in the index relcache entry, perhaps.
Rather fortuitously, we don't seem to need to do anything to get this
change to play nice with dump/reload or pg_upgrade scenarios: the new
opclass definition is automatically selected to replace existing index
definitions, and the on-disk data remains compatible. Also, if a user has
created a custom opclass definition for a non-builtin type, this doesn't
break that, since CREATE INDEX will prefer an exact match to opcintype
over a match to ANYARRAY. However, if there's anyone out there with
handwritten DDL that explicitly specifies _bool_ops or one of the other
replaced opclass names, they'll need to adjust that.
Tom Lane, reviewed by Enrique Meneses
Discussion: <14436.1470940379@sss.pgh.pa.us>
2016-09-26 20:52:44 +02:00
<productname>PostgreSQL</productname> includes a GIN operator class
for arrays, which supports indexed queries using these operators:
2006-09-14 13:16:27 +02:00
2020-11-25 20:04:28 +01:00
<synopsis>
<@ @> = &&
</synopsis>
2006-09-14 13:16:27 +02:00
2017-11-23 15:39:47 +01:00
(See <xref linkend="functions-array"/> for the meaning of
2006-09-14 13:16:27 +02:00
these operators.)
Add an in-core GiST index opclass for inet/cidr types.
This operator class can accelerate subnet/supernet tests as well as
btree-equivalent ordered comparisons. It also handles a new network
operator inet && inet (overlaps, a/k/a "is supernet or subnet of"),
which is expected to be useful in exclusion constraints.
Ideally this opclass would be the default for GiST with inet/cidr data,
but we can't mark it that way until we figure out how to do a more or
less graceful transition from the current situation, in which the
really-completely-bogus inet/cidr opclasses in contrib/btree_gist are
marked as default. Having the opclass in core and not default is better
than not having it at all, though.
While at it, add new documentation sections to allow us to officially
document GiST/GIN/SP-GiST opclasses, something there was never a clear
place to do before. I filled these in with some simple tables listing
the existing opclasses and the operators they support, but there's
certainly scope to put more information there.
Emre Hasegeli, reviewed by Andreas Karlsson, further hacking by me
2014-04-08 21:46:14 +02:00
The GIN operator classes included in the standard distribution are
2017-11-23 15:39:47 +01:00
documented in <xref linkend="gin-builtin-opclasses-table"/>.
2007-11-14 00:36:26 +01:00
Many other GIN operator
2017-10-09 03:44:17 +02:00
classes are available in the <literal>contrib</literal> collection or as separate
2017-11-23 15:39:47 +01:00
projects. For more information see <xref linkend="gin"/>.
2006-09-14 13:16:27 +02:00
</para>
2020-11-25 20:04:28 +01:00
</sect2>
<sect2 id="indexes-types-brin">
<title>BRIN</title>
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
<indexterm>
<primary>index</primary>
<secondary>BRIN</secondary>
</indexterm>
<indexterm>
<primary>BRIN</primary>
<see>index</see>
</indexterm>
2020-11-25 20:04:28 +01:00
<para>
2016-05-08 22:36:19 +02:00
BRIN indexes (a shorthand for Block Range INdexes) store summaries about
the values stored in consecutive physical block ranges of a table.
2020-11-25 20:04:28 +01:00
Thus, they are most effective for columns whose values are well-correlated
with the physical order of the table rows.
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
Like GiST, SP-GiST and GIN,
BRIN can support many different indexing strategies,
and the particular operators with which a BRIN index can be used
vary depending on the indexing strategy.
2015-09-11 03:22:21 +02:00
For data types that have a linear sort order, the indexed data
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
corresponds to the minimum and maximum values of the
2016-05-08 22:36:19 +02:00
values in the column for each block range. This supports indexed queries
using these operators:
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
2020-11-25 20:04:28 +01:00
<synopsis>
< <= = >= >
</synopsis>
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
The BRIN operator classes included in the standard distribution are
2017-11-23 15:39:47 +01:00
documented in <xref linkend="brin-builtin-opclasses-table"/>.
For more information see <xref linkend="brin"/>.
BRIN: Block Range Indexes
BRIN is a new index access method intended to accelerate scans of very
large tables, without the maintenance overhead of btrees or other
traditional indexes. They work by maintaining "summary" data about
block ranges. Bitmap index scans work by reading each summary tuple and
comparing them with the query quals; all pages in the range are returned
in a lossy TID bitmap if the quals are consistent with the values in the
summary tuple, otherwise not. Normal index scans are not supported
because these indexes do not store TIDs.
As new tuples are added into the index, the summary information is
updated (if the block range in which the tuple is added is already
summarized) or not; in the latter case, a subsequent pass of VACUUM or
the brin_summarize_new_values() function will create the summary
information.
For data types with natural 1-D sort orders, the summary info consists
of the maximum and the minimum values of each indexed column within each
page range. This type of operator class we call "Minmax", and we
supply a bunch of them for most data types with B-tree opclasses.
Since the BRIN code is generalized, other approaches are possible for
things such as arrays, geometric types, ranges, etc; even for things
such as enum types we could do something different than minmax with
better results. In this commit I only include minmax.
Catalog version bumped due to new builtin catalog entries.
There's more that could be done here, but this is a good step forwards.
Loosely based on ideas from Simon Riggs; code mostly by Álvaro Herrera,
with contribution by Heikki Linnakangas.
Patch reviewed by: Amit Kapila, Heikki Linnakangas, Robert Haas.
Testing help from Jeff Janes, Erik Rijkers, Emanuel Calvo.
PS:
The research leading to these results has received funding from the
European Union's Seventh Framework Programme (FP7/2007-2013) under
grant agreement n° 318633.
2014-11-07 20:38:14 +01:00
</para>
2020-11-25 20:04:28 +01:00
</sect2>
2001-02-20 23:27:56 +01:00
</sect1>
2001-05-17 23:50:18 +02:00
<sect1 id="indexes-multicolumn">
2001-10-31 21:38:26 +01:00
<title>Multicolumn Indexes</title>
2001-02-20 23:27:56 +01:00
2001-05-17 23:50:18 +02:00
<indexterm zone="indexes-multicolumn">
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2001-10-31 21:38:26 +01:00
<secondary>multicolumn</secondary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-02-20 23:27:56 +01:00
<para>
2005-09-12 21:17:45 +02:00
An index can be defined on more than one column of a table. For example, if
2001-02-20 23:27:56 +01:00
you have a table of this form:
<programlisting>
CREATE TABLE test2 (
major int,
minor int,
name varchar
);
</programlisting>
2003-11-06 23:21:47 +01:00
(say, you keep your <filename class="directory">/dev</filename>
2009-04-27 18:27:36 +02:00
directory in a database...) and you frequently issue queries like:
2001-02-20 23:27:56 +01:00
<programlisting>
SELECT name FROM test2 WHERE major = <replaceable>constant</replaceable> AND minor = <replaceable>constant</replaceable>;
</programlisting>
2009-06-17 23:58:49 +02:00
then it might be appropriate to define an index on the columns
2001-02-20 23:27:56 +01:00
<structfield>major</structfield> and
2007-02-01 01:28:19 +01:00
<structfield>minor</structfield> together, e.g.:
2001-02-20 23:27:56 +01:00
<programlisting>
CREATE INDEX test2_mm_idx ON test2 (major, minor);
</programlisting>
1999-07-22 17:11:05 +02:00
</para>
<para>
2021-04-06 00:41:09 +02:00
Currently, only the B-tree, GiST, GIN, and BRIN index types support
multiple-key-column indexes. Whether there can be multiple key
columns is independent of whether <literal>INCLUDE</literal> columns
can be added to the index. Indexes can have up to 32 columns,
including <literal>INCLUDE</literal> columns. (This limit can be
2001-11-21 06:53:41 +01:00
altered when building <productname>PostgreSQL</productname>; see the
2003-05-28 18:04:02 +02:00
file <filename>pg_config_manual.h</filename>.)
1999-07-22 17:11:05 +02:00
</para>
<para>
2005-09-12 21:17:45 +02:00
A multicolumn B-tree index can be used with query conditions that
involve any subset of the index's columns, but the index is most
efficient when there are constraints on the leading (leftmost) columns.
The exact rule is that equality constraints on leading columns, plus
any inequality constraints on the first column that does not have an
equality constraint, will be used to limit the portion of the index
that is scanned. Constraints on columns to the right of these columns
are checked in the index, so they save visits to the table proper, but
they do not reduce the portion of the index that has to be scanned.
For example, given an index on <literal>(a, b, c)</literal> and a
2017-10-09 03:44:17 +02:00
query condition <literal>WHERE a = 5 AND b >= 42 AND c < 77</literal>,
2005-09-12 21:17:45 +02:00
the index would have to be scanned from the first entry with
2017-10-09 03:44:17 +02:00
<literal>a</literal> = 5 and <literal>b</literal> = 42 up through the last entry with
<literal>a</literal> = 5. Index entries with <literal>c</literal> >= 77 would be
2005-09-12 21:17:45 +02:00
skipped, but they'd still have to be scanned through.
This index could in principle be used for queries that have constraints
2017-10-09 03:44:17 +02:00
on <literal>b</literal> and/or <literal>c</literal> with no constraint on <literal>a</literal>
2005-09-12 21:17:45 +02:00
— but the entire index would have to be scanned, so in most cases
the planner would prefer a sequential table scan over using the index.
1999-07-22 17:11:05 +02:00
</para>
<para>
2006-05-24 13:01:39 +02:00
A multicolumn GiST index can be used with query conditions that
2008-07-11 23:06:29 +02:00
involve any subset of the index's columns. Conditions on additional
columns restrict the entries returned by the index, but the condition on
the first column is the most important one for determining how much of
the index needs to be scanned. A GiST index will be relatively
ineffective if its first column has only a few distinct values, even if
2006-05-24 13:01:39 +02:00
there are many distinct values in additional columns.
2005-09-12 21:17:45 +02:00
</para>
2008-07-11 23:06:29 +02:00
<para>
A multicolumn GIN index can be used with query conditions that
involve any subset of the index's columns. Unlike B-tree or GiST,
index search effectiveness is the same regardless of which index column(s)
the query conditions use.
</para>
2016-03-29 00:11:12 +02:00
<para>
A multicolumn BRIN index can be used with query conditions that
involve any subset of the index's columns. Like GIN and unlike B-tree or
GiST, index search effectiveness is the same regardless of which index
column(s) the query conditions use. The only reason to have multiple BRIN
indexes instead of one multicolumn BRIN index on a single table is to have
a different <literal>pages_per_range</literal> storage parameter.
</para>
2005-09-12 21:17:45 +02:00
<para>
Of course, each column must be used with operators appropriate to the index
type; clauses that involve other operators will not be considered.
2001-02-20 23:27:56 +01:00
</para>
<para>
2005-09-12 21:17:45 +02:00
Multicolumn indexes should be used sparingly. In most situations,
2001-02-20 23:27:56 +01:00
an index on a single column is sufficient and saves space and time.
2003-05-28 18:04:02 +02:00
Indexes with more than three columns are unlikely to be helpful
2005-09-12 21:17:45 +02:00
unless the usage of the table is extremely stylized. See also
2017-11-23 15:39:47 +01:00
<xref linkend="indexes-bitmap-scans"/> and
<xref linkend="indexes-index-only-scans"/> for some discussion of the
2009-04-27 18:27:36 +02:00
merits of different index configurations.
2001-02-20 23:27:56 +01:00
</para>
</sect1>
2007-02-14 21:47:15 +01:00
<sect1 id="indexes-ordering">
2017-10-09 03:44:17 +02:00
<title>Indexes and <literal>ORDER BY</literal></title>
2007-02-14 21:47:15 +01:00
<indexterm zone="indexes-ordering">
<primary>index</primary>
2017-10-09 03:44:17 +02:00
<secondary>and <literal>ORDER BY</literal></secondary>
2007-02-14 21:47:15 +01:00
</indexterm>
<para>
In addition to simply finding the rows to be returned by a query,
an index may be able to deliver them in a specific sorted order.
2017-10-09 03:44:17 +02:00
This allows a query's <literal>ORDER BY</literal> specification to be honored
2007-02-14 21:47:15 +01:00
without a separate sorting step. Of the index types currently
supported by <productname>PostgreSQL</productname>, only B-tree
can produce sorted output — the other index types return
matching rows in an unspecified, implementation-dependent order.
</para>
<para>
2017-10-09 03:44:17 +02:00
The planner will consider satisfying an <literal>ORDER BY</literal> specification
2009-06-17 23:58:49 +02:00
either by scanning an available index that matches the specification,
2007-02-14 21:47:15 +01:00
or by scanning the table in physical order and doing an explicit
sort. For a query that requires scanning a large fraction of the
2009-06-17 23:58:49 +02:00
table, an explicit sort is likely to be faster than using an index
2009-04-27 18:27:36 +02:00
because it requires
2009-06-17 23:58:49 +02:00
less disk I/O due to following a sequential access pattern. Indexes are
2007-02-14 21:47:15 +01:00
more useful when only a few rows need be fetched. An important
2017-10-09 03:44:17 +02:00
special case is <literal>ORDER BY</literal> in combination with
<literal>LIMIT</literal> <replaceable>n</replaceable>: an explicit sort will have to process
all the data to identify the first <replaceable>n</replaceable> rows, but if there is
an index matching the <literal>ORDER BY</literal>, the first <replaceable>n</replaceable>
2007-02-14 21:47:15 +01:00
rows can be retrieved directly, without scanning the remainder at all.
</para>
<para>
By default, B-tree indexes store their entries in ascending order
Make heap TID a tiebreaker nbtree index column.
Make nbtree treat all index tuples as having a heap TID attribute.
Index searches can distinguish duplicates by heap TID, since heap TID is
always guaranteed to be unique. This general approach has numerous
benefits for performance, and is prerequisite to teaching VACUUM to
perform "retail index tuple deletion".
Naively adding a new attribute to every pivot tuple has unacceptable
overhead (it bloats internal pages), so suffix truncation of pivot
tuples is added. This will usually truncate away the "extra" heap TID
attribute from pivot tuples during a leaf page split, and may also
truncate away additional user attributes. This can increase fan-out,
especially in a multi-column index. Truncation can only occur at the
attribute granularity, which isn't particularly effective, but works
well enough for now. A future patch may add support for truncating
"within" text attributes by generating truncated key values using new
opclass infrastructure.
Only new indexes (BTREE_VERSION 4 indexes) will have insertions that
treat heap TID as a tiebreaker attribute, or will have pivot tuples
undergo suffix truncation during a leaf page split (on-disk
compatibility with versions 2 and 3 is preserved). Upgrades to version
4 cannot be performed on-the-fly, unlike upgrades from version 2 to
version 3. contrib/amcheck continues to work with version 2 and 3
indexes, while also enforcing stricter invariants when verifying version
4 indexes. These stricter invariants are the same invariants described
by "3.1.12 Sequencing" from the Lehman and Yao paper.
A later patch will enhance the logic used by nbtree to pick a split
point. This patch is likely to negatively impact performance without
smarter choices around the precise point to split leaf pages at. Making
these two mostly-distinct sets of enhancements into distinct commits
seems like it might clarify their design, even though neither commit is
particularly useful on its own.
The maximum allowed size of new tuples is reduced by an amount equal to
the space required to store an extra MAXALIGN()'d TID in a new high key
during leaf page splits. The user-facing definition of the "1/3 of a
page" restriction is already imprecise, and so does not need to be
revised. However, there should be a compatibility note in the v12
release notes.
Author: Peter Geoghegan
Reviewed-By: Heikki Linnakangas, Alexander Korotkov
Discussion: https://postgr.es/m/CAH2-WzkVb0Kom=R+88fDFb=JSxZMFvbHVC6Mn9LJ2n=X=kS-Uw@mail.gmail.com
2019-03-20 18:04:01 +01:00
with nulls last (table TID is treated as a tiebreaker column among
otherwise equal entries). This means that a forward scan of an
index on column <literal>x</literal> produces output satisfying <literal>ORDER BY x</literal>
2017-10-09 03:44:17 +02:00
(or more verbosely, <literal>ORDER BY x ASC NULLS LAST</literal>). The
2007-02-14 21:47:15 +01:00
index can also be scanned backward, producing output satisfying
2017-10-09 03:44:17 +02:00
<literal>ORDER BY x DESC</literal>
(or more verbosely, <literal>ORDER BY x DESC NULLS FIRST</literal>, since
<literal>NULLS FIRST</literal> is the default for <literal>ORDER BY DESC</literal>).
2007-02-14 21:47:15 +01:00
</para>
<para>
You can adjust the ordering of a B-tree index by including the
2017-10-09 03:44:17 +02:00
options <literal>ASC</literal>, <literal>DESC</literal>, <literal>NULLS FIRST</literal>,
and/or <literal>NULLS LAST</literal> when creating the index; for example:
2007-02-14 21:47:15 +01:00
<programlisting>
CREATE INDEX test2_info_nulls_low ON test2 (info NULLS FIRST);
CREATE INDEX test3_desc_index ON test3 (id DESC NULLS LAST);
</programlisting>
An index stored in ascending order with nulls first can satisfy
2017-10-09 03:44:17 +02:00
either <literal>ORDER BY x ASC NULLS FIRST</literal> or
<literal>ORDER BY x DESC NULLS LAST</literal> depending on which direction
2007-02-14 21:47:15 +01:00
it is scanned in.
</para>
<para>
You might wonder why bother providing all four options, when two
options together with the possibility of backward scan would cover
2017-10-09 03:44:17 +02:00
all the variants of <literal>ORDER BY</literal>. In single-column indexes
2007-02-14 21:47:15 +01:00
the options are indeed redundant, but in multicolumn indexes they can be
2017-10-09 03:44:17 +02:00
useful. Consider a two-column index on <literal>(x, y)</literal>: this can
satisfy <literal>ORDER BY x, y</literal> if we scan forward, or
<literal>ORDER BY x DESC, y DESC</literal> if we scan backward.
2007-02-14 21:47:15 +01:00
But it might be that the application frequently needs to use
2017-10-09 03:44:17 +02:00
<literal>ORDER BY x ASC, y DESC</literal>. There is no way to get that
2009-06-17 23:58:49 +02:00
ordering from a plain index, but it is possible if the index is defined
2017-10-09 03:44:17 +02:00
as <literal>(x ASC, y DESC)</literal> or <literal>(x DESC, y ASC)</literal>.
2007-02-14 21:47:15 +01:00
</para>
<para>
Obviously, indexes with non-default sort orderings are a fairly
specialized feature, but sometimes they can produce tremendous
2009-06-17 23:58:49 +02:00
speedups for certain queries. Whether it's worth maintaining such an
2007-02-14 21:47:15 +01:00
index depends on how often you use queries that require a special
sort ordering.
</para>
</sect1>
2005-09-12 21:17:45 +02:00
<sect1 id="indexes-bitmap-scans">
<title>Combining Multiple Indexes</title>
<indexterm zone="indexes-bitmap-scans">
<primary>index</primary>
<secondary>combining multiple indexes</secondary>
</indexterm>
<indexterm zone="indexes-bitmap-scans">
<primary>bitmap scan</primary>
</indexterm>
<para>
A single index scan can only use query clauses that use the index's
columns with operators of its operator class and are joined with
2017-10-09 03:44:17 +02:00
<literal>AND</literal>. For example, given an index on <literal>(a, b)</literal>
a query condition like <literal>WHERE a = 5 AND b = 6</literal> could
use the index, but a query like <literal>WHERE a = 5 OR b = 6</literal> could not
2005-09-12 21:17:45 +02:00
directly use the index.
</para>
<para>
2009-04-27 18:27:36 +02:00
Fortunately,
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> has the ability to combine multiple indexes
2005-09-12 21:17:45 +02:00
(including multiple uses of the same index) to handle cases that cannot
2017-10-09 03:44:17 +02:00
be implemented by single index scans. The system can form <literal>AND</literal>
and <literal>OR</literal> conditions across several index scans. For example,
a query like <literal>WHERE x = 42 OR x = 47 OR x = 53 OR x = 99</literal>
could be broken down into four separate scans of an index on <literal>x</literal>,
2005-09-12 21:17:45 +02:00
each scan using one of the query clauses. The results of these scans are
then ORed together to produce the result. Another example is that if we
2017-10-09 03:44:17 +02:00
have separate indexes on <literal>x</literal> and <literal>y</literal>, one possible
implementation of a query like <literal>WHERE x = 5 AND y = 6</literal> is to
2005-09-12 21:17:45 +02:00
use each index with the appropriate query clause and then AND together
the index results to identify the result rows.
</para>
<para>
To combine multiple indexes, the system scans each needed index and
2017-10-09 03:44:17 +02:00
prepares a <firstterm>bitmap</firstterm> in memory giving the locations of
2005-09-12 21:17:45 +02:00
table rows that are reported as matching that index's conditions.
The bitmaps are then ANDed and ORed together as needed by the query.
Finally, the actual table rows are visited and returned. The table rows
are visited in physical order, because that is how the bitmap is laid
out; this means that any ordering of the original indexes is lost, and
so a separate sort step will be needed if the query has an <literal>ORDER
2017-10-09 03:44:17 +02:00
BY</literal> clause. For this reason, and because each additional index scan
2005-09-12 21:17:45 +02:00
adds extra time, the planner will sometimes choose to use a simple index
scan even though additional indexes are available that could have been
used as well.
</para>
<para>
In all but the simplest applications, there are various combinations of
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
indexes that might be useful, and the database developer must make
2005-11-05 00:14:02 +01:00
trade-offs to decide which indexes to provide. Sometimes multicolumn
2005-09-12 21:17:45 +02:00
indexes are best, but sometimes it's better to create separate indexes
and rely on the index-combination feature. For example, if your
workload includes a mix of queries that sometimes involve only column
2017-10-09 03:44:17 +02:00
<literal>x</literal>, sometimes only column <literal>y</literal>, and sometimes both
2005-09-12 21:17:45 +02:00
columns, you might choose to create two separate indexes on
2017-10-09 03:44:17 +02:00
<literal>x</literal> and <literal>y</literal>, relying on index combination to
2005-09-12 21:17:45 +02:00
process the queries that use both columns. You could also create a
2017-10-09 03:44:17 +02:00
multicolumn index on <literal>(x, y)</literal>. This index would typically be
2005-09-12 21:17:45 +02:00
more efficient than index combination for queries involving both
2017-11-23 15:39:47 +01:00
columns, but as discussed in <xref linkend="indexes-multicolumn"/>, it
2017-10-09 03:44:17 +02:00
would be almost useless for queries involving only <literal>y</literal>, so it
2009-04-27 18:27:36 +02:00
should not be the only index. A combination of the multicolumn index
2017-10-09 03:44:17 +02:00
and a separate index on <literal>y</literal> would serve reasonably well. For
queries involving only <literal>x</literal>, the multicolumn index could be
2005-09-12 21:17:45 +02:00
used, though it would be larger and hence slower than an index on
2017-10-09 03:44:17 +02:00
<literal>x</literal> alone. The last alternative is to create all three
2005-09-12 21:17:45 +02:00
indexes, but this is probably only reasonable if the table is searched
much more often than it is updated and all three types of query are
common. If one of the types of query is much less common than the
others, you'd probably settle for creating just the two indexes that
best match the common types.
</para>
</sect1>
2001-05-17 23:50:18 +02:00
<sect1 id="indexes-unique">
<title>Unique Indexes</title>
2001-02-20 23:27:56 +01:00
2001-05-17 23:50:18 +02:00
<indexterm zone="indexes-unique">
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2001-05-13 00:51:36 +02:00
<secondary>unique</secondary>
</indexterm>
2001-02-20 23:27:56 +01:00
<para>
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
Indexes can also be used to enforce uniqueness of a column's value,
2001-02-20 23:27:56 +01:00
or the uniqueness of the combined values of more than one column.
<synopsis>
2018-11-22 19:24:57 +01:00
CREATE UNIQUE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> (<replaceable>column</replaceable> <optional>, ...</optional>);
2001-02-20 23:27:56 +01:00
</synopsis>
2001-11-18 01:59:00 +01:00
Currently, only B-tree indexes can be declared unique.
2001-02-20 23:27:56 +01:00
</para>
<para>
When an index is declared unique, multiple table rows with equal
2009-04-27 18:27:36 +02:00
indexed values are not allowed. Null values are not considered
2003-11-06 23:21:47 +01:00
equal. A multicolumn unique index will only reject cases where all
2018-11-22 19:24:57 +01:00
indexed columns are equal in multiple rows.
2001-02-20 23:27:56 +01:00
</para>
<para>
2003-11-06 23:21:47 +01:00
<productname>PostgreSQL</productname> automatically creates a unique
2009-04-27 18:27:36 +02:00
index when a unique constraint or primary key is defined for a table.
2008-07-11 23:06:29 +02:00
The index covers the columns that make up the primary key or unique
2009-04-27 18:27:36 +02:00
constraint (a multicolumn index, if appropriate), and is the mechanism
2003-11-06 23:21:47 +01:00
that enforces the constraint.
2001-02-20 23:27:56 +01:00
</para>
2001-10-31 21:38:26 +01:00
<note>
<para>
2015-08-31 23:05:23 +02:00
There's no need to manually
2003-11-06 23:21:47 +01:00
create indexes on unique columns; doing so would just duplicate
the automatically-created index.
2001-10-31 21:38:26 +01:00
</para>
</note>
2001-02-20 23:27:56 +01:00
</sect1>
2003-05-28 18:04:02 +02:00
<sect1 id="indexes-expressional">
<title>Indexes on Expressions</title>
2001-02-20 23:27:56 +01:00
2003-05-28 18:04:02 +02:00
<indexterm zone="indexes-expressional">
2003-08-31 19:32:24 +02:00
<primary>index</primary>
<secondary sortas="expressions">on expressions</secondary>
2001-05-13 00:51:36 +02:00
</indexterm>
2001-02-20 23:27:56 +01:00
<para>
2009-06-17 23:58:49 +02:00
An index column need not be just a column of the underlying table,
2003-05-28 18:04:02 +02:00
but can be a function or scalar expression computed from one or
2009-06-17 23:58:49 +02:00
more columns of the table. This feature is useful to obtain fast
2003-05-28 18:04:02 +02:00
access to tables based on the results of computations.
2001-02-20 23:27:56 +01:00
</para>
<para>
For example, a common way to do case-insensitive comparisons is to
2001-11-19 10:05:02 +01:00
use the <function>lower</function> function:
2001-02-20 23:27:56 +01:00
<programlisting>
SELECT * FROM test1 WHERE lower(col1) = 'value';
</programlisting>
2009-04-27 18:27:36 +02:00
This query can use an index if one has been
2003-03-13 02:30:29 +01:00
defined on the result of the <literal>lower(col1)</literal>
2009-04-27 18:27:36 +02:00
function:
2001-02-20 23:27:56 +01:00
<programlisting>
CREATE INDEX test1_lower_col1_idx ON test1 (lower(col1));
</programlisting>
</para>
2003-11-06 23:21:47 +01:00
<para>
2017-10-09 03:44:17 +02:00
If we were to declare this index <literal>UNIQUE</literal>, it would prevent
creation of rows whose <literal>col1</literal> values differ only in case,
as well as rows whose <literal>col1</literal> values are actually identical.
2003-11-06 23:21:47 +01:00
Thus, indexes on expressions can be used to enforce constraints that
are not definable as simple unique constraints.
</para>
2001-02-20 23:27:56 +01:00
<para>
2009-04-27 18:27:36 +02:00
As another example, if one often does queries like:
2003-05-28 18:04:02 +02:00
<programlisting>
SELECT * FROM people WHERE (first_name || ' ' || last_name) = 'John Smith';
</programlisting>
then it might be worth creating an index like this:
<programlisting>
CREATE INDEX people_names ON people ((first_name || ' ' || last_name));
</programlisting>
2001-02-20 23:27:56 +01:00
</para>
2003-05-28 18:04:02 +02:00
<para>
2017-10-09 03:44:17 +02:00
The syntax of the <command>CREATE INDEX</command> command normally requires
2003-05-28 18:04:02 +02:00
writing parentheses around index expressions, as shown in the second
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
example. The parentheses can be omitted when the expression is just
2003-05-28 18:04:02 +02:00
a function call, as in the first example.
</para>
<para>
2005-09-12 21:17:45 +02:00
Index expressions are relatively expensive to maintain, because the
2021-12-22 22:29:16 +01:00
derived expression(s) must be computed for each row insertion
and non-HOT update. However, the index expressions are
2017-10-09 03:44:17 +02:00
<emphasis>not</emphasis> recomputed during an indexed search, since they are
2005-09-12 21:17:45 +02:00
already stored in the index. In both examples above, the system
2017-10-09 03:44:17 +02:00
sees the query as just <literal>WHERE indexedcolumn = 'constant'</literal>
2005-09-12 21:17:45 +02:00
and so the speed of the search is equivalent to any other simple index
query. Thus, indexes on expressions are useful when retrieval speed
is more important than insertion and update speed.
1999-07-22 17:11:05 +02:00
</para>
2001-02-20 23:27:56 +01:00
</sect1>
1999-07-22 17:11:05 +02:00
2000-03-26 21:47:17 +02:00
2001-10-31 21:38:26 +01:00
<sect1 id="indexes-partial">
<title>Partial Indexes</title>
1999-07-22 17:11:05 +02:00
2001-10-31 21:38:26 +01:00
<indexterm zone="indexes-partial">
2003-08-31 19:32:24 +02:00
<primary>index</primary>
2001-05-13 00:51:36 +02:00
<secondary>partial</secondary>
</indexterm>
2001-10-31 21:38:26 +01:00
<para>
A <firstterm>partial index</firstterm> is an index built over a
subset of a table; the subset is defined by a conditional
expression (called the <firstterm>predicate</firstterm> of the
2009-04-27 18:27:36 +02:00
partial index). The index contains entries only for those table
2005-09-12 21:17:45 +02:00
rows that satisfy the predicate. Partial indexes are a specialized
feature, but there are several situations in which they are useful.
2001-10-31 21:38:26 +01:00
</para>
<para>
2005-09-12 21:17:45 +02:00
One major reason for using a partial index is to avoid indexing common
2001-12-04 02:22:13 +01:00
values. Since a query searching for a common value (one that
accounts for more than a few percent of all the table rows) will not
2001-10-31 21:38:26 +01:00
use the index anyway, there is no point in keeping those rows in the
2009-06-17 23:58:49 +02:00
index at all. This reduces the size of the index, which will speed
up those queries that do use the index. It will also speed up many table
2001-11-19 10:05:02 +01:00
update operations because the index does not need to be
2017-11-23 15:39:47 +01:00
updated in all cases. <xref linkend="indexes-partial-ex1"/> shows a
2001-10-31 21:38:26 +01:00
possible application of this idea.
</para>
<example id="indexes-partial-ex1">
<title>Setting up a Partial Index to Exclude Common Values</title>
<para>
Suppose you are storing web server access logs in a database.
2003-03-13 02:30:29 +01:00
Most accesses originate from the IP address range of your organization but
2001-10-31 21:38:26 +01:00
some are from elsewhere (say, employees on dial-up connections).
2001-12-04 02:22:13 +01:00
If your searches by IP are primarily for outside accesses,
you probably do not need to index the IP range that corresponds to your
2001-10-31 21:38:26 +01:00
organization's subnet.
</para>
<para>
Assume a table like this:
<programlisting>
CREATE TABLE access_log (
url varchar,
client_ip inet,
...
);
</programlisting>
</para>
<para>
To create a partial index that suits our example, use a command
such as this:
<programlisting>
CREATE INDEX access_log_client_ip_ix ON access_log (client_ip)
2009-06-17 23:58:49 +02:00
WHERE NOT (client_ip > inet '192.168.100.0' AND
2009-04-27 18:27:36 +02:00
client_ip < inet '192.168.100.255');
2001-10-31 21:38:26 +01:00
</programlisting>
</para>
1999-07-22 17:11:05 +02:00
<para>
2001-10-31 21:38:26 +01:00
A typical query that can use this index would be:
<programlisting>
2009-04-27 18:27:36 +02:00
SELECT *
FROM access_log
WHERE url = '/index.html' AND client_ip = inet '212.78.10.32';
2001-10-31 21:38:26 +01:00
</programlisting>
2019-04-04 00:28:18 +02:00
Here the query's IP address is covered by the partial index. The
following query cannot use the partial index, as it uses an IP address
that is excluded from the index:
2001-10-31 21:38:26 +01:00
<programlisting>
2009-04-27 18:27:36 +02:00
SELECT *
FROM access_log
2019-04-04 00:28:18 +02:00
WHERE url = '/index.html' AND client_ip = inet '192.168.100.23';
2001-10-31 21:38:26 +01:00
</programlisting>
1999-07-22 17:11:05 +02:00
</para>
<para>
2001-10-31 21:38:26 +01:00
Observe that this kind of partial index requires that the common
2009-04-27 18:27:36 +02:00
values be predetermined, so such partial indexes are best used for
2019-04-04 00:28:18 +02:00
data distributions that do not change. Such indexes can be recreated
2009-04-27 18:27:36 +02:00
occasionally to adjust for new data distributions, but this adds
2009-06-17 23:58:49 +02:00
maintenance effort.
2001-10-31 21:38:26 +01:00
</para>
</example>
<para>
2009-06-17 23:58:49 +02:00
Another possible use for a partial index is to exclude values from the
2005-09-12 21:17:45 +02:00
index that the
2001-10-31 21:38:26 +01:00
typical query workload is not interested in; this is shown in <xref
2017-11-23 15:39:47 +01:00
linkend="indexes-partial-ex2"/>. This results in the same
2001-10-31 21:38:26 +01:00
advantages as listed above, but it prevents the
2001-12-04 02:22:13 +01:00
<quote>uninteresting</quote> values from being accessed via that
2009-04-27 18:27:36 +02:00
index, even if an index scan might be profitable in that
2001-10-31 21:38:26 +01:00
case. Obviously, setting up partial indexes for this kind of
scenario will require a lot of care and experimentation.
</para>
1999-07-22 17:11:05 +02:00
2001-10-31 21:38:26 +01:00
<example id="indexes-partial-ex2">
<title>Setting up a Partial Index to Exclude Uninteresting Values</title>
<para>
2001-12-04 02:22:13 +01:00
If you have a table that contains both billed and unbilled orders,
2001-10-31 21:38:26 +01:00
where the unbilled orders take up a small fraction of the total
2001-12-04 02:22:13 +01:00
table and yet those are the most-accessed rows, you can improve
performance by creating an index on just the unbilled rows. The
command to create the index would look like this:
2001-10-31 21:38:26 +01:00
<programlisting>
CREATE INDEX orders_unbilled_index ON orders (order_nr)
WHERE billed is not true;
</programlisting>
1999-07-22 17:11:05 +02:00
</para>
<para>
2007-02-01 01:28:19 +01:00
A possible query to use this index would be:
2001-10-31 21:38:26 +01:00
<programlisting>
2005-01-22 23:56:36 +01:00
SELECT * FROM orders WHERE billed is not true AND order_nr < 10000;
2001-10-31 21:38:26 +01:00
</programlisting>
However, the index can also be used in queries that do not involve
2017-10-09 03:44:17 +02:00
<structfield>order_nr</structfield> at all, e.g.:
2001-10-31 21:38:26 +01:00
<programlisting>
2005-01-22 23:56:36 +01:00
SELECT * FROM orders WHERE billed is not true AND amount > 5000.00;
2001-10-31 21:38:26 +01:00
</programlisting>
This is not as efficient as a partial index on the
2017-10-09 03:44:17 +02:00
<structfield>amount</structfield> column would be, since the system has to
2001-12-04 02:22:13 +01:00
scan the entire index. Yet, if there are relatively few unbilled
orders, using this partial index just to find the unbilled orders
could be a win.
1999-07-22 17:11:05 +02:00
</para>
<para>
2001-10-31 21:38:26 +01:00
Note that this query cannot use this index:
<programlisting>
SELECT * FROM orders WHERE order_nr = 3501;
</programlisting>
2009-04-27 18:27:36 +02:00
The order 3501 might be among the billed or unbilled
2001-10-31 21:38:26 +01:00
orders.
1999-07-22 17:11:05 +02:00
</para>
2001-10-31 21:38:26 +01:00
</example>
<para>
2017-11-23 15:39:47 +01:00
<xref linkend="indexes-partial-ex2"/> also illustrates that the
2001-10-31 21:38:26 +01:00
indexed column and the column used in the predicate do not need to
match. <productname>PostgreSQL</productname> supports partial
2001-12-04 02:22:13 +01:00
indexes with arbitrary predicates, so long as only columns of the
2001-10-31 21:38:26 +01:00
table being indexed are involved. However, keep in mind that the
2001-12-04 02:22:13 +01:00
predicate must match the conditions used in the queries that
are supposed to benefit from the index. To be precise, a partial
index can be used in a query only if the system can recognize that
2017-10-09 03:44:17 +02:00
the <literal>WHERE</literal> condition of the query mathematically implies
2003-03-13 02:30:29 +01:00
the predicate of the index.
2001-10-31 21:38:26 +01:00
<productname>PostgreSQL</productname> does not have a sophisticated
theorem prover that can recognize mathematically equivalent
2003-03-13 02:30:29 +01:00
expressions that are written in different forms. (Not
2001-10-31 21:38:26 +01:00
only is such a general theorem prover extremely difficult to
create, it would probably be too slow to be of any real use.)
2001-12-04 02:22:13 +01:00
The system can recognize simple inequality implications, for example
<quote>x < 1</quote> implies <quote>x < 2</quote>; otherwise
2003-11-06 23:21:47 +01:00
the predicate condition must exactly match part of the query's
2017-10-09 03:44:17 +02:00
<literal>WHERE</literal> condition
2009-04-27 18:27:36 +02:00
or the index will not be recognized as usable. Matching takes
2006-09-04 21:58:02 +02:00
place at query planning time, not at run time. As a result,
2009-04-27 18:27:36 +02:00
parameterized query clauses do not work with a partial index. For
2008-07-11 23:06:29 +02:00
example a prepared query with a parameter might specify
<quote>x < ?</quote> which will never imply
<quote>x < 2</quote> for all possible values of the parameter.
2001-10-31 21:38:26 +01:00
</para>
2001-12-04 02:22:13 +01:00
<para>
A third possible use for partial indexes does not require the
index to be used in queries at all. The idea here is to create
a unique index over a subset of a table, as in <xref
2017-11-23 15:39:47 +01:00
linkend="indexes-partial-ex3"/>. This enforces uniqueness
2001-12-04 02:22:13 +01:00
among the rows that satisfy the index predicate, without constraining
those that do not.
</para>
<example id="indexes-partial-ex3">
<title>Setting up a Partial Unique Index</title>
<para>
Suppose that we have a table describing test outcomes. We wish
2017-10-09 03:44:17 +02:00
to ensure that there is only one <quote>successful</quote> entry for
2001-12-04 02:22:13 +01:00
a given subject and target combination, but there might be any number of
2017-10-09 03:44:17 +02:00
<quote>unsuccessful</quote> entries. Here is one way to do it:
2001-12-04 02:22:13 +01:00
<programlisting>
2003-03-13 02:30:29 +01:00
CREATE TABLE tests (
subject text,
target text,
success boolean,
...
);
2001-12-04 02:22:13 +01:00
CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target)
WHERE success;
</programlisting>
2009-04-27 18:27:36 +02:00
This is a particularly efficient approach when there are few
2020-05-22 01:49:30 +02:00
successful tests and many unsuccessful ones. It is also possible to
allow only one null in a column by creating a unique partial index
with an <literal>IS NULL</literal> restriction.
2001-12-04 02:22:13 +01:00
</para>
2019-12-27 20:49:08 +01:00
2001-12-04 02:22:13 +01:00
</example>
2001-10-31 21:38:26 +01:00
<para>
Finally, a partial index can also be used to override the system's
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
query plan choices. Also, data sets with peculiar
distributions might cause the system to use an index when it really
2001-10-31 21:38:26 +01:00
should not. In that case the index can be set up so that it is not
available for the offending query. Normally,
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> makes reasonable choices about index
2001-10-31 21:38:26 +01:00
usage (e.g., it avoids them when retrieving common values, so the
earlier example really only saves index size, it is not required to
avoid index usage), and grossly incorrect plan choices are cause
for a bug report.
</para>
<para>
Keep in mind that setting up a partial index indicates that you
know at least as much as the query planner knows, in particular you
know when an index might be profitable. Forming this knowledge
requires experience and understanding of how indexes in
2020-02-20 00:52:18 +01:00
<productname>PostgreSQL</productname> work. In most cases, the
advantage of a partial index over a regular index will be minimal.
There are cases where they are quite counterproductive, as in <xref
linkend="indexes-partial-ex4"/>.
2001-10-31 21:38:26 +01:00
</para>
2020-02-20 00:52:18 +01:00
<example id="indexes-partial-ex4">
<title>Do Not Use Partial Indexes as a Substitute for Partitioning</title>
<para>
You might be tempted to create a large set of non-overlapping partial
indexes, for example
<programlisting>
CREATE INDEX mytable_cat_1 ON mytable (data) WHERE category = 1;
CREATE INDEX mytable_cat_2 ON mytable (data) WHERE category = 2;
CREATE INDEX mytable_cat_3 ON mytable (data) WHERE category = 3;
...
CREATE INDEX mytable_cat_<replaceable>N</replaceable> ON mytable (data) WHERE category = <replaceable>N</replaceable>;
</programlisting>
This is a bad idea! Almost certainly, you'll be better off with a
single non-partial index, declared like
<programlisting>
CREATE INDEX mytable_cat_data ON mytable (category, data);
</programlisting>
(Put the category column first, for the reasons described in
<xref linkend="indexes-multicolumn"/>.) While a search in this larger
index might have to descend through a couple more tree levels than a
search in a smaller index, that's almost certainly going to be cheaper
than the planner effort needed to select the appropriate one of the
partial indexes. The core of the problem is that the system does not
understand the relationship among the partial indexes, and will
laboriously test each one to see if it's applicable to the current
query.
</para>
<para>
If your table is large enough that a single index really is a bad idea,
you should look into using partitioning instead (see
<xref linkend="ddl-partitioning"/>). With that mechanism, the system
does understand that the tables and indexes are non-overlapping, so
far better performance is possible.
</para>
</example>
2001-10-31 21:38:26 +01:00
<para>
More information about partial indexes can be found in <xref
2017-11-23 15:39:47 +01:00
linkend="ston89b"/>, <xref linkend="olson93"/>, and <xref
linkend="seshadri95"/>.
2001-10-31 21:38:26 +01:00
</para>
</sect1>
2005-09-12 21:17:45 +02:00
2018-11-22 19:24:57 +01:00
<sect1 id="indexes-index-only-scans">
<title>Index-Only Scans and Covering Indexes</title>
<indexterm zone="indexes-index-only-scans">
<primary>index</primary>
<secondary>index-only scans</secondary>
</indexterm>
<indexterm zone="indexes-index-only-scans">
<primary>index-only scan</primary>
</indexterm>
<indexterm zone="indexes-index-only-scans">
<primary>index</primary>
<secondary>covering</secondary>
</indexterm>
<indexterm zone="indexes-index-only-scans">
<primary>covering index</primary>
</indexterm>
<para>
All indexes in <productname>PostgreSQL</productname>
are <firstterm>secondary</firstterm> indexes, meaning that each index is
stored separately from the table's main data area (which is called the
table's <firstterm>heap</firstterm>
in <productname>PostgreSQL</productname> terminology). This means that
in an ordinary index scan, each row retrieval requires fetching data from
both the index and the heap. Furthermore, while the index entries that
match a given indexable <literal>WHERE</literal> condition are usually
close together in the index, the table rows they reference might be
anywhere in the heap. The heap-access portion of an index scan thus
involves a lot of random access into the heap, which can be slow,
particularly on traditional rotating media. (As described in
<xref linkend="indexes-bitmap-scans"/>, bitmap scans try to alleviate
this cost by doing the heap accesses in sorted order, but that only goes
so far.)
</para>
<para>
To solve this performance problem, <productname>PostgreSQL</productname>
supports <firstterm>index-only scans</firstterm>, which can answer
queries from an index alone without any heap access. The basic idea is
to return values directly out of each index entry instead of consulting
the associated heap entry. There are two fundamental restrictions on
when this method can be used:
<orderedlist>
<listitem>
<para>
The index type must support index-only scans. B-tree indexes always
do. GiST and SP-GiST indexes support index-only scans for some
operator classes but not others. Other index types have no support.
The underlying requirement is that the index must physically store, or
else be able to reconstruct, the original data value for each index
entry. As a counterexample, GIN indexes cannot support index-only
scans because each index entry typically holds only part of the
original data value.
</para>
</listitem>
<listitem>
<para>
The query must reference only columns stored in the index. For
example, given an index on columns <literal>x</literal>
and <literal>y</literal> of a table that also has a
column <literal>z</literal>, these queries could use index-only scans:
<programlisting>
SELECT x, y FROM tab WHERE x = 'key';
SELECT x FROM tab WHERE x = 'key' AND y < 42;
</programlisting>
but these queries could not:
<programlisting>
SELECT x, z FROM tab WHERE x = 'key';
SELECT x FROM tab WHERE x = 'key' AND z < 42;
</programlisting>
(Expression indexes and partial indexes complicate this rule,
as discussed below.)
</para>
</listitem>
</orderedlist>
</para>
<para>
If these two fundamental requirements are met, then all the data values
required by the query are available from the index, so an index-only scan
is physically possible. But there is an additional requirement for any
table scan in <productname>PostgreSQL</productname>: it must verify that
each retrieved row be <quote>visible</quote> to the query's MVCC
snapshot, as discussed in <xref linkend="mvcc"/>. Visibility information
is not stored in index entries, only in heap entries; so at first glance
it would seem that every row retrieval would require a heap access
anyway. And this is indeed the case, if the table row has been modified
recently. However, for seldom-changing data there is a way around this
problem. <productname>PostgreSQL</productname> tracks, for each page in
a table's heap, whether all rows stored in that page are old enough to be
visible to all current and future transactions. This information is
stored in a bit in the table's <firstterm>visibility map</firstterm>. An
index-only scan, after finding a candidate index entry, checks the
visibility map bit for the corresponding heap page. If it's set, the row
is known visible and so the data can be returned with no further work.
If it's not set, the heap entry must be visited to find out whether it's
visible, so no performance advantage is gained over a standard index
scan. Even in the successful case, this approach trades visibility map
accesses for heap accesses; but since the visibility map is four orders
of magnitude smaller than the heap it describes, far less physical I/O is
needed to access it. In most situations the visibility map remains
cached in memory all the time.
</para>
<para>
In short, while an index-only scan is possible given the two fundamental
requirements, it will be a win only if a significant fraction of the
table's heap pages have their all-visible map bits set. But tables in
which a large fraction of the rows are unchanging are common enough to
make this type of scan very useful in practice.
</para>
<para>
<indexterm>
<primary><literal>INCLUDE</literal></primary>
<secondary>in index definitions</secondary>
</indexterm>
To make effective use of the index-only scan feature, you might choose to
create a <firstterm>covering index</firstterm>, which is an index
specifically designed to include the columns needed by a particular
type of query that you run frequently. Since queries typically need to
retrieve more columns than just the ones they search
on, <productname>PostgreSQL</productname> allows you to create an index
in which some columns are just <quote>payload</quote> and are not part
of the search key. This is done by adding an <literal>INCLUDE</literal>
clause listing the extra columns. For example, if you commonly run
queries like
<programlisting>
SELECT y FROM tab WHERE x = 'key';
</programlisting>
the traditional approach to speeding up such queries would be to create
an index on <literal>x</literal> only. However, an index defined as
<programlisting>
CREATE INDEX tab_x_y ON tab(x) INCLUDE (y);
</programlisting>
could handle these queries as index-only scans,
because <literal>y</literal> can be obtained from the index without
visiting the heap.
</para>
<para>
Because column <literal>y</literal> is not part of the index's search
key, it does not have to be of a data type that the index can handle;
it's merely stored in the index and is not interpreted by the index
machinery. Also, if the index is a unique index, that is
<programlisting>
CREATE UNIQUE INDEX tab_x_y ON tab(x) INCLUDE (y);
</programlisting>
the uniqueness condition applies to just column <literal>x</literal>,
not to the combination of <literal>x</literal> and <literal>y</literal>.
(An <literal>INCLUDE</literal> clause can also be written
in <literal>UNIQUE</literal> and <literal>PRIMARY KEY</literal>
constraints, providing alternative syntax for setting up an index like
this.)
</para>
<para>
It's wise to be conservative about adding non-key payload columns to an
index, especially wide columns. If an index tuple exceeds the
maximum size allowed for the index type, data insertion will fail.
In any case, non-key columns duplicate data from the index's table
and bloat the size of the index, thus potentially slowing searches.
And remember that there is little point in including payload columns in an
index unless the table changes slowly enough that an index-only scan is
likely to not need to access the heap. If the heap tuple must be visited
anyway, it costs nothing more to get the column's value from there.
Other restrictions are that expressions are not currently supported as
2021-04-06 00:41:09 +02:00
included columns, and that only B-tree, GiST and SP-GiST indexes currently
support included columns.
2018-11-22 19:24:57 +01:00
</para>
<para>
Before <productname>PostgreSQL</productname> had
the <literal>INCLUDE</literal> feature, people sometimes made covering
indexes by writing the payload columns as ordinary index columns,
that is writing
<programlisting>
CREATE INDEX tab_x_y ON tab(x, y);
</programlisting>
even though they had no intention of ever using <literal>y</literal> as
part of a <literal>WHERE</literal> clause. This works fine as long as
the extra columns are trailing columns; making them be leading columns is
unwise for the reasons explained in <xref linkend="indexes-multicolumn"/>.
However, this method doesn't support the case where you want the index to
Make heap TID a tiebreaker nbtree index column.
Make nbtree treat all index tuples as having a heap TID attribute.
Index searches can distinguish duplicates by heap TID, since heap TID is
always guaranteed to be unique. This general approach has numerous
benefits for performance, and is prerequisite to teaching VACUUM to
perform "retail index tuple deletion".
Naively adding a new attribute to every pivot tuple has unacceptable
overhead (it bloats internal pages), so suffix truncation of pivot
tuples is added. This will usually truncate away the "extra" heap TID
attribute from pivot tuples during a leaf page split, and may also
truncate away additional user attributes. This can increase fan-out,
especially in a multi-column index. Truncation can only occur at the
attribute granularity, which isn't particularly effective, but works
well enough for now. A future patch may add support for truncating
"within" text attributes by generating truncated key values using new
opclass infrastructure.
Only new indexes (BTREE_VERSION 4 indexes) will have insertions that
treat heap TID as a tiebreaker attribute, or will have pivot tuples
undergo suffix truncation during a leaf page split (on-disk
compatibility with versions 2 and 3 is preserved). Upgrades to version
4 cannot be performed on-the-fly, unlike upgrades from version 2 to
version 3. contrib/amcheck continues to work with version 2 and 3
indexes, while also enforcing stricter invariants when verifying version
4 indexes. These stricter invariants are the same invariants described
by "3.1.12 Sequencing" from the Lehman and Yao paper.
A later patch will enhance the logic used by nbtree to pick a split
point. This patch is likely to negatively impact performance without
smarter choices around the precise point to split leaf pages at. Making
these two mostly-distinct sets of enhancements into distinct commits
seems like it might clarify their design, even though neither commit is
particularly useful on its own.
The maximum allowed size of new tuples is reduced by an amount equal to
the space required to store an extra MAXALIGN()'d TID in a new high key
during leaf page splits. The user-facing definition of the "1/3 of a
page" restriction is already imprecise, and so does not need to be
revised. However, there should be a compatibility note in the v12
release notes.
Author: Peter Geoghegan
Reviewed-By: Heikki Linnakangas, Alexander Korotkov
Discussion: https://postgr.es/m/CAH2-WzkVb0Kom=R+88fDFb=JSxZMFvbHVC6Mn9LJ2n=X=kS-Uw@mail.gmail.com
2019-03-20 18:04:01 +01:00
enforce uniqueness on the key column(s).
</para>
<para>
<firstterm>Suffix truncation</firstterm> always removes non-key
columns from upper B-Tree levels. As payload columns, they are
never used to guide index scans. The truncation process also
removes one or more trailing key column(s) when the remaining
prefix of key column(s) happens to be sufficient to describe tuples
on the lowest B-Tree level. In practice, covering indexes without
an <literal>INCLUDE</literal> clause often avoid storing columns
that are effectively payload in the upper levels. However,
explicitly defining payload columns as non-key columns
<emphasis>reliably</emphasis> keeps the tuples in upper levels
small.
2018-11-22 19:24:57 +01:00
</para>
<para>
In principle, index-only scans can be used with expression indexes.
For example, given an index on <literal>f(x)</literal>
where <literal>x</literal> is a table column, it should be possible to
execute
<programlisting>
SELECT f(x) FROM tab WHERE f(x) < 1;
</programlisting>
as an index-only scan; and this is very attractive
if <literal>f()</literal> is an expensive-to-compute function.
However, <productname>PostgreSQL</productname>'s planner is currently not
very smart about such cases. It considers a query to be potentially
executable by index-only scan only when all <emphasis>columns</emphasis>
needed by the query are available from the index. In this
example, <literal>x</literal> is not needed except in the
context <literal>f(x)</literal>, but the planner does not notice that and
concludes that an index-only scan is not possible. If an index-only scan
seems sufficiently worthwhile, this can be worked around by
adding <literal>x</literal> as an included column, for example
<programlisting>
CREATE INDEX tab_f_x ON tab (f(x)) INCLUDE (x);
</programlisting>
An additional caveat, if the goal is to avoid
recalculating <literal>f(x)</literal>, is that the planner won't
necessarily match uses of <literal>f(x)</literal> that aren't in
indexable <literal>WHERE</literal> clauses to the index column. It will
usually get this right in simple queries such as shown above, but not in
queries that involve joins. These deficiencies may be remedied in future
versions of <productname>PostgreSQL</productname>.
</para>
<para>
Partial indexes also have interesting interactions with index-only scans.
Consider the partial index shown in <xref linkend="indexes-partial-ex3"/>:
<programlisting>
CREATE UNIQUE INDEX tests_success_constraint ON tests (subject, target)
WHERE success;
</programlisting>
In principle, we could do an index-only scan on this index to satisfy a
query like
<programlisting>
SELECT target FROM tests WHERE subject = 'some-subject' AND success;
</programlisting>
But there's a problem: the <literal>WHERE</literal> clause refers
to <literal>success</literal> which is not available as a result column
of the index. Nonetheless, an index-only scan is possible because the
plan does not need to recheck that part of the <literal>WHERE</literal>
clause at run time: all entries found in the index necessarily
have <literal>success = true</literal> so this need not be explicitly
checked in the plan. <productname>PostgreSQL</productname> versions 9.6
and later will recognize such cases and allow index-only scans to be
generated, but older versions will not.
</para>
</sect1>
2005-09-12 21:17:45 +02:00
<sect1 id="indexes-opclass">
2006-12-23 01:43:13 +01:00
<title>Operator Classes and Operator Families</title>
2005-09-12 21:17:45 +02:00
<indexterm zone="indexes-opclass">
<primary>operator class</primary>
</indexterm>
2006-12-23 01:43:13 +01:00
<indexterm zone="indexes-opclass">
<primary>operator family</primary>
</indexterm>
2005-09-12 21:17:45 +02:00
<para>
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
An index definition can specify an <firstterm>operator
2005-09-12 21:17:45 +02:00
class</firstterm> for each column of an index.
<synopsis>
Implement operator class parameters
PostgreSQL provides set of template index access methods, where opclasses have
much freedom in the semantics of indexing. These index AMs are GiST, GIN,
SP-GiST and BRIN. There opclasses define representation of keys, operations on
them and supported search strategies. So, it's natural that opclasses may be
faced some tradeoffs, which require user-side decision. This commit implements
opclass parameters allowing users to set some values, which tell opclass how to
index the particular dataset.
This commit doesn't introduce new storage in system catalog. Instead it uses
pg_attribute.attoptions, which is used for table column storage options but
unused for index attributes.
In order to evade changing signature of each opclass support function, we
implement unified way to pass options to opclass support functions. Options
are set to fn_expr as the constant bytea expression. It's possible due to the
fact that opclass support functions are executed outside of expressions, so
fn_expr is unused for them.
This commit comes with some examples of opclass options usage. We parametrize
signature length in GiST. That applies to multiple opclasses: tsvector_ops,
gist__intbig_ops, gist_ltree_ops, gist__ltree_ops, gist_trgm_ops and
gist_hstore_ops. Also we parametrize maximum number of integer ranges for
gist__int_ops. However, the main future usage of this feature is expected
to be json, where users would be able to specify which way to index particular
json parts.
Catversion is bumped.
Discussion: https://postgr.es/m/d22c3a18-31c7-1879-fc11-4c1ce2f5e5af%40postgrespro.ru
Author: Nikita Glukhov, revised by me
Reviwed-by: Nikolay Shaplov, Robert Haas, Tom Lane, Tomas Vondra, Alvaro Herrera
2020-03-30 18:17:11 +02:00
CREATE INDEX <replaceable>name</replaceable> ON <replaceable>table</replaceable> (<replaceable>column</replaceable> <replaceable>opclass</replaceable> [ ( <replaceable>opclass_options</replaceable> ) ] <optional><replaceable>sort options</replaceable></optional> <optional>, ...</optional>);
2005-09-12 21:17:45 +02:00
</synopsis>
The operator class identifies the operators to be used by the index
for that column. For example, a B-tree index on the type <type>int4</type>
would use the <literal>int4_ops</literal> class; this operator
class includes comparison functions for values of type <type>int4</type>.
In practice the default operator class for the column's data type is
2009-04-27 18:27:36 +02:00
usually sufficient. The main reason for having operator classes is
2005-09-12 21:17:45 +02:00
that for some data types, there could be more than one meaningful
index behavior. For example, we might want to sort a complex-number data
type either by absolute value or by real part. We could do this by
defining two operator classes for the data type and then selecting
2007-02-14 21:47:15 +01:00
the proper class when making an index. The operator class determines
the basic sort ordering (which can then be modified by adding sort options
2011-02-08 22:04:18 +01:00
<literal>COLLATE</literal>,
2017-10-09 03:44:17 +02:00
<literal>ASC</literal>/<literal>DESC</literal> and/or
<literal>NULLS FIRST</literal>/<literal>NULLS LAST</literal>).
2005-09-12 21:17:45 +02:00
</para>
<para>
There are also some built-in operator classes besides the default ones:
<itemizedlist>
<listitem>
<para>
The operator classes <literal>text_pattern_ops</literal>,
2008-05-27 02:13:09 +02:00
<literal>varchar_pattern_ops</literal>, and
<literal>bpchar_pattern_ops</literal> support B-tree indexes on
the types <type>text</type>, <type>varchar</type>, and
<type>char</type> respectively. The
2005-09-12 21:17:45 +02:00
difference from the default operator classes is that the values
are compared strictly character by character rather than
according to the locale-specific collation rules. This makes
these operator classes suitable for use by queries involving
pattern matching expressions (<literal>LIKE</literal> or POSIX
2008-09-23 11:20:39 +02:00
regular expressions) when the database does not use the standard
2005-09-12 21:17:45 +02:00
<quote>C</quote> locale. As an example, you might index a
<type>varchar</type> column like this:
<programlisting>
CREATE INDEX test_index ON test_table (col varchar_pattern_ops);
</programlisting>
Note that you should also create an index with the default operator
2017-10-09 03:44:17 +02:00
class if you want queries involving ordinary <literal><</literal>,
<literal><=</literal>, <literal>></literal>, or <literal>>=</literal> comparisons
2008-05-27 02:13:09 +02:00
to use an index. Such queries cannot use the
2005-09-12 21:17:45 +02:00
<literal><replaceable>xxx</replaceable>_pattern_ops</literal>
2008-05-27 02:13:09 +02:00
operator classes. (Ordinary equality comparisons can use these
2009-04-27 18:27:36 +02:00
operator classes, however.) It is possible to create multiple
2005-09-12 21:17:45 +02:00
indexes on the same column with different operator classes.
If you do use the C locale, you do not need the
<literal><replaceable>xxx</replaceable>_pattern_ops</literal>
operator classes, because an index with the default operator class
is usable for pattern-matching queries in the C locale.
</para>
</listitem>
</itemizedlist>
</para>
<para>
The following query shows all defined operator classes:
<programlisting>
SELECT am.amname AS index_method,
Add an in-core GiST index opclass for inet/cidr types.
This operator class can accelerate subnet/supernet tests as well as
btree-equivalent ordered comparisons. It also handles a new network
operator inet && inet (overlaps, a/k/a "is supernet or subnet of"),
which is expected to be useful in exclusion constraints.
Ideally this opclass would be the default for GiST with inet/cidr data,
but we can't mark it that way until we figure out how to do a more or
less graceful transition from the current situation, in which the
really-completely-bogus inet/cidr opclasses in contrib/btree_gist are
marked as default. Having the opclass in core and not default is better
than not having it at all, though.
While at it, add new documentation sections to allow us to officially
document GiST/GIN/SP-GiST opclasses, something there was never a clear
place to do before. I filled these in with some simple tables listing
the existing opclasses and the operators they support, but there's
certainly scope to put more information there.
Emre Hasegeli, reviewed by Andreas Karlsson, further hacking by me
2014-04-08 21:46:14 +02:00
opc.opcname AS opclass_name,
opc.opcintype::regtype AS indexed_type,
opc.opcdefault AS is_default
2005-09-12 21:17:45 +02:00
FROM pg_am am, pg_opclass opc
2006-12-23 01:43:13 +01:00
WHERE opc.opcmethod = am.oid
2005-09-12 21:17:45 +02:00
ORDER BY index_method, opclass_name;
</programlisting>
2006-12-23 01:43:13 +01:00
</para>
2005-09-12 21:17:45 +02:00
2006-12-23 01:43:13 +01:00
<para>
An operator class is actually just a subset of a larger structure called an
2017-10-09 03:44:17 +02:00
<firstterm>operator family</firstterm>. In cases where several data types have
2006-12-23 01:43:13 +01:00
similar behaviors, it is frequently useful to define cross-data-type
operators and allow these to work with indexes. To do this, the operator
classes for each of the types must be grouped into the same operator
family. The cross-type operators are members of the family, but are not
associated with any single class within the family.
</para>
Add an in-core GiST index opclass for inet/cidr types.
This operator class can accelerate subnet/supernet tests as well as
btree-equivalent ordered comparisons. It also handles a new network
operator inet && inet (overlaps, a/k/a "is supernet or subnet of"),
which is expected to be useful in exclusion constraints.
Ideally this opclass would be the default for GiST with inet/cidr data,
but we can't mark it that way until we figure out how to do a more or
less graceful transition from the current situation, in which the
really-completely-bogus inet/cidr opclasses in contrib/btree_gist are
marked as default. Having the opclass in core and not default is better
than not having it at all, though.
While at it, add new documentation sections to allow us to officially
document GiST/GIN/SP-GiST opclasses, something there was never a clear
place to do before. I filled these in with some simple tables listing
the existing opclasses and the operators they support, but there's
certainly scope to put more information there.
Emre Hasegeli, reviewed by Andreas Karlsson, further hacking by me
2014-04-08 21:46:14 +02:00
<para>
This expanded version of the previous query shows the operator family
each operator class belongs to:
<programlisting>
SELECT am.amname AS index_method,
opc.opcname AS opclass_name,
opf.opfname AS opfamily_name,
opc.opcintype::regtype AS indexed_type,
opc.opcdefault AS is_default
FROM pg_am am, pg_opclass opc, pg_opfamily opf
WHERE opc.opcmethod = am.oid AND
opc.opcfamily = opf.oid
ORDER BY index_method, opclass_name;
</programlisting>
</para>
2006-12-23 01:43:13 +01:00
<para>
This query shows all defined operator families and all
the operators included in each family:
2005-09-12 21:17:45 +02:00
<programlisting>
SELECT am.amname AS index_method,
2006-12-23 01:43:13 +01:00
opf.opfname AS opfamily_name,
amop.amopopr::regoperator AS opfamily_operator
FROM pg_am am, pg_opfamily opf, pg_amop amop
WHERE opf.opfmethod = am.oid AND
amop.amopfamily = opf.oid
ORDER BY index_method, opfamily_name, opfamily_operator;
2005-09-12 21:17:45 +02:00
</programlisting>
</para>
2020-08-02 23:00:26 +02:00
<tip>
<para>
<xref linkend="app-psql"/> has
commands <command>\dAc</command>, <command>\dAf</command>,
and <command>\dAo</command>, which provide slightly more sophisticated
versions of these queries.
</para>
</tip>
2005-09-12 21:17:45 +02:00
</sect1>
2011-02-08 22:04:18 +01:00
<sect1 id="indexes-collations">
2011-04-22 23:43:18 +02:00
<title>Indexes and Collations</title>
2011-02-08 22:04:18 +01:00
<para>
2011-04-22 23:43:18 +02:00
An index can support only one collation per index column.
If multiple collations are of interest, multiple indexes may be needed.
2011-02-08 22:04:18 +01:00
</para>
<para>
Consider these statements:
<programlisting>
CREATE TABLE test1c (
id integer,
content varchar COLLATE "x"
);
CREATE INDEX test1c_content_index ON test1c (content);
</programlisting>
2011-04-22 23:43:18 +02:00
The index automatically uses the collation of the
underlying column. So a query of the form
2011-02-08 22:04:18 +01:00
<programlisting>
2011-04-22 23:43:18 +02:00
SELECT * FROM test1c WHERE content > <replaceable>constant</replaceable>;
2011-02-08 22:04:18 +01:00
</programlisting>
2011-04-22 23:43:18 +02:00
could use the index, because the comparison will by default use the
collation of the column. However, this index cannot accelerate queries
that involve some other collation. So if queries of the form, say,
2011-02-08 22:04:18 +01:00
<programlisting>
SELECT * FROM test1c WHERE content > <replaceable>constant</replaceable> COLLATE "y";
</programlisting>
2011-04-22 23:43:18 +02:00
are also of interest, an additional index could be created that supports
the <literal>"y"</literal> collation, like this:
2011-02-08 22:04:18 +01:00
<programlisting>
2011-04-22 23:43:18 +02:00
CREATE INDEX test1c_content_y_index ON test1c (content COLLATE "y");
2011-02-08 22:04:18 +01:00
</programlisting>
</para>
</sect1>
2001-10-31 21:38:26 +01:00
<sect1 id="indexes-examine">
<title>Examining Index Usage</title>
2003-08-31 19:32:24 +02:00
<indexterm zone="indexes-examine">
<primary>index</primary>
<secondary>examining usage</secondary>
</indexterm>
2001-10-31 21:38:26 +01:00
<para>
2017-10-09 03:44:17 +02:00
Although indexes in <productname>PostgreSQL</productname> do not need
2009-04-27 18:27:36 +02:00
maintenance or tuning, it is still important to check
2001-10-31 21:38:26 +01:00
which indexes are actually used by the real-life query workload.
2003-09-30 05:22:33 +02:00
Examining index usage for an individual query is done with the
2017-11-23 15:39:47 +01:00
<xref linkend="sql-explain"/>
2003-11-06 23:21:47 +01:00
command; its application for this purpose is
2017-11-23 15:39:47 +01:00
illustrated in <xref linkend="using-explain"/>.
2003-09-30 05:22:33 +02:00
It is also possible to gather overall statistics about index usage
2017-11-23 15:39:47 +01:00
in a running server, as described in <xref linkend="monitoring-stats"/>.
2001-10-31 21:38:26 +01:00
</para>
<para>
It is difficult to formulate a general procedure for determining
2009-04-27 18:27:36 +02:00
which indexes to create. There are a number of typical cases that
2001-10-31 21:38:26 +01:00
have been shown in the examples throughout the previous sections.
2009-04-27 18:27:36 +02:00
A good deal of experimentation is often necessary.
The rest of this section gives some tips for that:
2001-10-31 21:38:26 +01:00
</para>
<itemizedlist>
<listitem>
<para>
2017-11-23 15:39:47 +01:00
Always run <xref linkend="sql-analyze"/>
2003-11-06 23:21:47 +01:00
first. This command
2001-10-31 21:38:26 +01:00
collects statistics about the distribution of the values in the
2009-04-27 18:27:36 +02:00
table. This information is required to estimate the number of rows
2001-10-31 21:38:26 +01:00
returned by a query, which is needed by the planner to assign
realistic costs to each possible query plan. In absence of any
real statistics, some default values are assumed, which are
almost certain to be inaccurate. Examining an application's
index usage without having run <command>ANALYZE</command> is
2010-01-01 22:53:49 +01:00
therefore a lost cause.
2017-11-23 15:39:47 +01:00
See <xref linkend="vacuum-for-statistics"/>
and <xref linkend="autovacuum"/> for more information.
2001-10-31 21:38:26 +01:00
</para>
</listitem>
<listitem>
<para>
Use real data for experimentation. Using test data for setting
up indexes will tell you what indexes you need for the test data,
but that is all.
</para>
<para>
2004-12-24 00:07:38 +01:00
It is especially fatal to use very small test data sets.
2001-10-31 21:38:26 +01:00
While selecting 1000 out of 100000 rows could be a candidate for
an index, selecting 1 out of 100 rows will hardly be, because the
2009-04-27 18:27:36 +02:00
100 rows probably fit within a single disk page, and there
2001-10-31 21:38:26 +01:00
is no plan that can beat sequentially fetching 1 disk page.
</para>
<para>
Also be careful when making up test data, which is often
2009-04-27 18:27:36 +02:00
unavoidable when the application is not yet in production.
2001-10-31 21:38:26 +01:00
Values that are very similar, completely random, or inserted in
sorted order will skew the statistics away from the distribution
that real data would have.
</para>
</listitem>
<listitem>
<para>
When indexes are not used, it can be useful for testing to force
their use. There are run-time parameters that can turn off
2017-11-23 15:39:47 +01:00
various plan types (see <xref linkend="runtime-config-query-enable"/>).
2002-11-11 21:14:04 +01:00
For instance, turning off sequential scans
2017-10-09 03:44:17 +02:00
(<varname>enable_seqscan</varname>) and nested-loop joins
(<varname>enable_nestloop</varname>), which are the most basic plans,
2001-10-31 21:38:26 +01:00
will force the system to use a different plan. If the system
still chooses a sequential scan or nested-loop join then there is
2009-04-27 18:27:36 +02:00
probably a more fundamental reason why the index is not being
2005-09-12 21:17:45 +02:00
used; for example, the query condition does not match the index.
2001-10-31 21:38:26 +01:00
(What kind of query can use what kind of index is explained in
the previous sections.)
</para>
</listitem>
<listitem>
<para>
If forcing index usage does use the index, then there are two
possibilities: Either the system is right and using the index is
indeed not appropriate, or the cost estimates of the query plans
are not reflecting reality. So you should time your query with
and without indexes. The <command>EXPLAIN ANALYZE</command>
command can be useful here.
</para>
</listitem>
<listitem>
<para>
If it turns out that the cost estimates are wrong, there are,
again, two possibilities. The total cost is computed from the
per-row costs of each plan node times the selectivity estimate of
2005-09-12 21:17:45 +02:00
the plan node. The costs estimated for the plan nodes can be adjusted
via run-time parameters (described in <xref
2017-11-23 15:39:47 +01:00
linkend="runtime-config-query-constants"/>).
2002-11-11 21:14:04 +01:00
An inaccurate selectivity estimate is due to
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
insufficient statistics. It might be possible to improve this by
2003-11-06 23:21:47 +01:00
tuning the statistics-gathering parameters (see
2017-11-23 15:39:47 +01:00
<xref linkend="sql-altertable"/>).
2001-10-31 21:38:26 +01:00
</para>
<para>
If you do not succeed in adjusting the costs to be more
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
appropriate, then you might have to resort to forcing index usage
explicitly. You might also want to contact the
2017-10-09 03:44:17 +02:00
<productname>PostgreSQL</productname> developers to examine the issue.
2001-10-31 21:38:26 +01:00
</para>
</listitem>
</itemizedlist>
</sect1>
</chapter>