mirror of
https://git.postgresql.org/git/postgresql.git
synced 2024-09-16 01:40:33 +02:00
9b5c8d45f6
"consistent" functions, and remove pg_amop.opreqcheck, as per recent discussion. The main immediate benefit of this is that we no longer need 8.3's ugly hack of requiring @@@ rather than @@ to test weight-using tsquery searches on GIN indexes. In future it should be possible to optimize some other queries better than is done now, by detecting at runtime whether the index match is exact or not. Tom Lane, after an idea of Heikki's, and with some help from Teodor.
278 lines
9.1 KiB
Plaintext
278 lines
9.1 KiB
Plaintext
<!-- $PostgreSQL: pgsql/doc/src/sgml/gist.sgml,v 1.30 2008/04/14 17:05:32 tgl Exp $ -->
|
|
|
|
<chapter id="GiST">
|
|
<title>GiST Indexes</title>
|
|
|
|
<indexterm>
|
|
<primary>index</primary>
|
|
<secondary>GiST</secondary>
|
|
</indexterm>
|
|
|
|
<sect1 id="gist-intro">
|
|
<title>Introduction</title>
|
|
|
|
<para>
|
|
<acronym>GiST</acronym> stands for Generalized Search Tree. It is a
|
|
balanced, tree-structured access method, that acts as a base template in
|
|
which to implement arbitrary indexing schemes. B-trees, R-trees and many
|
|
other indexing schemes can be implemented in <acronym>GiST</acronym>.
|
|
</para>
|
|
|
|
<para>
|
|
One advantage of <acronym>GiST</acronym> is that it allows the development
|
|
of custom data types with the appropriate access methods, by
|
|
an expert in the domain of the data type, rather than a database expert.
|
|
</para>
|
|
|
|
<para>
|
|
Some of the information here is derived from the University of California at
|
|
Berkeley's GiST Indexing Project
|
|
<ulink url="http://gist.cs.berkeley.edu/">web site</ulink> and
|
|
<ulink url="http://www.sai.msu.su/~megera/postgres/gist/papers/concurrency/access-methods-for-next-generation.pdf.gz">
|
|
Marcel Kornacker's thesis, Access Methods for Next-Generation Database Systems</ulink>.
|
|
The <acronym>GiST</acronym>
|
|
implementation in <productname>PostgreSQL</productname> is primarily
|
|
maintained by Teodor Sigaev and Oleg Bartunov, and there is more
|
|
information on their
|
|
<ulink url="http://www.sai.msu.su/~megera/postgres/gist/">website</ulink>.
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="gist-extensibility">
|
|
<title>Extensibility</title>
|
|
|
|
<para>
|
|
Traditionally, implementing a new index access method meant a lot of
|
|
difficult work. It was necessary to understand the inner workings of the
|
|
database, such as the lock manager and Write-Ahead Log. The
|
|
<acronym>GiST</acronym> interface has a high level of abstraction,
|
|
requiring the access method implementer to only implement the semantics of
|
|
the data type being accessed. The <acronym>GiST</acronym> layer itself
|
|
takes care of concurrency, logging and searching the tree structure.
|
|
</para>
|
|
|
|
<para>
|
|
This extensibility should not be confused with the extensibility of the
|
|
other standard search trees in terms of the data they can handle. For
|
|
example, <productname>PostgreSQL</productname> supports extensible B-trees
|
|
and hash indexes. That means that you can use
|
|
<productname>PostgreSQL</productname> to build a B-tree or hash over any
|
|
data type you want. But B-trees only support range predicates
|
|
(<literal><</literal>, <literal>=</literal>, <literal>></literal>),
|
|
and hash indexes only support equality queries.
|
|
</para>
|
|
|
|
<para>
|
|
So if you index, say, an image collection with a
|
|
<productname>PostgreSQL</productname> B-tree, you can only issue queries
|
|
such as <quote>is imagex equal to imagey</quote>, <quote>is imagex less
|
|
than imagey</quote> and <quote>is imagex greater than imagey</quote>?
|
|
Depending on how you define <quote>equals</quote>, <quote>less than</quote>
|
|
and <quote>greater than</quote> in this context, this could be useful.
|
|
However, by using a <acronym>GiST</acronym> based index, you could create
|
|
ways to ask domain-specific questions, perhaps <quote>find all images of
|
|
horses</quote> or <quote>find all over-exposed images</quote>.
|
|
</para>
|
|
|
|
<para>
|
|
All it takes to get a <acronym>GiST</acronym> access method up and running
|
|
is to implement seven user-defined methods, which define the behavior of
|
|
keys in the tree. Of course these methods have to be pretty fancy to
|
|
support fancy queries, but for all the standard queries (B-trees,
|
|
R-trees, etc.) they're relatively straightforward. In short,
|
|
<acronym>GiST</acronym> combines extensibility along with generality, code
|
|
reuse, and a clean interface.
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="gist-implementation">
|
|
<title>Implementation</title>
|
|
|
|
<para>
|
|
There are seven methods that an index operator class for
|
|
<acronym>GiST</acronym> must provide:
|
|
</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>consistent</term>
|
|
<listitem>
|
|
<para>
|
|
Given a predicate <literal>p</literal> on a tree page, and a user
|
|
query, <literal>q</literal>, this method will return false if it is
|
|
certain that both <literal>p</literal> and <literal>q</literal> cannot
|
|
be true for a given data item. For a true result, a
|
|
<literal>recheck</> flag must also be returned; this indicates whether
|
|
the predicate implies the query (<literal>recheck</> = false) or
|
|
not (<literal>recheck</> = true).
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>union</term>
|
|
<listitem>
|
|
<para>
|
|
This method consolidates information in the tree. Given a set of
|
|
entries, this function generates a new predicate that is true for all
|
|
the entries.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>compress</term>
|
|
<listitem>
|
|
<para>
|
|
Converts the data item into a format suitable for physical storage in
|
|
an index page.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>decompress</term>
|
|
<listitem>
|
|
<para>
|
|
The reverse of the <function>compress</function> method. Converts the
|
|
index representation of the data item into a format that can be
|
|
manipulated by the database.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>penalty</term>
|
|
<listitem>
|
|
<para>
|
|
Returns a value indicating the <quote>cost</quote> of inserting the new
|
|
entry into a particular branch of the tree. items will be inserted
|
|
down the path of least <function>penalty</function> in the tree.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>picksplit</term>
|
|
<listitem>
|
|
<para>
|
|
When a page split is necessary, this function decides which entries on
|
|
the page are to stay on the old page, and which are to move to the new
|
|
page.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>same</term>
|
|
<listitem>
|
|
<para>
|
|
Returns true if two entries are identical, false otherwise.
|
|
</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="gist-examples">
|
|
<title>Examples</title>
|
|
|
|
<para>
|
|
The <productname>PostgreSQL</productname> source distribution includes
|
|
several examples of index methods implemented using
|
|
<acronym>GiST</acronym>. The core system currently provides text search
|
|
support (indexing for <type>tsvector</> and <type>tsquery</>) as well as
|
|
R-Tree equivalent functionality for some of the built-in geometric data types
|
|
(see <filename>src/backend/access/gist/gistproc.c</>). The following
|
|
<filename>contrib</> modules also contain <acronym>GiST</acronym>
|
|
operator classes:
|
|
</para>
|
|
|
|
<variablelist>
|
|
<varlistentry>
|
|
<term>btree_gist</term>
|
|
<listitem>
|
|
<para>B-Tree equivalent functionality for several data types</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>cube</term>
|
|
<listitem>
|
|
<para>Indexing for multidimensional cubes</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>hstore</term>
|
|
<listitem>
|
|
<para>Module for storing (key, value) pairs</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>intarray</term>
|
|
<listitem>
|
|
<para>RD-Tree for one-dimensional array of int4 values</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>ltree</term>
|
|
<listitem>
|
|
<para>Indexing for tree-like structures</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>pg_trgm</term>
|
|
<listitem>
|
|
<para>Text similarity using trigram matching</para>
|
|
</listitem>
|
|
</varlistentry>
|
|
|
|
<varlistentry>
|
|
<term>seg</term>
|
|
<listitem>
|
|
<para>Indexing for <quote>float ranges</quote></para>
|
|
</listitem>
|
|
</varlistentry>
|
|
</variablelist>
|
|
|
|
</sect1>
|
|
|
|
<sect1 id="gist-recovery">
|
|
<title>Crash Recovery</title>
|
|
|
|
<para>
|
|
Usually, replay of the WAL log is sufficient to restore the integrity
|
|
of a GiST index following a database crash. However, there are some
|
|
corner cases in which the index state is not fully rebuilt. The index
|
|
will still be functionally correct, but there might be some performance
|
|
degradation. When this occurs, the index can be repaired by
|
|
<command>VACUUM</>ing its table, or by rebuilding the index using
|
|
<command>REINDEX</>. In some cases a plain <command>VACUUM</> is
|
|
not sufficient, and either <command>VACUUM FULL</> or <command>REINDEX</>
|
|
is needed. The need for one of these procedures is indicated by occurrence
|
|
of this log message during crash recovery:
|
|
<programlisting>
|
|
LOG: index NNN/NNN/NNN needs VACUUM or REINDEX to finish crash recovery
|
|
</programlisting>
|
|
or this log message during routine index insertions:
|
|
<programlisting>
|
|
LOG: index "FOO" needs VACUUM or REINDEX to finish crash recovery
|
|
</programlisting>
|
|
If a plain <command>VACUUM</> finds itself unable to complete recovery
|
|
fully, it will return a notice:
|
|
<programlisting>
|
|
NOTICE: index "FOO" needs VACUUM FULL or REINDEX to finish crash recovery
|
|
</programlisting>
|
|
</para>
|
|
</sect1>
|
|
|
|
</chapter>
|