2008-04-14 19:05:34 +02:00
|
|
|
<!-- $PostgreSQL: pgsql/doc/src/sgml/gist.sgml,v 1.30 2008/04/14 17:05:32 tgl Exp $ -->
|
2003-09-29 20:18:35 +02:00
|
|
|
|
2005-01-08 23:13:38 +01:00
|
|
|
<chapter id="GiST">
|
2003-10-31 23:41:21 +01:00
|
|
|
<title>GiST Indexes</title>
|
|
|
|
|
2005-01-08 23:13:38 +01:00
|
|
|
<indexterm>
|
|
|
|
<primary>index</primary>
|
|
|
|
<secondary>GiST</secondary>
|
|
|
|
</indexterm>
|
2005-11-07 18:36:47 +01:00
|
|
|
|
|
|
|
<sect1 id="gist-intro">
|
|
|
|
<title>Introduction</title>
|
|
|
|
|
|
|
|
<para>
|
2003-10-31 23:41:21 +01:00
|
|
|
<acronym>GiST</acronym> stands for Generalized Search Tree. It is a
|
|
|
|
balanced, tree-structured access method, that acts as a base template in
|
2005-11-07 18:36:47 +01:00
|
|
|
which to implement arbitrary indexing schemes. B-trees, R-trees and many
|
2003-10-31 23:41:21 +01:00
|
|
|
other indexing schemes can be implemented in <acronym>GiST</acronym>.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
One advantage of <acronym>GiST</acronym> is that it allows the development
|
|
|
|
of custom data types with the appropriate access methods, by
|
|
|
|
an expert in the domain of the data type, rather than a database expert.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
2005-03-31 05:54:38 +02:00
|
|
|
Some of the information here is derived from the University of California at
|
2005-06-29 03:23:49 +02:00
|
|
|
Berkeley's GiST Indexing Project
|
|
|
|
<ulink url="http://gist.cs.berkeley.edu/">web site</ulink> and
|
2005-10-21 15:59:05 +02:00
|
|
|
<ulink url="http://www.sai.msu.su/~megera/postgres/gist/papers/concurrency/access-methods-for-next-generation.pdf.gz">
|
2005-06-29 03:23:49 +02:00
|
|
|
Marcel Kornacker's thesis, Access Methods for Next-Generation Database Systems</ulink>.
|
|
|
|
The <acronym>GiST</acronym>
|
2003-10-31 23:41:21 +01:00
|
|
|
implementation in <productname>PostgreSQL</productname> is primarily
|
|
|
|
maintained by Teodor Sigaev and Oleg Bartunov, and there is more
|
2005-06-29 03:23:49 +02:00
|
|
|
information on their
|
|
|
|
<ulink url="http://www.sai.msu.su/~megera/postgres/gist/">website</ulink>.
|
2003-10-31 23:41:21 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
</sect1>
|
|
|
|
|
2005-10-21 03:41:28 +02:00
|
|
|
<sect1 id="gist-extensibility">
|
2003-10-31 23:41:21 +01:00
|
|
|
<title>Extensibility</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Traditionally, implementing a new index access method meant a lot of
|
|
|
|
difficult work. It was necessary to understand the inner workings of the
|
|
|
|
database, such as the lock manager and Write-Ahead Log. The
|
|
|
|
<acronym>GiST</acronym> interface has a high level of abstraction,
|
2005-11-05 00:14:02 +01:00
|
|
|
requiring the access method implementer to only implement the semantics of
|
2003-10-31 23:41:21 +01:00
|
|
|
the data type being accessed. The <acronym>GiST</acronym> layer itself
|
|
|
|
takes care of concurrency, logging and searching the tree structure.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
This extensibility should not be confused with the extensibility of the
|
|
|
|
other standard search trees in terms of the data they can handle. For
|
2005-11-07 18:36:47 +01:00
|
|
|
example, <productname>PostgreSQL</productname> supports extensible B-trees
|
|
|
|
and hash indexes. That means that you can use
|
|
|
|
<productname>PostgreSQL</productname> to build a B-tree or hash over any
|
|
|
|
data type you want. But B-trees only support range predicates
|
2003-10-31 23:41:21 +01:00
|
|
|
(<literal><</literal>, <literal>=</literal>, <literal>></literal>),
|
2005-11-07 18:36:47 +01:00
|
|
|
and hash indexes only support equality queries.
|
2003-10-31 23:41:21 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
So if you index, say, an image collection with a
|
2005-11-07 18:36:47 +01:00
|
|
|
<productname>PostgreSQL</productname> B-tree, you can only issue queries
|
2003-10-31 23:41:21 +01:00
|
|
|
such as <quote>is imagex equal to imagey</quote>, <quote>is imagex less
|
|
|
|
than imagey</quote> and <quote>is imagex greater than imagey</quote>?
|
|
|
|
Depending on how you define <quote>equals</quote>, <quote>less than</quote>
|
|
|
|
and <quote>greater than</quote> in this context, this could be useful.
|
|
|
|
However, by using a <acronym>GiST</acronym> based index, you could create
|
|
|
|
ways to ask domain-specific questions, perhaps <quote>find all images of
|
|
|
|
horses</quote> or <quote>find all over-exposed images</quote>.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
All it takes to get a <acronym>GiST</acronym> access method up and running
|
|
|
|
is to implement seven user-defined methods, which define the behavior of
|
|
|
|
keys in the tree. Of course these methods have to be pretty fancy to
|
2005-11-07 18:36:47 +01:00
|
|
|
support fancy queries, but for all the standard queries (B-trees,
|
2003-10-31 23:41:21 +01:00
|
|
|
R-trees, etc.) they're relatively straightforward. In short,
|
|
|
|
<acronym>GiST</acronym> combines extensibility along with generality, code
|
|
|
|
reuse, and a clean interface.
|
|
|
|
</para>
|
|
|
|
|
|
|
|
</sect1>
|
|
|
|
|
2005-10-21 03:41:28 +02:00
|
|
|
<sect1 id="gist-implementation">
|
2003-10-31 23:41:21 +01:00
|
|
|
<title>Implementation</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
There are seven methods that an index operator class for
|
|
|
|
<acronym>GiST</acronym> must provide:
|
|
|
|
</para>
|
|
|
|
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
|
|
<term>consistent</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Given a predicate <literal>p</literal> on a tree page, and a user
|
|
|
|
query, <literal>q</literal>, this method will return false if it is
|
|
|
|
certain that both <literal>p</literal> and <literal>q</literal> cannot
|
2008-04-14 19:05:34 +02:00
|
|
|
be true for a given data item. For a true result, a
|
|
|
|
<literal>recheck</> flag must also be returned; this indicates whether
|
|
|
|
the predicate implies the query (<literal>recheck</> = false) or
|
|
|
|
not (<literal>recheck</> = true).
|
2003-10-31 23:41:21 +01:00
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>union</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
This method consolidates information in the tree. Given a set of
|
|
|
|
entries, this function generates a new predicate that is true for all
|
|
|
|
the entries.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>compress</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Converts the data item into a format suitable for physical storage in
|
|
|
|
an index page.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>decompress</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
The reverse of the <function>compress</function> method. Converts the
|
|
|
|
index representation of the data item into a format that can be
|
|
|
|
manipulated by the database.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>penalty</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Returns a value indicating the <quote>cost</quote> of inserting the new
|
|
|
|
entry into a particular branch of the tree. items will be inserted
|
|
|
|
down the path of least <function>penalty</function> in the tree.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>picksplit</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
When a page split is necessary, this function decides which entries on
|
|
|
|
the page are to stay on the old page, and which are to move to the new
|
|
|
|
page.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>same</term>
|
|
|
|
<listitem>
|
|
|
|
<para>
|
|
|
|
Returns true if two entries are identical, false otherwise.
|
|
|
|
</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
</variablelist>
|
|
|
|
|
|
|
|
</sect1>
|
|
|
|
|
2005-10-21 03:41:28 +02:00
|
|
|
<sect1 id="gist-examples">
|
2003-10-31 23:41:21 +01:00
|
|
|
<title>Examples</title>
|
|
|
|
|
|
|
|
<para>
|
2005-10-21 03:41:28 +02:00
|
|
|
The <productname>PostgreSQL</productname> source distribution includes
|
|
|
|
several examples of index methods implemented using
|
2007-11-14 00:36:26 +01:00
|
|
|
<acronym>GiST</acronym>. The core system currently provides text search
|
|
|
|
support (indexing for <type>tsvector</> and <type>tsquery</>) as well as
|
|
|
|
R-Tree equivalent functionality for some of the built-in geometric data types
|
2005-10-21 03:41:28 +02:00
|
|
|
(see <filename>src/backend/access/gist/gistproc.c</>). The following
|
|
|
|
<filename>contrib</> modules also contain <acronym>GiST</acronym>
|
|
|
|
operator classes:
|
2003-10-31 23:41:21 +01:00
|
|
|
</para>
|
|
|
|
|
|
|
|
<variablelist>
|
|
|
|
<varlistentry>
|
|
|
|
<term>btree_gist</term>
|
|
|
|
<listitem>
|
2005-11-05 00:14:02 +01:00
|
|
|
<para>B-Tree equivalent functionality for several data types</para>
|
2003-10-31 23:41:21 +01:00
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>cube</term>
|
|
|
|
<listitem>
|
2006-10-23 20:10:32 +02:00
|
|
|
<para>Indexing for multidimensional cubes</para>
|
2003-10-31 23:41:21 +01:00
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
2007-11-14 00:36:26 +01:00
|
|
|
<varlistentry>
|
|
|
|
<term>hstore</term>
|
|
|
|
<listitem>
|
|
|
|
<para>Module for storing (key, value) pairs</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
2003-10-31 23:41:21 +01:00
|
|
|
<varlistentry>
|
|
|
|
<term>intarray</term>
|
|
|
|
<listitem>
|
|
|
|
<para>RD-Tree for one-dimensional array of int4 values</para>
|
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>ltree</term>
|
|
|
|
<listitem>
|
2005-10-21 03:41:28 +02:00
|
|
|
<para>Indexing for tree-like structures</para>
|
2003-10-31 23:41:21 +01:00
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
2005-10-21 03:41:28 +02:00
|
|
|
<term>pg_trgm</term>
|
2003-10-31 23:41:21 +01:00
|
|
|
<listitem>
|
2005-10-21 03:41:28 +02:00
|
|
|
<para>Text similarity using trigram matching</para>
|
2003-10-31 23:41:21 +01:00
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
|
|
|
|
<varlistentry>
|
|
|
|
<term>seg</term>
|
|
|
|
<listitem>
|
2005-10-21 03:41:28 +02:00
|
|
|
<para>Indexing for <quote>float ranges</quote></para>
|
2003-10-31 23:41:21 +01:00
|
|
|
</listitem>
|
|
|
|
</varlistentry>
|
|
|
|
</variablelist>
|
|
|
|
|
|
|
|
</sect1>
|
|
|
|
|
2005-10-21 03:41:28 +02:00
|
|
|
<sect1 id="gist-recovery">
|
|
|
|
<title>Crash Recovery</title>
|
|
|
|
|
|
|
|
<para>
|
|
|
|
Usually, replay of the WAL log is sufficient to restore the integrity
|
|
|
|
of a GiST index following a database crash. However, there are some
|
|
|
|
corner cases in which the index state is not fully rebuilt. The index
|
Update documentation on may/can/might:
Standard English uses "may", "can", and "might" in different ways:
may - permission, "You may borrow my rake."
can - ability, "I can lift that log."
might - possibility, "It might rain today."
Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice. Similarly, "It may crash" is better stated, "It might crash".
Also update two error messages mentioned in the documenation to match.
2007-01-31 21:56:20 +01:00
|
|
|
will still be functionally correct, but there might be some performance
|
2005-10-21 03:41:28 +02:00
|
|
|
degradation. When this occurs, the index can be repaired by
|
|
|
|
<command>VACUUM</>ing its table, or by rebuilding the index using
|
|
|
|
<command>REINDEX</>. In some cases a plain <command>VACUUM</> is
|
|
|
|
not sufficient, and either <command>VACUUM FULL</> or <command>REINDEX</>
|
|
|
|
is needed. The need for one of these procedures is indicated by occurrence
|
|
|
|
of this log message during crash recovery:
|
|
|
|
<programlisting>
|
|
|
|
LOG: index NNN/NNN/NNN needs VACUUM or REINDEX to finish crash recovery
|
|
|
|
</programlisting>
|
|
|
|
or this log message during routine index insertions:
|
|
|
|
<programlisting>
|
|
|
|
LOG: index "FOO" needs VACUUM or REINDEX to finish crash recovery
|
|
|
|
</programlisting>
|
|
|
|
If a plain <command>VACUUM</> finds itself unable to complete recovery
|
|
|
|
fully, it will return a notice:
|
|
|
|
<programlisting>
|
|
|
|
NOTICE: index "FOO" needs VACUUM FULL or REINDEX to finish crash recovery
|
|
|
|
</programlisting>
|
|
|
|
</para>
|
|
|
|
</sect1>
|
|
|
|
|
2003-10-31 23:41:21 +01:00
|
|
|
</chapter>
|