BRIN Indexes
index
BRIN
Introduction
BRIN stands for Block Range Index.
BRIN is designed for handling very large tables
in which certain columns have some natural correlation with their
physical location within the table.
A block range> is a group of pages that are physically
adjacent in the table; for each block range, some summary info is stored
by the index.
For example, a table storing a store's sale orders might have
a date column on which each order was placed, and most of the time
the entries for earlier orders will appear earlier in the table as well;
a table storing a ZIP code column might have all codes for a city
grouped together naturally.
BRIN indexes can satisfy queries via regular bitmap
index scans, and will return all tuples in all pages within each range if
the summary info stored by the index is consistent> with the
query conditions.
The query executor is in charge of rechecking these tuples and discarding
those that do not match the query conditions — in other words, these
indexes are lossy.
Because a BRIN index is very small, scanning the index
adds little overhead compared to a sequential scan, but may avoid scanning
large parts of the table that are known not to contain matching tuples.
The specific data that a BRIN index will store,
as well as the specific queries that the index will be able to satisfy,
depend on the operator class selected for each column of the index.
Data types having a linear sort order can have operator classes that
store the minimum and maximum value within each block range, for instance;
geometrical types might store the bounding box for all the objects
in the block range.
The size of the block range is determined at index creation time by
the pages_per_range> storage parameter. The number of index
entries will be equal to the size of the relation in pages divided by
the selected value for pages_per_range>. Therefore, the smaller
the number, the larger the index becomes (because of the need to
store more index entries), but at the same time the summary data stored can
be more precise and more data blocks can be skipped during an index scan.
Index Maintenance
At the time of creation, all existing index pages are scanned and a
summary index tuple is created for each range, including the
possibly-incomplete range at the end.
As new pages are filled with data, page ranges that are already
summarized will cause the summary information to be updated with data
from the new tuples.
When a new page is created that does not fall within the last
summarized range, that range does not automatically acquire a summary
tuple; those tuples remain unsummarized until a summarization run is
invoked later, creating initial summaries.
This process can be invoked manually using the
brin_summarize_new_pages(regclass) function,
or automatically when VACUUM processes the table.
Built-in Operator Classes
The core PostgreSQL distribution
includes the BRIN operator classes shown in
.
The minmax>
operator classes store the minimum and the maximum values appearing
in the indexed column within the range. The inclusion>
operator classes store a value which includes the values in the indexed
column within the range.
Built-in BRIN Operator Classes
Name
Indexed Data Type
Indexable Operators
abstime_minmax_ops
abstime
<
<=
=
>=
>
int8_minmax_ops
bigint
<
<=
=
>=
>
bit_minmax_ops
bit
<
<=
=
>=
>
varbit_minmax_ops
bit varying
<
<=
=
>=
>
box_inclusion_ops>
box
<<>
&<>
&&>
&>>
>>>
~=>
@>>
<@>
&<|>
<<|>
|>>
|&>>
bytea_minmax_ops
bytea
<
<=
=
>=
>
bpchar_minmax_ops
character
<
<=
=
>=
>
char_minmax_ops
"char"
<
<=
=
>=
>
date_minmax_ops
date
<
<=
=
>=
>
float8_minmax_ops
double precision
<
<=
=
>=
>
inet_minmax_ops
inet
<
<=
=
>=
>
network_inclusion_ops
inet
&&>
>>=>
<<=
=
>>>
<<
int4_minmax_ops
integer
<
<=
=
>=
>
interval_minmax_ops
interval
<
<=
=
>=
>
macaddr_minmax_ops
macaddr
<
<=
=
>=
>
name_minmax_ops
name
<
<=
=
>=
>
numeric_minmax_ops
numeric
<
<=
=
>=
>
pg_lsn_minmax_ops
pg_lsn
<
<=
=
>=
>
oid_minmax_ops
oid
<
<=
=
>=
>
range_inclusion_ops>
any range type
<<>
&<>
&&>
&>>
>>>
@>>
<@>
-|->
=>
<
<=
=
>
>=
float4_minmax_ops
real
<
<=
=
>=
>
reltime_minmax_ops
reltime
<
<=
=
>=
>
int2_minmax_ops
smallint
<
<=
=
>=
>
text_minmax_ops
text
<
<=
=
>=
>
tid_minmax_ops
tid
<
<=
=
>=
>
timestamp_minmax_ops
timestamp without time zone
<
<=
=
>=
>
timestamptz_minmax_ops
timestamp with time zone
<
<=
=
>=
>
time_minmax_ops
time without time zone
<
<=
=
>=
>
timetz_minmax_ops
time with time zone
<
<=
=
>=
>
uuid_minmax_ops
uuid
<
<=
=
>=
>
Extensibility
The BRIN interface has a high level of abstraction,
requiring the access method implementer only to implement the semantics
of the data type being accessed. The BRIN layer
itself takes care of concurrency, logging and searching the index structure.
All it takes to get a BRIN access method working is to
implement a few user-defined methods, which define the behavior of
summary values stored in the index and the way they interact with
scan keys.
In short, BRIN combines
extensibility with generality, code reuse, and a clean interface.
There are four methods that an operator class for BRIN
must provide:
BrinOpcInfo *opcInfo(Oid type_oid)>
Returns internal information about the indexed columns' summary data.
The return value must point to a palloc'd BrinOpcInfo>,
which has this definition:
typedef struct BrinOpcInfo
{
/* Number of columns stored in an index column of this opclass */
uint16 oi_nstored;
/* Opaque pointer for the opclass' private use */
void *oi_opaque;
/* Type cache entries of the stored columns */
TypeCacheEntry *oi_typcache[FLEXIBLE_ARRAY_MEMBER];
} BrinOpcInfo;
BrinOpcInfo>.oi_opaque> can be used by the
operator class routines to pass information between support procedures
during an index scan.
bool consistent(BrinDesc *bdesc, BrinValues *column,
ScanKey key)
Returns whether the ScanKey is consistent with the given indexed
values for a range.
The attribute number to use is passed as part of the scan key.
bool addValue(BrinDesc *bdesc, BrinValues *column,
Datum newval, bool isnull)
Given an index tuple and an indexed value, modifies the indicated
attribute of the tuple so that it additionally represents the new value.
If any modification was done to the tuple, true is
returned.
bool unionTuples(BrinDesc *bdesc, BrinValues *a,
BrinValues *b)
Consolidates two index tuples. Given two index tuples, modifies the
indicated attribute of the first of them so that it represents both tuples.
The second tuple is not modified.
The core distribution includes support for two types of operator classes:
minmax and inclusion. Operator class definitions using them are shipped for
in-core data types as appropriate. Additional operator classes can be
defined by the user for other data types using equivalent definitions,
without having to write any source code; appropriate catalog entries being
declared is enough. Note that assumptions about the semantics of operator
strategies are embedded in the support procedures' source code.
Operator classes that implement completely different semantics are also
possible, provided implementations of the four main support procedures
described above are written. Note that backwards compatibility across major
releases is not guaranteed: for example, additional support procedures might
be required in later releases.
To write an operator class for a data type that implements a totally
ordered set, it is possible to use the minmax support procedures
alongside the corresponding operators, as shown in
.
All operator class members (procedures and operators) are mandatory.
Procedure and Support Numbers for Minmax Operator Classes
Operator class member
Object
Support Procedure 1
internal function brin_minmax_opcinfo()
Support Procedure 2
internal function brin_minmax_add_value()
Support Procedure 3
internal function brin_minmax_consistent()
Support Procedure 4
internal function brin_minmax_union()
Operator Strategy 1
operator less-than
Operator Strategy 2
operator less-than-or-equal-to
Operator Strategy 3
operator equal-to
Operator Strategy 4
operator greater-than-or-equal-to
Operator Strategy 5
operator greater-than
To write an operator class for a complex datatype which has values
included within another type, it's possible to use the inclusion support
procedures alongside the corresponding operators, as shown
in . It requires
only a single additional function, which can be written in any language.
More functions can be defined for additional functionality. All operators
are optional. Some operators require other operators, as shown as
dependencies on the table.
Procedure and Support Numbers for Inclusion Operator Classes
Operator class member
Object
Dependency
Support Procedure 1
internal function brin_inclusion_opcinfo()
Support Procedure 2
internal function brin_inclusion_add_value()
Support Procedure 3
internal function brin_inclusion_consistent()
Support Procedure 4
internal function brin_inclusion_union()
Support Procedure 11
function to merge two elements
Support Procedure 12
optional function to check whether two elements are mergeable
Support Procedure 13
optional function to check if an element is contained within another
Support Procedure 14
optional function to check whether an element is empty
Operator Strategy 1
operator left-of
Operator Strategy 4
Operator Strategy 2
operator does-not-extend-to-the-right-of
Operator Strategy 5
Operator Strategy 3
operator overlaps
Operator Strategy 4
operator right-of
Operator Strategy 2
Operator Strategy 5
operator does-not-extend-to-the-right-of
Operator Strategy 1
Operator Strategy 6, 18
operator same-as-or-equal-to
Operator Strategy 7
Operator Strategy 7, 13, 16, 24, 25
operator contains-or-equal-to
Operator Strategy 8, 14, 26, 27
operator is-contained-by-or-equal-to
Operator Strategy 3
Operator Strategy 9
operator does-not-extend-above
Operator Strategy 11
Operator Strategy 10
operator is-below
Operator Strategy 12
Operator Strategy 11
operator is-above
Operator Strategy 9
Operator Strategy 12
operator does-not-extend-below
Operator Strategy 10
Operator Strategy 20
operator less-than
Operator Strategy 4
Operator Strategy 21
operator less-than-or-equal-to
Operator Strategy 4
Operator Strategy 22
operator greater-than
Operator Strategy 1
Operator Strategy 23
operator greater-than-or-equal-to
Operator Strategy 1
Support procedure numbers 1-10 are reserved for the BRIN internal
functions, so the SQL level functions start with number 11. Support
function number 11 is the main function required to build the index.
It should accept two arguments with the same datatype as the opclass,
and return the union of them. The inclusion opclass can store union
values with different datatypes if it is defined with the
STORAGE> parameter. The return value of the union
function should match the STORAGE> datatype.
Support procedure numbers 12 and 14 are provided to support
irregularities of built-in datatypes. Procedure number 12
is used to support network addresses from different families which
are not mergeable. Procedure number 14 is used to support
empty ranges. Procedure number 13 is an optional but
recommended one, which allows the new value to be checked before
it is passed to the union function. As the BRIN framework can shortcut
some operations when the union is not changed, using this
function can improve index performance.
Both minmax and inclusion opclasses support cross-datatype
operators, though with these the dependencies become more complicated.
The minmax opclass requires a full set of operators to be
defined with both arguments having the same datatype. It allows
additional datatypes to be supported by defining extra sets
of operators. Inclusion opclass operator strategies are dependent
on another operator strategy as shown in
, or the same
operator strategy as themselves. They require the dependency
operator to be defined with the STORAGE> datatype as the
left-hand-side argument and the other supported datatype to be the
right-hand-side argument of the supported operator. See
float4_minmax_ops> as an example of minmax, and
box_inclusion_ops> as an example of inclusion.