rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Improve user-facing documentation for partial/parallel aggregation. 

Add a section to xaggr.sgml, as we have done in the past for other
extensions to the aggregation functionality.  Assorted wordsmithing
and other minor improvements.

David Rowley and Tom Lane
This commit is contained in:
Tom Lane 2016-06-22 19:14:16 -04:00
parent 63ae052367
commit 2d673424fa
2 changed files with 142 additions and 29 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

71
doc/src/sgml/ref/create_aggregate.sgml
View File

@ -50,9 +50,8 @@ CREATE AGGREGATE <replaceable class="parameter">name</replaceable> ( [ [ <replac
[ , FINALFUNC = <replaceable class="PARAMETER">ffunc</replaceable> ]
[ , FINALFUNC_EXTRA ]
[ , INITCOND = <replaceable class="PARAMETER">initial_condition</replaceable> ]
[ , HYPOTHETICAL ]
[ , PARALLEL = { SAFE | RESTRICTED | UNSAFE } ]
[ , HYPOTHETICAL ]
)
<phrase>or the old syntax</phrase>
@ -221,6 +220,17 @@ CREATE AGGREGATE <replaceable class="PARAMETER">name</replaceable> (
aggregate-input rows as an additional <quote>hypothetical</> row.
</para>
<para>
An aggregate can optionally support <firstterm>partial aggregation</>,
as described in <xref linkend="xaggr-partial-aggregates">.
This requires specifying the <literal>COMBINEFUNC</> parameter.
If the <replaceable class="PARAMETER">state_data_type</replaceable>
is <type>internal</>, it's usually also appropriate to provide the
<literal>SERIALFUNC</> and <literal>DESERIALFUNC</> parameters so that
parallel aggregation is possible. Note that the aggregate must also be
marked <literal>PARALLEL SAFE</> to enable parallel aggregation.
</para>
<para>
Aggregates that behave like <function>MIN</> or <function>MAX</> can
sometimes be optimized by looking into an index instead of scanning every
@ -408,12 +418,7 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
<para>
The <replaceable class="PARAMETER">combinefunc</replaceable> function
may optionally be specified to allow the aggregate function to support
partial aggregation. This is a prerequisite to allow the aggregate to
participate in certain optimizations such as parallel aggregation.
</para>
<para>
If provided,
partial aggregation. If provided,
the <replaceable class="PARAMETER">combinefunc</replaceable> must
combine two <replaceable class="PARAMETER">state_data_type</replaceable>
values, each containing the result of aggregation over some subset of
@ -422,20 +427,15 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
represents the result of aggregating over both sets of inputs. This
function can be thought of as
an <replaceable class="PARAMETER">sfunc</replaceable>, where instead of
acting upon individual input rows and adding these to the aggregate
state, it adds another aggregate state to the aggregate state.
Typically, it is not possible to define
a <replaceable class="PARAMETER">combinefunc</replaceable> for aggregate
functions that are sensitive to the order of the input values, since the
relative ordering of the inputs that went into the subset states is
indeterminate.
acting upon an individual input row and adding it to the running
aggregate state, it adds another aggregate state to the running state.
</para>
<para>
The <replaceable class="PARAMETER">combinefunc</replaceable> must accept
two arguments of
The <replaceable class="PARAMETER">combinefunc</replaceable> must be
declared as taking two arguments of
the <replaceable class="PARAMETER">state_data_type</replaceable> and
return a value of
returning a value of
the <replaceable class="PARAMETER">state_data_type</replaceable>.
Optionally this function may be <quote>strict</quote>. In this case the
function will not be called when either of the input states are null;
@ -446,11 +446,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
For aggregate functions
whose <replaceable class="PARAMETER">state_data_type</replaceable>
is <type>internal</type>,
the <replaceable class="PARAMETER">combinefunc</replaceable> must not be
strict. In this scenario
the <replaceable class="PARAMETER">combinefunc</replaceable> must ensure
that null states are handled correctly and that the state being returned
is properly stored in the aggregate memory context.
the <replaceable class="PARAMETER">combinefunc</replaceable> must not
be strict. In this case
the <replaceable class="PARAMETER">combinefunc</replaceable> must
ensure that null states are handled correctly and that the state being
returned is properly stored in the aggregate memory context.
</para>
</listitem>
</varlistentry>
@ -586,6 +586,22 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
</listitem>
</varlistentry>
<varlistentry>
<term><literal>PARALLEL</literal></term>
<listitem>
<para>
The meanings of <literal>PARALLEL SAFE</>, <literal>PARALLEL
RESTRICTED</>, and <literal>PARALLEL UNSAFE</> are the same as
for <xref linkend="sql-createfunction">. An aggregate will not be
considered for parallelization if it is marked <literal>PARALLEL
UNSAFE</> (which is the default!) or <literal>PARALLEL RESTRICTED</>.
Note that the parallel-safety markings of the aggregate's support
functions are not consulted by the planner, only the marking of the
aggregate itself.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term><literal>HYPOTHETICAL</literal></term>
<listitem>
@ -686,10 +702,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1;
</para>
<para>
The meaning of <literal>PARALLEL SAFE</>, <literal>PARALLEL RESTRICTED</>,
and <literal>PARALLEL UNSAFE</> is the same as for
<xref linkend="sql-createfunction">.
</para>
Partial (including parallel) aggregation is currently not supported for
ordered-set aggregates. Also, it will never be used for aggregate calls
that include <literal>DISTINCT</> or <literal>ORDER BY</> clauses, since
those semantics cannot be supported during partial aggregation.
</para>
</refsect1>
<refsect1>

100
doc/src/sgml/xaggr.sgml
View File

@ -23,7 +23,7 @@
A <firstterm>final function</firstterm>
can also be specified, in case the desired result of the aggregate
is different from the data that needs to be kept in the running
state value. The final function takes the last state value
state value. The final function takes the ending state value
and returns whatever is wanted as the aggregate result.
In principle, the transition and final functions are just ordinary
functions that could also be used outside the context of the
@ -509,6 +509,102 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households;
and therefore there is no need for them to support moving-aggregate mode.
</para>
</sect2>
<sect2 id="xaggr-partial-aggregates">
<title>Partial Aggregation</title>
<indexterm>
<primary>aggregate function</primary>
<secondary>partial aggregation</secondary>
</indexterm>
<para>
Optionally, an aggregate function can support <firstterm>partial
aggregation</>. The idea of partial aggregation is to run the aggregate's
state transition function over different subsets of the input data
independently, and then to combine the state values resulting from those
subsets to produce the same state value that would have resulted from
scanning all the input in a single operation. This mode can be used for
parallel aggregation by having different worker processes scan different
portions of a table. Each worker produces a partial state value, and at
the end those state values are combined to produce a final state value.
(In the future this mode might also be used for purposes such as combining
aggregations over local and remote tables; but that is not implemented
yet.)
</para>
<para>
To support partial aggregation, the aggregate definition must provide
a <firstterm>combine function</>, which takes two values of the
aggregate's state type (representing the results of aggregating over two
subsets of the input rows) and produces a new value of the state type,
representing what the state would have been after aggregating over the
combination of those sets of rows. It is unspecified what the relative
order of the input rows from the two sets would have been. This means
that it's usually impossible to define a useful combine function for
aggregates that are sensitive to input row order.
</para>
<para>
As simple examples, <literal>MAX</> and <literal>MIN</> aggregates can be
made to support partial aggregation by specifying the combine function as
the same greater-of-two or lesser-of-two comparison function that is used
as their transition function. <literal>SUM</> aggregates just need an
addition function as combine function. (Again, this is the same as their
transition function, unless the state value is wider than the input data
type.)
</para>
<para>
The combine function is treated much like a transition function that
happens to take a value of the state type, not of the underlying input
type, as its second argument. In particular, the rules for dealing
with null values and strict functions are similar. Also, if the aggregate
definition specifies a non-null <literal>initcond</>, keep in mind that
that will be used not only as the initial state for each partial
aggregation run, but also as the initial state for the combine function,
which will be called to combine each partial result into that state.
</para>
<para>
If the aggregate's state type is declared as <type>internal</>, it is
the combine function's responsibility that its result is allocated in
the correct memory context for aggregate state values. This means in
particular that when the first input is <literal>NULL</> it's invalid
to simply return the second input, as that value will be in the wrong
context and will not have sufficient lifespan.
</para>
<para>
When the aggregate's state type is declared as <type>internal</>, it is
usually also appropriate for the aggregate definition to provide a
<firstterm>serialization function</> and a <firstterm>deserialization
function</>, which allow such a state value to be copied from one process
to another. Without these functions, parallel aggregation cannot be
performed, and future applications such as local/remote aggregation will
probably not work either.
</para>
<para>
A serialization function must take a single argument of
type <type>internal</> and return a result of type <type>bytea</>, which
represents the state value packaged up into a flat blob of bytes.
Conversely, a deserialization function reverses that conversion. It must
take two arguments of types <type>bytea</> and <type>internal</>, and
return a result of type <type>internal</>. (The second argument is unused
and is always zero, but it is required for type-safety reasons.) The
result of the deserialization function should simply be allocated in the
current memory context, as unlike the combine function's result, it is not
long-lived.
</para>
<para>
Worth noting also is that for an aggregate to be executed in parallel,
the aggregate itself must be marked <literal>PARALLEL SAFE</>. The
parallel-safety markings on its support functions are not consulted.
</para>
</sect2>
<sect2 id="xaggr-support-functions">
@ -521,7 +617,7 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households;
<para>
A function written in C can detect that it is being called as an
aggregate transition or final function by calling
aggregate support function by calling
<function>AggCheckCallContext</>, for example:
<programlisting>
if (AggCheckCallContext(fcinfo, NULL))