diff --git a/doc/src/sgml/ref/create_aggregate.sgml b/doc/src/sgml/ref/create_aggregate.sgml index 6dc1931997..6a8acfb4f9 100644 --- a/doc/src/sgml/ref/create_aggregate.sgml +++ b/doc/src/sgml/ref/create_aggregate.sgml @@ -50,9 +50,8 @@ CREATE AGGREGATE name ( [ [ ffunc ] [ , FINALFUNC_EXTRA ] [ , INITCOND = initial_condition ] - [ , HYPOTHETICAL ] [ , PARALLEL = { SAFE | RESTRICTED | UNSAFE } ] - + [ , HYPOTHETICAL ] ) or the old syntax @@ -221,6 +220,17 @@ CREATE AGGREGATE name ( aggregate-input rows as an additional hypothetical row. + + An aggregate can optionally support partial aggregation, + as described in . + This requires specifying the COMBINEFUNC parameter. + If the state_data_type + is internal, it's usually also appropriate to provide the + SERIALFUNC and DESERIALFUNC parameters so that + parallel aggregation is possible. Note that the aggregate must also be + marked PARALLEL SAFE to enable parallel aggregation. + + Aggregates that behave like MIN or MAX can sometimes be optimized by looking into an index instead of scanning every @@ -408,12 +418,7 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; The combinefunc function may optionally be specified to allow the aggregate function to support - partial aggregation. This is a prerequisite to allow the aggregate to - participate in certain optimizations such as parallel aggregation. - - - - If provided, + partial aggregation. If provided, the combinefunc must combine two state_data_type values, each containing the result of aggregation over some subset of @@ -422,20 +427,15 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; represents the result of aggregating over both sets of inputs. This function can be thought of as an sfunc, where instead of - acting upon individual input rows and adding these to the aggregate - state, it adds another aggregate state to the aggregate state. - Typically, it is not possible to define - a combinefunc for aggregate - functions that are sensitive to the order of the input values, since the - relative ordering of the inputs that went into the subset states is - indeterminate. + acting upon an individual input row and adding it to the running + aggregate state, it adds another aggregate state to the running state. - The combinefunc must accept - two arguments of + The combinefunc must be + declared as taking two arguments of the state_data_type and - return a value of + returning a value of the state_data_type. Optionally this function may be strict. In this case the function will not be called when either of the input states are null; @@ -446,11 +446,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; For aggregate functions whose state_data_type is internal, - the combinefunc must not be - strict. In this scenario - the combinefunc must ensure - that null states are handled correctly and that the state being returned - is properly stored in the aggregate memory context. + the combinefunc must not + be strict. In this case + the combinefunc must + ensure that null states are handled correctly and that the state being + returned is properly stored in the aggregate memory context. @@ -586,6 +586,22 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; + + PARALLEL + + + The meanings of PARALLEL SAFE, PARALLEL + RESTRICTED, and PARALLEL UNSAFE are the same as + for . An aggregate will not be + considered for parallelization if it is marked PARALLEL + UNSAFE (which is the default!) or PARALLEL RESTRICTED. + Note that the parallel-safety markings of the aggregate's support + functions are not consulted by the planner, only the marking of the + aggregate itself. + + + + HYPOTHETICAL @@ -686,10 +702,11 @@ SELECT col FROM tab ORDER BY col USING sortop LIMIT 1; - The meaning of PARALLEL SAFE, PARALLEL RESTRICTED, - and PARALLEL UNSAFE is the same as for - . - + Partial (including parallel) aggregation is currently not supported for + ordered-set aggregates. Also, it will never be used for aggregate calls + that include DISTINCT or ORDER BY clauses, since + those semantics cannot be supported during partial aggregation. + diff --git a/doc/src/sgml/xaggr.sgml b/doc/src/sgml/xaggr.sgml index ef7cff4879..fa98572ed6 100644 --- a/doc/src/sgml/xaggr.sgml +++ b/doc/src/sgml/xaggr.sgml @@ -23,7 +23,7 @@ A final function can also be specified, in case the desired result of the aggregate is different from the data that needs to be kept in the running - state value. The final function takes the last state value + state value. The final function takes the ending state value and returns whatever is wanted as the aggregate result. In principle, the transition and final functions are just ordinary functions that could also be used outside the context of the @@ -509,6 +509,102 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households; and therefore there is no need for them to support moving-aggregate mode. + + + + Partial Aggregation + + + aggregate function + partial aggregation + + + + Optionally, an aggregate function can support partial + aggregation. The idea of partial aggregation is to run the aggregate's + state transition function over different subsets of the input data + independently, and then to combine the state values resulting from those + subsets to produce the same state value that would have resulted from + scanning all the input in a single operation. This mode can be used for + parallel aggregation by having different worker processes scan different + portions of a table. Each worker produces a partial state value, and at + the end those state values are combined to produce a final state value. + (In the future this mode might also be used for purposes such as combining + aggregations over local and remote tables; but that is not implemented + yet.) + + + + To support partial aggregation, the aggregate definition must provide + a combine function, which takes two values of the + aggregate's state type (representing the results of aggregating over two + subsets of the input rows) and produces a new value of the state type, + representing what the state would have been after aggregating over the + combination of those sets of rows. It is unspecified what the relative + order of the input rows from the two sets would have been. This means + that it's usually impossible to define a useful combine function for + aggregates that are sensitive to input row order. + + + + As simple examples, MAX and MIN aggregates can be + made to support partial aggregation by specifying the combine function as + the same greater-of-two or lesser-of-two comparison function that is used + as their transition function. SUM aggregates just need an + addition function as combine function. (Again, this is the same as their + transition function, unless the state value is wider than the input data + type.) + + + + The combine function is treated much like a transition function that + happens to take a value of the state type, not of the underlying input + type, as its second argument. In particular, the rules for dealing + with null values and strict functions are similar. Also, if the aggregate + definition specifies a non-null initcond, keep in mind that + that will be used not only as the initial state for each partial + aggregation run, but also as the initial state for the combine function, + which will be called to combine each partial result into that state. + + + + If the aggregate's state type is declared as internal, it is + the combine function's responsibility that its result is allocated in + the correct memory context for aggregate state values. This means in + particular that when the first input is NULL it's invalid + to simply return the second input, as that value will be in the wrong + context and will not have sufficient lifespan. + + + + When the aggregate's state type is declared as internal, it is + usually also appropriate for the aggregate definition to provide a + serialization function and a deserialization + function, which allow such a state value to be copied from one process + to another. Without these functions, parallel aggregation cannot be + performed, and future applications such as local/remote aggregation will + probably not work either. + + + + A serialization function must take a single argument of + type internal and return a result of type bytea, which + represents the state value packaged up into a flat blob of bytes. + Conversely, a deserialization function reverses that conversion. It must + take two arguments of types bytea and internal, and + return a result of type internal. (The second argument is unused + and is always zero, but it is required for type-safety reasons.) The + result of the deserialization function should simply be allocated in the + current memory context, as unlike the combine function's result, it is not + long-lived. + + + + Worth noting also is that for an aggregate to be executed in parallel, + the aggregate itself must be marked PARALLEL SAFE. The + parallel-safety markings on its support functions are not consulted. + + @@ -521,7 +617,7 @@ SELECT percentile_disc(0.5) WITHIN GROUP (ORDER BY income) FROM households; A function written in C can detect that it is being called as an - aggregate transition or final function by calling + aggregate support function by calling AggCheckCallContext, for example: if (AggCheckCallContext(fcinfo, NULL))