1996-08-28 03:59:28 +02:00
|
|
|
/*-------------------------------------------------------------------------
|
|
|
|
|
*
|
1999-02-14 00:22:53 +01:00
|
|
|
* array.h
|
2005-11-17 23:14:56 +01:00
|
|
|
* Declarations for Postgres arrays.
|
|
|
|
|
*
|
|
|
|
|
* A standard varlena array has the following internal structure:
|
2007-02-28 00:48:10 +01:00
|
|
|
* <vl_len_> - standard varlena header word
|
2005-11-17 23:14:56 +01:00
|
|
|
* <ndim> - number of dimensions of the array
|
|
|
|
|
* <dataoffset> - offset to stored data, or 0 if no nulls bitmap
|
|
|
|
|
* <elemtype> - element type OID
|
|
|
|
|
* <dimensions> - length of each array axis (C array of int)
|
|
|
|
|
* <lower bnds> - lower boundary of each dimension (C array of int)
|
|
|
|
|
* <null bitmap> - bitmap showing locations of nulls (OPTIONAL)
|
|
|
|
|
* <actual data> - whatever is the stored data
|
|
|
|
|
*
|
|
|
|
|
* The <dimensions> and <lower bnds> arrays each have ndim elements.
|
|
|
|
|
*
|
|
|
|
|
* The <null bitmap> may be omitted if the array contains no NULL elements.
|
|
|
|
|
* If it is absent, the <dataoffset> field is zero and the offset to the
|
|
|
|
|
* stored data must be computed on-the-fly. If the bitmap is present,
|
|
|
|
|
* <dataoffset> is nonzero and is equal to the offset from the array start
|
|
|
|
|
* to the first data element (including any alignment padding). The bitmap
|
|
|
|
|
* follows the same conventions as tuple null bitmaps, ie, a 1 indicates
|
|
|
|
|
* a non-null entry and the LSB of each bitmap byte is used first.
|
|
|
|
|
*
|
|
|
|
|
* The actual data starts on a MAXALIGN boundary. Individual items in the
|
|
|
|
|
* array are aligned as specified by the array element type. They are
|
|
|
|
|
* stored in row-major order (last subscript varies most rapidly).
|
|
|
|
|
*
|
|
|
|
|
* NOTE: it is important that array elements of toastable datatypes NOT be
|
|
|
|
|
* toasted, since the tupletoaster won't know they are there. (We could
|
|
|
|
|
* support compressed toasted items; only out-of-line items are dangerous.
|
|
|
|
|
* However, it seems preferable to store such items uncompressed and allow
|
|
|
|
|
* the toaster to compress the whole array as one input.)
|
|
|
|
|
*
|
|
|
|
|
*
|
|
|
|
|
* The OIDVECTOR and INT2VECTOR datatypes are storage-compatible with
|
|
|
|
|
* generic arrays, but they support only one-dimensional arrays with no
|
2016-08-18 20:48:51 +02:00
|
|
|
* nulls (and no null bitmap). They don't support being toasted, either.
|
2005-11-17 23:14:56 +01:00
|
|
|
*
|
|
|
|
|
* There are also some "fixed-length array" datatypes, such as NAME and
|
|
|
|
|
* POINT. These are simply a sequence of a fixed number of items each
|
|
|
|
|
* of a fixed-length datatype, with no overhead; the item size must be
|
|
|
|
|
* a multiple of its alignment requirement, because we do no padding.
|
|
|
|
|
* We support subscripting on these types, but array_in() and array_out()
|
|
|
|
|
* only work with varlena arrays.
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
* In addition, arrays are a major user of the "expanded object" TOAST
|
|
|
|
|
* infrastructure. This allows a varlena array to be converted to a
|
|
|
|
|
* separate representation that may include "deconstructed" Datum/isnull
|
|
|
|
|
* arrays holding the elements.
|
|
|
|
|
*
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
2023-01-02 21:00:37 +01:00
|
|
|
* Portions Copyright (c) 1996-2023, PostgreSQL Global Development Group
|
2000-01-26 06:58:53 +01:00
|
|
|
* Portions Copyright (c) 1994, Regents of the University of California
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
2010-09-20 22:08:53 +02:00
|
|
|
* src/include/utils/array.h
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
|
|
|
|
*-------------------------------------------------------------------------
|
|
|
|
|
*/
|
|
|
|
|
#ifndef ARRAY_H
|
|
|
|
|
#define ARRAY_H
|
|
|
|
|
|
2000-05-29 23:02:32 +02:00
|
|
|
#include "fmgr.h"
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
#include "utils/expandeddatum.h"
|
|
|
|
|
|
Support arrays over domains.
Allowing arrays with a domain type as their element type was left un-done
in the original domain patch, but not for any very good reason. This
omission leads to such surprising results as array_agg() not working on
a domain column, because the parser can't identify a suitable output type
for the polymorphic aggregate.
In order to fix this, first clean up the APIs of coerce_to_domain() and
some internal functions in parse_coerce.c so that we consistently pass
around a CoercionContext along with CoercionForm. Previously, we sometimes
passed an "isExplicit" boolean flag instead, which is strictly less
information; and coerce_to_domain() didn't even get that, but instead had
to reverse-engineer isExplicit from CoercionForm. That's contrary to the
documentation in primnodes.h that says that CoercionForm only affects
display and not semantics. I don't think this change fixes any live bugs,
but it makes things more consistent. The main reason for doing it though
is that now build_coercion_expression() receives ccontext, which it needs
in order to be able to recursively invoke coerce_to_target_type().
Next, reimplement ArrayCoerceExpr so that the node does not directly know
any details of what has to be done to the individual array elements while
performing the array coercion. Instead, the per-element processing is
represented by a sub-expression whose input is a source array element and
whose output is a target array element. This simplifies life in
parse_coerce.c, because it can build that sub-expression by a recursive
invocation of coerce_to_target_type(). The executor now handles the
per-element processing as a compiled expression instead of hard-wired code.
The main advantage of this is that we can use a single ArrayCoerceExpr to
handle as many as three successive steps per element: base type conversion,
typmod coercion, and domain constraint checking. The old code used two
stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty
inefficient, and adding yet another array deconstruction to do domain
constraint checking seemed very unappetizing.
In the case where we just need a single, very simple coercion function,
doing this straightforwardly leads to a noticeable increase in the
per-array-element runtime cost. Hence, add an additional shortcut evalfunc
in execExprInterp.c that skips unnecessary overhead for that specific form
of expression. The runtime speed of simple cases is within 1% or so of
where it was before, while cases that previously required two levels of
array processing are significantly faster.
Finally, create an implicit array type for every domain type, as we do for
base types, enums, etc. Everything except the array-coercion case seems
to just work without further effort.
Tom Lane, reviewed by Andrew Dunstan
Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
|
|
|
/* avoid including execnodes.h here */
|
|
|
|
|
struct ExprState;
|
|
|
|
|
struct ExprContext;
|
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
|
Support subscripting of arbitrary types, not only arrays.
This patch generalizes the subscripting infrastructure so that any
data type can be subscripted, if it provides a handler function to
define what that means. Traditional variable-length (varlena) arrays
all use array_subscript_handler(), while the existing fixed-length
types that support subscripting use raw_array_subscript_handler().
It's expected that other types that want to use subscripting notation
will define their own handlers. (This patch provides no such new
features, though; it only lays the foundation for them.)
To do this, move the parser's semantic processing of subscripts
(including coercion to whatever data type is required) into a
method callback supplied by the handler. On the execution side,
replace the ExecEvalSubscriptingRef* layer of functions with direct
calls to callback-supplied execution routines. (Thus, essentially
no new run-time overhead should be caused by this patch. Indeed,
there is room to remove some overhead by supplying specialized
execution routines. This patch does a little bit in that line,
but more could be done.)
Additional work is required here and there to remove formerly
hard-wired assumptions about the result type, collation, etc
of a SubscriptingRef expression node; and to remove assumptions
that the subscript values must be integers.
One useful side-effect of this is that we now have a less squishy
mechanism for identifying whether a data type is a "true" array:
instead of wiring in weird rules about typlen, we can look to see
if pg_type.typsubscript == F_ARRAY_SUBSCRIPT_HANDLER. For this
to be bulletproof, we have to forbid user-defined types from using
that handler directly; but there seems no good reason for them to
do so.
This patch also removes assumptions that the number of subscripts
is limited to MAXDIM (6), or indeed has any hard-wired limit.
That limit still applies to types handled by array_subscript_handler
or raw_array_subscript_handler, but to discourage other dependencies
on this constant, I've moved it from c.h to utils/array.h.
Dmitry Dolgov, reviewed at various times by Tom Lane, Arthur Zakirov,
Peter Eisentraut, Pavel Stehule
Discussion: https://postgr.es/m/CA+q6zcVDuGBv=M0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w@mail.gmail.com
Discussion: https://postgr.es/m/CA+q6zcVovR+XY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA@mail.gmail.com
2020-12-09 18:40:37 +01:00
|
|
|
/*
|
|
|
|
|
* Maximum number of array subscripts (arbitrary limit)
|
|
|
|
|
*/
|
|
|
|
|
#define MAXDIM 6
|
|
|
|
|
|
2000-07-17 05:05:41 +02:00
|
|
|
/*
|
|
|
|
|
* Arrays are varlena objects, so must meet the varlena convention that
|
|
|
|
|
* the first int32 of the object contains the total object size in bytes.
|
2007-02-28 00:48:10 +01:00
|
|
|
* Be sure to use VARSIZE() and SET_VARSIZE() to access it, though!
|
2005-03-29 02:17:27 +02:00
|
|
|
*
|
|
|
|
|
* CAUTION: if you change the header for ordinary arrays you will also
|
|
|
|
|
* need to change the headers for oidvector and int2vector!
|
2000-07-17 05:05:41 +02:00
|
|
|
*/
|
2019-08-16 19:33:30 +02:00
|
|
|
typedef struct ArrayType
|
1996-08-28 03:59:28 +02:00
|
|
|
{
|
2007-02-28 00:48:10 +01:00
|
|
|
int32 vl_len_; /* varlena header (do not touch directly!) */
|
1996-08-28 03:59:28 +02:00
|
|
|
int ndim; /* # of dimensions */
|
2005-11-17 23:14:56 +01:00
|
|
|
int32 dataoffset; /* offset to data, or 0 if no bitmap */
|
2002-08-26 19:54:02 +02:00
|
|
|
Oid elemtype; /* element type OID */
|
1996-08-28 03:59:28 +02:00
|
|
|
} ArrayType;
|
|
|
|
|
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
/*
|
|
|
|
|
* An expanded array is contained within a private memory context (as
|
|
|
|
|
* all expanded objects must be) and has a control structure as below.
|
|
|
|
|
*
|
|
|
|
|
* The expanded array might contain a regular "flat" array if that was the
|
|
|
|
|
* original input and we've not modified it significantly. Otherwise, the
|
|
|
|
|
* contents are represented by Datum/isnull arrays plus dimensionality and
|
|
|
|
|
* type information. We could also have both forms, if we've deconstructed
|
|
|
|
|
* the original array for access purposes but not yet changed it. For pass-
|
|
|
|
|
* by-reference element types, the Datums would point into the flat array in
|
|
|
|
|
* this situation. Once we start modifying array elements, new pass-by-ref
|
|
|
|
|
* elements are separately palloc'd within the memory context.
|
|
|
|
|
*/
|
|
|
|
|
#define EA_MAGIC 689375833 /* ID for debugging crosschecks */
|
|
|
|
|
|
|
|
|
|
typedef struct ExpandedArrayHeader
|
|
|
|
|
{
|
|
|
|
|
/* Standard header for expanded objects */
|
|
|
|
|
ExpandedObjectHeader hdr;
|
|
|
|
|
|
|
|
|
|
/* Magic value identifying an expanded array (for debugging only) */
|
|
|
|
|
int ea_magic;
|
|
|
|
|
|
|
|
|
|
/* Dimensionality info (always valid) */
|
|
|
|
|
int ndims; /* # of dimensions */
|
|
|
|
|
int *dims; /* array dimensions */
|
|
|
|
|
int *lbound; /* index lower bounds for each dimension */
|
|
|
|
|
|
|
|
|
|
/* Element type info (always valid) */
|
|
|
|
|
Oid element_type; /* element type OID */
|
|
|
|
|
int16 typlen; /* needed info about element datatype */
|
|
|
|
|
bool typbyval;
|
|
|
|
|
char typalign;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* If we have a Datum-array representation of the array, it's kept here;
|
|
|
|
|
* else dvalues/dnulls are NULL. The dvalues and dnulls arrays are always
|
|
|
|
|
* palloc'd within the object private context, but may change size from
|
|
|
|
|
* time to time. For pass-by-ref element types, dvalues entries might
|
|
|
|
|
* point either into the fstartptr..fendptr area, or to separately
|
|
|
|
|
* palloc'd chunks. Elements should always be fully detoasted, as they
|
|
|
|
|
* are in the standard flat representation.
|
|
|
|
|
*
|
|
|
|
|
* Even when dvalues is valid, dnulls can be NULL if there are no null
|
|
|
|
|
* elements.
|
|
|
|
|
*/
|
|
|
|
|
Datum *dvalues; /* array of Datums */
|
|
|
|
|
bool *dnulls; /* array of is-null flags for Datums */
|
|
|
|
|
int dvalueslen; /* allocated length of above arrays */
|
|
|
|
|
int nelems; /* number of valid entries in above arrays */
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* flat_size is the current space requirement for the flat equivalent of
|
|
|
|
|
* the expanded array, if known; otherwise it's 0. We store this to make
|
|
|
|
|
* consecutive calls of get_flat_size cheap.
|
|
|
|
|
*/
|
|
|
|
|
Size flat_size;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* fvalue points to the flat representation if it is valid, else it is
|
|
|
|
|
* NULL. If we have or ever had a flat representation then
|
|
|
|
|
* fstartptr/fendptr point to the start and end+1 of its data area; this
|
|
|
|
|
* is so that we can tell which Datum pointers point into the flat
|
|
|
|
|
* representation rather than being pointers to separately palloc'd data.
|
|
|
|
|
*/
|
|
|
|
|
ArrayType *fvalue; /* must be a fully detoasted array */
|
|
|
|
|
char *fstartptr; /* start of its data area */
|
|
|
|
|
char *fendptr; /* end+1 of its data area */
|
|
|
|
|
} ExpandedArrayHeader;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* Functions that can handle either a "flat" varlena array or an expanded
|
2019-07-01 02:34:17 +02:00
|
|
|
* array use this union to work with their input. Don't refer to "flt";
|
|
|
|
|
* instead, cast to ArrayType. This struct nominally requires 8-byte
|
|
|
|
|
* alignment on 64-bit, but it's often used for an ArrayType having 4-byte
|
|
|
|
|
* alignment. UBSan complains about referencing "flt" in such cases.
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
*/
|
|
|
|
|
typedef union AnyArrayType
|
|
|
|
|
{
|
|
|
|
|
ArrayType flt;
|
|
|
|
|
ExpandedArrayHeader xpn;
|
|
|
|
|
} AnyArrayType;
|
|
|
|
|
|
2007-02-28 00:48:10 +01:00
|
|
|
/*
|
|
|
|
|
* working state for accumArrayResult() and friends
|
2014-11-25 18:21:22 +01:00
|
|
|
* note that the input must be scalars (legal array elements)
|
2007-02-28 00:48:10 +01:00
|
|
|
*/
|
2003-06-27 02:33:26 +02:00
|
|
|
typedef struct ArrayBuildState
|
|
|
|
|
{
|
|
|
|
|
MemoryContext mcontext; /* where all the temp stuff is kept */
|
|
|
|
|
Datum *dvalues; /* array of accumulated Datums */
|
2005-11-17 23:14:56 +01:00
|
|
|
bool *dnulls; /* array of is-null flags for Datums */
|
2006-11-08 20:24:38 +01:00
|
|
|
int alen; /* allocated length of above arrays */
|
|
|
|
|
int nelems; /* number of valid entries in above arrays */
|
2003-06-27 02:33:26 +02:00
|
|
|
Oid element_type; /* data type of the Datums */
|
|
|
|
|
int16 typlen; /* needed info about datatype */
|
|
|
|
|
bool typbyval;
|
|
|
|
|
char typalign;
|
2015-02-22 02:24:48 +01:00
|
|
|
bool private_cxt; /* use private memory context */
|
2003-06-27 02:33:26 +02:00
|
|
|
} ArrayBuildState;
|
|
|
|
|
|
2014-11-25 18:21:22 +01:00
|
|
|
/*
|
|
|
|
|
* working state for accumArrayResultArr() and friends
|
|
|
|
|
* note that the input must be arrays, and the same array type is returned
|
|
|
|
|
*/
|
|
|
|
|
typedef struct ArrayBuildStateArr
|
|
|
|
|
{
|
|
|
|
|
MemoryContext mcontext; /* where all the temp stuff is kept */
|
|
|
|
|
char *data; /* accumulated data */
|
|
|
|
|
bits8 *nullbitmap; /* bitmap of is-null flags, or NULL if none */
|
|
|
|
|
int abytes; /* allocated length of "data" */
|
|
|
|
|
int nbytes; /* number of bytes used so far */
|
|
|
|
|
int aitems; /* allocated length of bitmap (in elements) */
|
|
|
|
|
int nitems; /* total number of elements in result */
|
|
|
|
|
int ndims; /* current dimensions of result */
|
|
|
|
|
int dims[MAXDIM];
|
|
|
|
|
int lbs[MAXDIM];
|
|
|
|
|
Oid array_type; /* data type of the arrays */
|
|
|
|
|
Oid element_type; /* data type of the array elements */
|
2015-02-22 02:24:48 +01:00
|
|
|
bool private_cxt; /* use private memory context */
|
2014-11-25 18:21:22 +01:00
|
|
|
} ArrayBuildStateArr;
|
|
|
|
|
|
|
|
|
|
/*
|
|
|
|
|
* working state for accumArrayResultAny() and friends
|
|
|
|
|
* these functions handle both cases
|
|
|
|
|
*/
|
|
|
|
|
typedef struct ArrayBuildStateAny
|
|
|
|
|
{
|
|
|
|
|
/* Exactly one of these is not NULL: */
|
|
|
|
|
ArrayBuildState *scalarstate;
|
|
|
|
|
ArrayBuildStateArr *arraystate;
|
|
|
|
|
} ArrayBuildStateAny;
|
|
|
|
|
|
2003-06-27 02:33:26 +02:00
|
|
|
/*
|
|
|
|
|
* structure to cache type metadata needed for array manipulation
|
|
|
|
|
*/
|
|
|
|
|
typedef struct ArrayMetaState
|
|
|
|
|
{
|
|
|
|
|
Oid element_type;
|
|
|
|
|
int16 typlen;
|
|
|
|
|
bool typbyval;
|
|
|
|
|
char typalign;
|
|
|
|
|
char typdelim;
|
2004-06-06 02:41:28 +02:00
|
|
|
Oid typioparam;
|
2003-06-27 02:33:26 +02:00
|
|
|
Oid typiofunc;
|
|
|
|
|
FmgrInfo proc;
|
|
|
|
|
} ArrayMetaState;
|
|
|
|
|
|
2005-03-24 22:50:38 +01:00
|
|
|
/*
|
|
|
|
|
* private state needed by array_map (here because caller must provide it)
|
|
|
|
|
*/
|
|
|
|
|
typedef struct ArrayMapState
|
|
|
|
|
{
|
|
|
|
|
ArrayMetaState inp_extra;
|
|
|
|
|
ArrayMetaState ret_extra;
|
|
|
|
|
} ArrayMapState;
|
|
|
|
|
|
2011-02-16 07:52:04 +01:00
|
|
|
/* ArrayIteratorData is private in arrayfuncs.c */
|
|
|
|
|
typedef struct ArrayIteratorData *ArrayIterator;
|
|
|
|
|
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
/* fmgr macros for regular varlena array objects */
|
2000-07-17 05:05:41 +02:00
|
|
|
#define DatumGetArrayTypeP(X) ((ArrayType *) PG_DETOAST_DATUM(X))
|
|
|
|
|
#define DatumGetArrayTypePCopy(X) ((ArrayType *) PG_DETOAST_DATUM_COPY(X))
|
|
|
|
|
#define PG_GETARG_ARRAYTYPE_P(n) DatumGetArrayTypeP(PG_GETARG_DATUM(n))
|
|
|
|
|
#define PG_GETARG_ARRAYTYPE_P_COPY(n) DatumGetArrayTypePCopy(PG_GETARG_DATUM(n))
|
|
|
|
|
#define PG_RETURN_ARRAYTYPE_P(x) PG_RETURN_POINTER(x)
|
|
|
|
|
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
/* fmgr macros for expanded array objects */
|
|
|
|
|
#define PG_GETARG_EXPANDED_ARRAY(n) DatumGetExpandedArray(PG_GETARG_DATUM(n))
|
|
|
|
|
#define PG_GETARG_EXPANDED_ARRAYX(n, metacache) \
|
|
|
|
|
DatumGetExpandedArrayX(PG_GETARG_DATUM(n), metacache)
|
|
|
|
|
#define PG_RETURN_EXPANDED_ARRAY(x) PG_RETURN_DATUM(EOHPGetRWDatum(&(x)->hdr))
|
|
|
|
|
|
|
|
|
|
/* fmgr macros for AnyArrayType (ie, get either varlena or expanded form) */
|
2017-09-18 21:21:23 +02:00
|
|
|
#define PG_GETARG_ANY_ARRAY_P(n) DatumGetAnyArrayP(PG_GETARG_DATUM(n))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/*
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
* Access macros for varlena array header fields.
|
2000-07-22 05:34:43 +02:00
|
|
|
*
|
1996-08-28 03:59:28 +02:00
|
|
|
* ARR_DIMS returns a pointer to an array of array dimensions (number of
|
|
|
|
|
* elements along the various array axes).
|
|
|
|
|
*
|
|
|
|
|
* ARR_LBOUND returns a pointer to an array of array lower bounds.
|
|
|
|
|
*
|
2004-09-16 05:15:54 +02:00
|
|
|
* That is: if the third axis of an array has elements 5 through 8, then
|
|
|
|
|
* ARR_DIMS(a)[2] == 4 and ARR_LBOUND(a)[2] == 5.
|
1996-08-28 03:59:28 +02:00
|
|
|
*
|
|
|
|
|
* Unlike C, the default lower bound is 1.
|
|
|
|
|
*/
|
2007-02-28 00:48:10 +01:00
|
|
|
#define ARR_SIZE(a) VARSIZE(a)
|
2005-11-17 23:14:56 +01:00
|
|
|
#define ARR_NDIM(a) ((a)->ndim)
|
|
|
|
|
#define ARR_HASNULL(a) ((a)->dataoffset != 0)
|
|
|
|
|
#define ARR_ELEMTYPE(a) ((a)->elemtype)
|
2000-07-22 05:34:43 +02:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
#define ARR_DIMS(a) \
|
2000-07-22 05:34:43 +02:00
|
|
|
((int *) (((char *) (a)) + sizeof(ArrayType)))
|
1996-08-28 03:59:28 +02:00
|
|
|
#define ARR_LBOUND(a) \
|
2000-07-22 05:34:43 +02:00
|
|
|
((int *) (((char *) (a)) + sizeof(ArrayType) + \
|
2005-11-17 23:14:56 +01:00
|
|
|
sizeof(int) * ARR_NDIM(a)))
|
|
|
|
|
|
|
|
|
|
#define ARR_NULLBITMAP(a) \
|
|
|
|
|
(ARR_HASNULL(a) ? \
|
|
|
|
|
(bits8 *) (((char *) (a)) + sizeof(ArrayType) + \
|
|
|
|
|
2 * sizeof(int) * ARR_NDIM(a)) \
|
|
|
|
|
: (bits8 *) NULL)
|
1996-08-28 03:59:28 +02:00
|
|
|
|
|
|
|
|
/*
|
2005-11-17 23:14:56 +01:00
|
|
|
* The total array header size (in bytes) for an array with the specified
|
|
|
|
|
* number of dimensions and total number of items.
|
1996-08-28 03:59:28 +02:00
|
|
|
*/
|
2005-11-17 23:14:56 +01:00
|
|
|
#define ARR_OVERHEAD_NONULLS(ndims) \
|
|
|
|
|
MAXALIGN(sizeof(ArrayType) + 2 * sizeof(int) * (ndims))
|
|
|
|
|
#define ARR_OVERHEAD_WITHNULLS(ndims, nitems) \
|
|
|
|
|
MAXALIGN(sizeof(ArrayType) + 2 * sizeof(int) * (ndims) + \
|
|
|
|
|
((nitems) + 7) / 8)
|
|
|
|
|
|
|
|
|
|
#define ARR_DATA_OFFSET(a) \
|
|
|
|
|
(ARR_HASNULL(a) ? (a)->dataoffset : ARR_OVERHEAD_NONULLS(ARR_NDIM(a)))
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2000-07-22 05:34:43 +02:00
|
|
|
/*
|
|
|
|
|
* Returns a pointer to the actual array data.
|
1996-08-28 03:59:28 +02:00
|
|
|
*/
|
2000-07-22 05:34:43 +02:00
|
|
|
#define ARR_DATA_PTR(a) \
|
2005-11-17 23:14:56 +01:00
|
|
|
(((char *) (a)) + ARR_DATA_OFFSET(a))
|
1996-08-28 03:59:28 +02:00
|
|
|
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
/*
|
|
|
|
|
* Macros for working with AnyArrayType inputs. Beware multiple references!
|
|
|
|
|
*/
|
|
|
|
|
#define AARR_NDIM(a) \
|
2019-07-01 02:34:17 +02:00
|
|
|
(VARATT_IS_EXPANDED_HEADER(a) ? \
|
|
|
|
|
(a)->xpn.ndims : ARR_NDIM((ArrayType *) (a)))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
#define AARR_HASNULL(a) \
|
|
|
|
|
(VARATT_IS_EXPANDED_HEADER(a) ? \
|
|
|
|
|
((a)->xpn.dvalues != NULL ? (a)->xpn.dnulls != NULL : ARR_HASNULL((a)->xpn.fvalue)) : \
|
2019-07-01 02:34:17 +02:00
|
|
|
ARR_HASNULL((ArrayType *) (a)))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
#define AARR_ELEMTYPE(a) \
|
2019-07-01 02:34:17 +02:00
|
|
|
(VARATT_IS_EXPANDED_HEADER(a) ? \
|
|
|
|
|
(a)->xpn.element_type : ARR_ELEMTYPE((ArrayType *) (a)))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
#define AARR_DIMS(a) \
|
2019-07-01 02:34:17 +02:00
|
|
|
(VARATT_IS_EXPANDED_HEADER(a) ? \
|
|
|
|
|
(a)->xpn.dims : ARR_DIMS((ArrayType *) (a)))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
#define AARR_LBOUND(a) \
|
2019-07-01 02:34:17 +02:00
|
|
|
(VARATT_IS_EXPANDED_HEADER(a) ? \
|
|
|
|
|
(a)->xpn.lbound : ARR_LBOUND((ArrayType *) (a)))
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2005-11-17 23:14:56 +01:00
|
|
|
/*
|
|
|
|
|
* GUC parameter
|
|
|
|
|
*/
|
2022-04-08 14:16:38 +02:00
|
|
|
extern PGDLLIMPORT bool Array_nulls;
|
2005-11-17 23:14:56 +01:00
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/*
|
|
|
|
|
* prototypes for functions defined in arrayfuncs.c
|
|
|
|
|
*/
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
extern void CopyArrayEls(ArrayType *array,
|
|
|
|
|
Datum *values,
|
|
|
|
|
bool *nulls,
|
|
|
|
|
int nitems,
|
|
|
|
|
int typlen,
|
|
|
|
|
bool typbyval,
|
|
|
|
|
char typalign,
|
|
|
|
|
bool freedata);
|
|
|
|
|
|
Rationalize the APIs of array element/slice access functions.
The four functions array_ref, array_set, array_get_slice, array_set_slice
have traditionally declared their array inputs and results as being of type
"ArrayType *". This is a lie, and has been since Berkeley days, because
they actually also support "fixed-length array" types such as "name" and
"point"; not to mention that the inputs could be toasted. These values
should be declared Datum instead to avoid confusion. The current coding
already risks possible misoptimization by compilers, and it'll get worse
when "expanded" array representations become a valid alternative.
However, there's a fair amount of code using array_ref and array_set with
arrays that *are* known to be ArrayType structures, and there might be more
such places in third-party code. Rather than cluttering those call sites
with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the
existing functions to array_get_element/array_set_element, fix their
signatures, then reincarnate array_ref/array_set as backwards compatibility
wrappers.
array_get_slice/array_set_slice have no such constituency in the core code,
and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
|
|
|
extern Datum array_get_element(Datum arraydatum, int nSubscripts, int *indx,
|
|
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign,
|
|
|
|
|
bool *isNull);
|
|
|
|
|
extern Datum array_set_element(Datum arraydatum, int nSubscripts, int *indx,
|
|
|
|
|
Datum dataValue, bool isNull,
|
|
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
|
|
|
|
|
extern Datum array_get_slice(Datum arraydatum, int nSubscripts,
|
|
|
|
|
int *upperIndx, int *lowerIndx,
|
2015-12-23 03:05:16 +01:00
|
|
|
bool *upperProvided, bool *lowerProvided,
|
Rationalize the APIs of array element/slice access functions.
The four functions array_ref, array_set, array_get_slice, array_set_slice
have traditionally declared their array inputs and results as being of type
"ArrayType *". This is a lie, and has been since Berkeley days, because
they actually also support "fixed-length array" types such as "name" and
"point"; not to mention that the inputs could be toasted. These values
should be declared Datum instead to avoid confusion. The current coding
already risks possible misoptimization by compilers, and it'll get worse
when "expanded" array representations become a valid alternative.
However, there's a fair amount of code using array_ref and array_set with
arrays that *are* known to be ArrayType structures, and there might be more
such places in third-party code. Rather than cluttering those call sites
with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the
existing functions to array_get_element/array_set_element, fix their
signatures, then reincarnate array_ref/array_set as backwards compatibility
wrappers.
array_get_slice/array_set_slice have no such constituency in the core code,
and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
|
|
|
|
|
extern Datum array_set_slice(Datum arraydatum, int nSubscripts,
|
|
|
|
|
int *upperIndx, int *lowerIndx,
|
2015-12-23 03:05:16 +01:00
|
|
|
bool *upperProvided, bool *lowerProvided,
|
Rationalize the APIs of array element/slice access functions.
The four functions array_ref, array_set, array_get_slice, array_set_slice
have traditionally declared their array inputs and results as being of type
"ArrayType *". This is a lie, and has been since Berkeley days, because
they actually also support "fixed-length array" types such as "name" and
"point"; not to mention that the inputs could be toasted. These values
should be declared Datum instead to avoid confusion. The current coding
already risks possible misoptimization by compilers, and it'll get worse
when "expanded" array representations become a valid alternative.
However, there's a fair amount of code using array_ref and array_set with
arrays that *are* known to be ArrayType structures, and there might be more
such places in third-party code. Rather than cluttering those call sites
with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the
existing functions to array_get_element/array_set_element, fix their
signatures, then reincarnate array_ref/array_set as backwards compatibility
wrappers.
array_get_slice/array_set_slice have no such constituency in the core code,
and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
|
|
|
Datum srcArrayDatum, bool isNull,
|
|
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
|
|
|
|
|
|
2000-06-13 09:35:40 +02:00
|
|
|
extern Datum array_ref(ArrayType *array, int nSubscripts, int *indx,
|
2005-11-17 23:14:56 +01:00
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign,
|
2002-08-26 19:54:02 +02:00
|
|
|
bool *isNull);
|
2000-06-13 09:35:40 +02:00
|
|
|
extern ArrayType *array_set(ArrayType *array, int nSubscripts, int *indx,
|
2005-11-17 23:14:56 +01:00
|
|
|
Datum dataValue, bool isNull,
|
|
|
|
|
int arraytyplen, int elmlen, bool elmbyval, char elmalign);
|
2000-07-22 05:34:43 +02:00
|
|
|
|
Support arrays over domains.
Allowing arrays with a domain type as their element type was left un-done
in the original domain patch, but not for any very good reason. This
omission leads to such surprising results as array_agg() not working on
a domain column, because the parser can't identify a suitable output type
for the polymorphic aggregate.
In order to fix this, first clean up the APIs of coerce_to_domain() and
some internal functions in parse_coerce.c so that we consistently pass
around a CoercionContext along with CoercionForm. Previously, we sometimes
passed an "isExplicit" boolean flag instead, which is strictly less
information; and coerce_to_domain() didn't even get that, but instead had
to reverse-engineer isExplicit from CoercionForm. That's contrary to the
documentation in primnodes.h that says that CoercionForm only affects
display and not semantics. I don't think this change fixes any live bugs,
but it makes things more consistent. The main reason for doing it though
is that now build_coercion_expression() receives ccontext, which it needs
in order to be able to recursively invoke coerce_to_target_type().
Next, reimplement ArrayCoerceExpr so that the node does not directly know
any details of what has to be done to the individual array elements while
performing the array coercion. Instead, the per-element processing is
represented by a sub-expression whose input is a source array element and
whose output is a target array element. This simplifies life in
parse_coerce.c, because it can build that sub-expression by a recursive
invocation of coerce_to_target_type(). The executor now handles the
per-element processing as a compiled expression instead of hard-wired code.
The main advantage of this is that we can use a single ArrayCoerceExpr to
handle as many as three successive steps per element: base type conversion,
typmod coercion, and domain constraint checking. The old code used two
stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty
inefficient, and adding yet another array deconstruction to do domain
constraint checking seemed very unappetizing.
In the case where we just need a single, very simple coercion function,
doing this straightforwardly leads to a noticeable increase in the
per-array-element runtime cost. Hence, add an additional shortcut evalfunc
in execExprInterp.c that skips unnecessary overhead for that specific form
of expression. The runtime speed of simple cases is within 1% or so of
where it was before, while cases that previously required two levels of
array processing are significantly faster.
Finally, create an implicit array type for every domain type, as we do for
base types, enums, etc. Everything except the array-coercion case seems
to just work without further effort.
Tom Lane, reviewed by Andrew Dunstan
Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
|
|
|
extern Datum array_map(Datum arrayd,
|
|
|
|
|
struct ExprState *exprstate, struct ExprContext *econtext,
|
|
|
|
|
Oid retType, ArrayMapState *amstate);
|
2000-06-13 09:35:40 +02:00
|
|
|
|
2005-11-17 23:14:56 +01:00
|
|
|
extern void array_bitmap_copy(bits8 *destbitmap, int destoffset,
|
|
|
|
|
const bits8 *srcbitmap, int srcoffset,
|
|
|
|
|
int nitems);
|
|
|
|
|
|
2000-07-17 05:05:41 +02:00
|
|
|
extern ArrayType *construct_array(Datum *elems, int nelems,
|
2002-08-26 19:54:02 +02:00
|
|
|
Oid elmtype,
|
|
|
|
|
int elmlen, bool elmbyval, char elmalign);
|
2022-07-01 10:51:45 +02:00
|
|
|
extern ArrayType *construct_array_builtin(Datum *elems, int nelems, Oid elmtype);
|
2003-04-09 01:20:04 +02:00
|
|
|
extern ArrayType *construct_md_array(Datum *elems,
|
2005-11-17 23:14:56 +01:00
|
|
|
bool *nulls,
|
2003-04-09 01:20:04 +02:00
|
|
|
int ndims,
|
|
|
|
|
int *dims,
|
|
|
|
|
int *lbs,
|
|
|
|
|
Oid elmtype, int elmlen, bool elmbyval, char elmalign);
|
2005-11-17 23:14:56 +01:00
|
|
|
extern ArrayType *construct_empty_array(Oid elmtype);
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
extern ExpandedArrayHeader *construct_empty_expanded_array(Oid element_type,
|
|
|
|
|
MemoryContext parentcontext,
|
|
|
|
|
ArrayMetaState *metacache);
|
2000-07-17 05:05:41 +02:00
|
|
|
extern void deconstruct_array(ArrayType *array,
|
2002-08-26 19:54:02 +02:00
|
|
|
Oid elmtype,
|
|
|
|
|
int elmlen, bool elmbyval, char elmalign,
|
2005-11-17 23:14:56 +01:00
|
|
|
Datum **elemsp, bool **nullsp, int *nelemsp);
|
2022-07-01 10:51:45 +02:00
|
|
|
extern void deconstruct_array_builtin(ArrayType *array,
|
|
|
|
|
Oid elmtype,
|
|
|
|
|
Datum **elemsp, bool **nullsp, int *nelemsp);
|
2011-01-09 02:24:08 +01:00
|
|
|
extern bool array_contains_nulls(ArrayType *array);
|
2014-11-25 18:21:22 +01:00
|
|
|
|
|
|
|
|
extern ArrayBuildState *initArrayResult(Oid element_type,
|
2015-02-22 02:24:48 +01:00
|
|
|
MemoryContext rcontext, bool subcontext);
|
2023-01-23 05:35:01 +01:00
|
|
|
extern ArrayBuildState *initArrayResultWithSize(Oid element_type,
|
|
|
|
|
MemoryContext rcontext,
|
|
|
|
|
bool subcontext, int initsize);
|
2003-06-27 02:33:26 +02:00
|
|
|
extern ArrayBuildState *accumArrayResult(ArrayBuildState *astate,
|
|
|
|
|
Datum dvalue, bool disnull,
|
|
|
|
|
Oid element_type,
|
|
|
|
|
MemoryContext rcontext);
|
|
|
|
|
extern Datum makeArrayResult(ArrayBuildState *astate,
|
|
|
|
|
MemoryContext rcontext);
|
|
|
|
|
extern Datum makeMdArrayResult(ArrayBuildState *astate, int ndims,
|
2008-12-28 19:54:01 +01:00
|
|
|
int *dims, int *lbs, MemoryContext rcontext, bool release);
|
1996-08-28 03:59:28 +02:00
|
|
|
|
2014-11-25 18:21:22 +01:00
|
|
|
extern ArrayBuildStateArr *initArrayResultArr(Oid array_type, Oid element_type,
|
2015-02-22 02:24:48 +01:00
|
|
|
MemoryContext rcontext, bool subcontext);
|
2014-11-25 18:21:22 +01:00
|
|
|
extern ArrayBuildStateArr *accumArrayResultArr(ArrayBuildStateArr *astate,
|
|
|
|
|
Datum dvalue, bool disnull,
|
|
|
|
|
Oid array_type,
|
|
|
|
|
MemoryContext rcontext);
|
|
|
|
|
extern Datum makeArrayResultArr(ArrayBuildStateArr *astate,
|
|
|
|
|
MemoryContext rcontext, bool release);
|
|
|
|
|
|
|
|
|
|
extern ArrayBuildStateAny *initArrayResultAny(Oid input_type,
|
2015-02-22 02:24:48 +01:00
|
|
|
MemoryContext rcontext, bool subcontext);
|
2014-11-25 18:21:22 +01:00
|
|
|
extern ArrayBuildStateAny *accumArrayResultAny(ArrayBuildStateAny *astate,
|
|
|
|
|
Datum dvalue, bool disnull,
|
|
|
|
|
Oid input_type,
|
|
|
|
|
MemoryContext rcontext);
|
|
|
|
|
extern Datum makeArrayResultAny(ArrayBuildStateAny *astate,
|
|
|
|
|
MemoryContext rcontext, bool release);
|
|
|
|
|
|
2015-03-18 20:01:34 +01:00
|
|
|
extern ArrayIterator array_create_iterator(ArrayType *arr, int slice_ndim, ArrayMetaState *mstate);
|
2011-02-16 07:52:04 +01:00
|
|
|
extern bool array_iterate(ArrayIterator iterator, Datum *value, bool *isnull);
|
|
|
|
|
extern void array_free_iterator(ArrayIterator iterator);
|
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
/*
|
|
|
|
|
* prototypes for functions defined in arrayutils.c
|
|
|
|
|
*/
|
|
|
|
|
|
2005-11-17 23:14:56 +01:00
|
|
|
extern int ArrayGetOffset(int n, const int *dim, const int *lb, const int *indx);
|
|
|
|
|
extern int ArrayGetOffset0(int n, const int *tup, const int *scale);
|
|
|
|
|
extern int ArrayGetNItems(int ndim, const int *dims);
|
Convert a few datatype input functions to use "soft" error reporting.
This patch converts the input functions for bool, int2, int4, int8,
float4, float8, numeric, and contrib/cube to the new soft-error style.
array_in and record_in are also converted. There's lots more to do,
but this is enough to provide proof-of-concept that the soft-error
API is usable, as well as reference examples for how to convert
input functions.
This patch is mostly by me, but it owes very substantial debt to
earlier work by Nikita Glukhov, Andrew Dunstan, and Amul Sul.
Thanks to Andres Freund for review.
Discussion: https://postgr.es/m/3bbbb0df-7382-bf87-9737-340ba096e034@postgrespro.ru
2022-12-09 16:14:53 +01:00
|
|
|
extern int ArrayGetNItemsSafe(int ndim, const int *dims,
|
|
|
|
|
struct Node *escontext);
|
2021-05-10 16:44:38 +02:00
|
|
|
extern void ArrayCheckBounds(int ndim, const int *dims, const int *lb);
|
Convert a few datatype input functions to use "soft" error reporting.
This patch converts the input functions for bool, int2, int4, int8,
float4, float8, numeric, and contrib/cube to the new soft-error style.
array_in and record_in are also converted. There's lots more to do,
but this is enough to provide proof-of-concept that the soft-error
API is usable, as well as reference examples for how to convert
input functions.
This patch is mostly by me, but it owes very substantial debt to
earlier work by Nikita Glukhov, Andrew Dunstan, and Amul Sul.
Thanks to Andres Freund for review.
Discussion: https://postgr.es/m/3bbbb0df-7382-bf87-9737-340ba096e034@postgrespro.ru
2022-12-09 16:14:53 +01:00
|
|
|
extern bool ArrayCheckBoundsSafe(int ndim, const int *dims, const int *lb,
|
|
|
|
|
struct Node *escontext);
|
2005-11-17 23:14:56 +01:00
|
|
|
extern void mda_get_range(int n, int *span, const int *st, const int *endp);
|
|
|
|
|
extern void mda_get_prod(int n, const int *range, int *prod);
|
|
|
|
|
extern void mda_get_offset_values(int n, int *dist, const int *prod, const int *span);
|
|
|
|
|
extern int mda_next_tuple(int n, int *curr, const int *span);
|
2007-06-15 22:56:52 +02:00
|
|
|
extern int32 *ArrayGetIntegerTypmods(ArrayType *arr, int *n);
|
1997-09-07 07:04:48 +02:00
|
|
|
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
/*
|
|
|
|
|
* prototypes for functions defined in array_expanded.c
|
|
|
|
|
*/
|
|
|
|
|
extern Datum expand_array(Datum arraydatum, MemoryContext parentcontext,
|
|
|
|
|
ArrayMetaState *metacache);
|
|
|
|
|
extern ExpandedArrayHeader *DatumGetExpandedArray(Datum d);
|
|
|
|
|
extern ExpandedArrayHeader *DatumGetExpandedArrayX(Datum d,
|
|
|
|
|
ArrayMetaState *metacache);
|
2017-09-18 21:21:23 +02:00
|
|
|
extern AnyArrayType *DatumGetAnyArrayP(Datum d);
|
Support "expanded" objects, particularly arrays, for better performance.
This patch introduces the ability for complex datatypes to have an
in-memory representation that is different from their on-disk format.
On-disk formats are typically optimized for minimal size, and in any case
they can't contain pointers, so they are often not well-suited for
computation. Now a datatype can invent an "expanded" in-memory format
that is better suited for its operations, and then pass that around among
the C functions that operate on the datatype. There are also provisions
(rudimentary as yet) to allow an expanded object to be modified in-place
under suitable conditions, so that operations like assignment to an element
of an array need not involve copying the entire array.
The initial application for this feature is arrays, but it is not hard
to foresee using it for other container types like JSON, XML and hstore.
I have hopes that it will be useful to PostGIS as well.
In this initial implementation, a few heuristics have been hard-wired
into plpgsql to improve performance for arrays that are stored in
plpgsql variables. We would like to generalize those hacks so that
other datatypes can obtain similar improvements, but figuring out some
appropriate APIs is left as a task for future work. (The heuristics
themselves are probably not optimal yet, either, as they sometimes
force expansion of arrays that would be better left alone.)
Preliminary performance testing shows impressive speed gains for plpgsql
functions that do element-by-element access or update of large arrays.
There are other cases that get a little slower, as a result of added array
format conversions; but we can hope to improve anything that's annoyingly
bad. In any case most applications should see a net win.
Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
|
|
|
extern void deconstruct_expanded_array(ExpandedArrayHeader *eah);
|
|
|
|
|
|
1996-08-28 03:59:28 +02:00
|
|
|
#endif /* ARRAY_H */
|
