rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/utils/adt/array_userfuncs.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

1694 lines
43 KiB
C
Raw Normal View History

First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/*-------------------------------------------------------------------------
*
* array_userfuncs.c
* Misc user-visible array support functions
*
Update copyright for 2023 Backpatch-through: 11
2023-01-02 21:00:37 +01:00
* Copyright (c) 2003-2023, PostgreSQL Global Development Group
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/utils/adt/array_userfuncs.c
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*
*-------------------------------------------------------------------------
*/
#include "postgres.h"
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
#include "catalog/pg_type.h"
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
#include "libpq/pqformat.h"
Use new overflow aware integer operations. A previous commit added inline functions that provide fast(er) and correct overflow checks for signed integer math. Use them in a significant portion of backend code. There's more to touch in both backend and frontend code, but these were the easily identifiable cases. The old overflow checks are noticeable in integer heavy workloads. A secondary benefit is that getting rid of overflow checks that rely on signed integer overflow wrapping around, will allow us to get rid of -fwrapv in the future. Which in turn slows down other code. Author: Andres Freund Discussion: https://postgr.es/m/20171024103954.ztmatprlglz3rwke@alap3.anarazel.de
2017-12-13 01:32:31 +01:00
#include "common/int.h"
Add array_sample() and array_shuffle() functions. These are useful in Monte Carlo applications. Martin Kalcher, reviewed/adjusted by Daniel Gustafsson and myself Discussion: https://postgr.es/m/9d160a44-7675-51e8-60cf-6d64b76db831@aboutsource.net
2023-04-07 17:47:07 +02:00
#include "common/pg_prng.h"
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
#include "port/pg_bitutils.h"
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
#include "utils/array.h"
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
#include "utils/datum.h"
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
#include "utils/builtins.h"
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
#include "utils/lsyscache.h"
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
#include "utils/typcache.h"
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
/*
* SerialIOData
* Used for caching element-type data in array_agg_serialize
*/
typedef struct SerialIOData
{
FmgrInfo typsend;
} SerialIOData;
/*
* DeserialIOData
* Used for caching element-type data in array_agg_deserialize
*/
typedef struct DeserialIOData
{
FmgrInfo typreceive;
Oid typioparam;
} DeserialIOData;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
static Datum array_position_common(FunctionCallInfo fcinfo);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
/*
* fetch_array_arg_replace_nulls
*
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
* Fetch an array-valued argument in expanded form; if it's null, construct an
* empty array value of the proper data type. Also cache basic element type
* information in fn_extra.
*
* Caution: if the input is a read/write pointer, this returns the input
* argument; so callers must be sure that their changes are "safe", that is
* they cannot leave the array in a corrupt state.
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
*
* If we're being called as an aggregate function, make sure any newly-made
* expanded array is allocated in the aggregate state context, so as to save
* copying operations.
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
*/
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
static ExpandedArrayHeader *
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
fetch_array_arg_replace_nulls(FunctionCallInfo fcinfo, int argno)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
ExpandedArrayHeader *eah;
Fix over-optimistic caching in fetch_array_arg_replace_nulls(). When I rewrote this in commit 56a79a869bedc4bf6c35853642694cc0b0594dd2, I forgot that it's possible for the input array type to change from one call to the next (this can happen when applying the function to pg_statistic columns, for instance). Fix that.
2015-02-25 20:19:13 +01:00
Oid element_type;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
ArrayMetaState *my_extra;
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
MemoryContext resultcxt;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/* If first time through, create datatype cache struct */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
my_extra = (ArrayMetaState *)
MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(ArrayMetaState));
my_extra->element_type = InvalidOid;
fcinfo->flinfo->fn_extra = my_extra;
}
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
/* Figure out which context we want the result in */
if (!AggCheckCallContext(fcinfo, &resultcxt))
resultcxt = CurrentMemoryContext;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/* Now collect the array value */
Fix over-optimistic caching in fetch_array_arg_replace_nulls(). When I rewrote this in commit 56a79a869bedc4bf6c35853642694cc0b0594dd2, I forgot that it's possible for the input array type to change from one call to the next (this can happen when applying the function to pg_statistic columns, for instance). Fix that.
2015-02-25 20:19:13 +01:00
if (!PG_ARGISNULL(argno))
{
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
MemoryContext oldcxt = MemoryContextSwitchTo(resultcxt);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
eah = PG_GETARG_EXPANDED_ARRAYX(argno, my_extra);
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
MemoryContextSwitchTo(oldcxt);
Fix over-optimistic caching in fetch_array_arg_replace_nulls(). When I rewrote this in commit 56a79a869bedc4bf6c35853642694cc0b0594dd2, I forgot that it's possible for the input array type to change from one call to the next (this can happen when applying the function to pg_statistic columns, for instance). Fix that.
2015-02-25 20:19:13 +01:00
}
else
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
{
Fix over-optimistic caching in fetch_array_arg_replace_nulls(). When I rewrote this in commit 56a79a869bedc4bf6c35853642694cc0b0594dd2, I forgot that it's possible for the input array type to change from one call to the next (this can happen when applying the function to pg_statistic columns, for instance). Fix that.
2015-02-25 20:19:13 +01:00
/* We have to look up the array type and element type */
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
Oid arr_typeid = get_fn_expr_argtype(fcinfo->flinfo, argno);
if (!OidIsValid(arr_typeid))
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("could not determine input data type")));
element_type = get_element_type(arr_typeid);
if (!OidIsValid(element_type))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("input data type is not an array")));
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
eah = construct_empty_expanded_array(element_type,
Improve speed of aggregates that use array_append as transition function. In the previous coding, if an aggregate's transition function returned an expanded array, nodeAgg.c and nodeWindowAgg.c would always copy it and thus force it into the flat representation. This led to ping-ponging between flat and expanded formats, which costs a lot. For an aggregate using array_append as transition function, I measured about a 15X slowdown compared to the pre-9.5 code, when working on simple int[] arrays. Of course, the old code was already O(N^2) in this usage due to copying flat arrays all the time, but it wasn't quite this inefficient. To fix, teach nodeAgg.c and nodeWindowAgg.c to allow expanded transition values without copying, so long as the transition function takes care to return the transition value already properly parented under the aggcontext. That puts a bit of extra responsibility on the transition function, but doing it this way allows us to not need any extra logic in the fast path of advance_transition_function (ie, with a pass-by-value transition value, or with a modified-in-place pass-by-reference value). We already know that that's a hot spot so I'm loath to add any cycles at all there. Also, while only array_append currently knows how to follow this convention, this solution allows other transition functions to opt-in without needing to have a whitelist in the core aggregation code. (The reason we would need a whitelist is that currently, if you pass a R/W expanded-object pointer to an arbitrary function, it's allowed to do anything with it including deleting it; that breaks the core agg code's assumption that it should free discarded values. Returning a value under aggcontext is the transition function's signal that it knows it is an aggregate transition function and will play nice. Possibly the API rules for expanded objects should be refined, but that would not be a back-patchable change.) With this fix, an aggregate using array_append is no longer O(N^2), so it's much faster than pre-9.5 code rather than much slower. It's still a bit slower than the bespoke infrastructure for array_agg, but the differential seems to be only about 10%-20% rather than orders of magnitude. Discussion: <6315.1477677885@sss.pgh.pa.us>
2016-10-30 17:27:41 +01:00
resultcxt,
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
my_extra);
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
return eah;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
}
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/*-----------------------------------------------------------------------------
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
* array_append :
* push an element onto the end of a one-dimensional array
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*----------------------------------------------------------------------------
*/
Datum
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
array_append(PG_FUNCTION_ARGS)
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
ExpandedArrayHeader *eah;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Datum newelem;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool isNull;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Datum result;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
int *dimv,
*lb;
int indx;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *my_extra;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
eah = fetch_array_arg_replace_nulls(fcinfo, 0);
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
isNull = PG_ARGISNULL(1);
if (isNull)
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(1);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (eah->ndims == 1)
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
{
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
/* append newelem */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb = eah->lbound;
dimv = eah->dims;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
Use new overflow aware integer operations. A previous commit added inline functions that provide fast(er) and correct overflow checks for signed integer math. Use them in a significant portion of backend code. There's more to touch in both backend and frontend code, but these were the easily identifiable cases. The old overflow checks are noticeable in integer heavy workloads. A secondary benefit is that getting rid of overflow checks that rely on signed integer overflow wrapping around, will allow us to get rid of -fwrapv in the future. Which in turn slows down other code. Author: Andres Freund Discussion: https://postgr.es/m/20171024103954.ztmatprlglz3rwke@alap3.anarazel.de
2017-12-13 01:32:31 +01:00
/* index of added elem is at lb[0] + (dimv[0] - 1) + 1 */
if (pg_add_s32_overflow(lb[0], dimv[0], &indx))
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
ereport(ERROR,
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
else if (eah->ndims == 0)
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
indx = 1;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
else
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
(errcode(ERRCODE_DATA_EXCEPTION),
errmsg("argument must be empty or one-dimensional array")));
/* Perform element insertion */
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
result = array_set_element(EOHPGetRWDatum(&eah->hdr),
1, &indx, newelem, isNull,
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
PG_RETURN_DATUM(result);
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
}
/*-----------------------------------------------------------------------------
* array_prepend :
* push an element onto the front of a one-dimensional array
*----------------------------------------------------------------------------
*/
Datum
array_prepend(PG_FUNCTION_ARGS)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
ExpandedArrayHeader *eah;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
Datum newelem;
bool isNull;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Datum result;
Remove unused variable. Per buildfarm.
2015-02-20 06:47:28 +01:00
int *lb;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
int indx;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
int lb0;
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
ArrayMetaState *my_extra;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
isNull = PG_ARGISNULL(0);
if (isNull)
newelem = (Datum) 0;
else
newelem = PG_GETARG_DATUM(0);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
eah = fetch_array_arg_replace_nulls(fcinfo, 1);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (eah->ndims == 1)
Array mega-patch. Joe Conway
2003-06-25 01:14:49 +02:00
{
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
/* prepend newelem */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb = eah->lbound;
lb0 = lb[0];
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Use new overflow aware integer operations. A previous commit added inline functions that provide fast(er) and correct overflow checks for signed integer math. Use them in a significant portion of backend code. There's more to touch in both backend and frontend code, but these were the easily identifiable cases. The old overflow checks are noticeable in integer heavy workloads. A secondary benefit is that getting rid of overflow checks that rely on signed integer overflow wrapping around, will allow us to get rid of -fwrapv in the future. Which in turn slows down other code. Author: Andres Freund Discussion: https://postgr.es/m/20171024103954.ztmatprlglz3rwke@alap3.anarazel.de
2017-12-13 01:32:31 +01:00
if (pg_sub_s32_overflow(lb0, 1, &indx))
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
ereport(ERROR,
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
Array mega-patch. Joe Conway
2003-06-25 01:14:49 +02:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
else if (eah->ndims == 0)
{
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
indx = 1;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb0 = 1;
}
Array mega-patch. Joe Conway
2003-06-25 01:14:49 +02:00
else
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
OK, some of these syntax errors should be given other codes.
2003-09-15 22:03:37 +02:00
(errcode(ERRCODE_DATA_EXCEPTION),
Some "feature not supported" errors are better syntax errors, because the feature they complain about isn't a feature or cannot be implemented without definitional changes.
2003-09-10 01:22:21 +02:00
errmsg("argument must be empty or one-dimensional array")));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
/* Perform element insertion */
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
Back out array mega-patch. Joe Conway
2003-06-25 23:30:34 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
result = array_set_element(EOHPGetRWDatum(&eah->hdr),
1, &indx, newelem, isNull,
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
-1, my_extra->typlen, my_extra->typbyval, my_extra->typalign);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Split array_push into separate array_append and array_prepend functions. There wasn't any good reason for a single C function to implement both these SQL functions: it saved very little code overall, and it required significant pushups to re-determine at runtime which case applied. Redoing it as two functions ends up with just slightly more lines of code, but it's simpler to understand, and faster too because we need not repeat syscache lookups on every call. An important side benefit is that this eliminates the only case in which different aliases of the same C function had both anyarray and anyelement arguments at the same position, which would almost always be a mistake. The opr_sanity regression test will now notice such mistakes since there's no longer a valid case where it happens.
2015-02-19 02:53:14 +01:00
/* Readjust result's LB to match the input's, as expected for prepend */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Assert(result == EOHPGetRWDatum(&eah->hdr));
if (eah->ndims == 1)
{
/* This is ok whether we've deconstructed or not */
eah->lbound[0] = lb0;
}
Change array_push and array_cat so that they retain the lower bound of the array (for array_push) or higher-dimensional array (for array_cat) rather than decrementing it as before. This avoids generating lower bounds other than one for any array operation within the SQL spec. Per recent discussion. Interestingly, this seems to have been the original behavior, because while updating the docs I noticed that a large fraction of relevant examples were *wrong* for the old behavior and are now right. Is it worth correcting this in the back-branch docs?
2005-11-19 02:50:08 +01:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
PG_RETURN_DATUM(result);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
}
/*-----------------------------------------------------------------------------
* array_cat :
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
* concatenate two nD arrays to form an nD array, or
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
* push an (n-1)D array onto the end of an nD array
*----------------------------------------------------------------------------
*/
Datum
array_cat(PG_FUNCTION_ARGS)
{
ArrayType *v1,
*v2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ArrayType *result;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
int *dims,
*lbs,
ndims,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nitems,
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
ndatabytes,
nbytes;
int *dims1,
*lbs1,
ndims1,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nitems1,
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
ndatabytes1;
int *dims2,
*lbs2,
ndims2,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nitems2,
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
ndatabytes2;
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
int i;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
char *dat1,
*dat2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *bitmap1,
*bitmap2;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Oid element_type;
Oid element_type1;
Oid element_type2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int32 dataoffset;
/* Concatenating a null array is a no-op, just return the other input */
if (PG_ARGISNULL(0))
{
if (PG_ARGISNULL(1))
PG_RETURN_NULL();
result = PG_GETARG_ARRAYTYPE_P(1);
PG_RETURN_ARRAYTYPE_P(result);
}
if (PG_ARGISNULL(1))
{
result = PG_GETARG_ARRAYTYPE_P(0);
PG_RETURN_ARRAYTYPE_P(result);
}
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
v1 = PG_GETARG_ARRAYTYPE_P(0);
v2 = PG_GETARG_ARRAYTYPE_P(1);
Make array_cat more paranoid about checking datatypes in empty arrays.
2004-12-17 21:59:58 +01:00
element_type1 = ARR_ELEMTYPE(v1);
element_type2 = ARR_ELEMTYPE(v2);
/* Check we have matching element types */
if (element_type1 != element_type2)
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot concatenate incompatible arrays"),
errdetail("Arrays with element types %s and %s are not "
"compatible for concatenation.",
format_type_be(element_type1),
format_type_be(element_type2))));
/* OK, use it */
element_type = element_type1;
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
/*----------
* We must have one of the following combinations of inputs:
* 1) one empty array, and one non-empty array
* 2) both arrays empty
* 3) two arrays with ndims1 == ndims2
* 4) ndims1 == ndims2 - 1
* 5) ndims1 == ndims2 + 1
*----------
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*/
ndims1 = ARR_NDIM(v1);
ndims2 = ARR_NDIM(v2);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* short circuit - if one input array is empty, and the other is not, we
* return the non-empty one as the result
*
* if both are empty, return the first one
*/
if (ndims1 == 0 && ndims2 > 0)
PG_RETURN_ARRAYTYPE_P(v2);
if (ndims2 == 0)
PG_RETURN_ARRAYTYPE_P(v1);
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
/* the rest fall under rule 3, 4, or 5 */
if (ndims1 != ndims2 &&
ndims1 != ndims2 - 1 &&
ndims1 != ndims2 + 1)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot concatenate incompatible arrays"),
errdetail("Arrays of %d and %d dimensions are not "
"compatible for concatenation.",
ndims1, ndims2)));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/* get argument array details */
lbs1 = ARR_LBOUND(v1);
lbs2 = ARR_LBOUND(v2);
dims1 = ARR_DIMS(v1);
dims2 = ARR_DIMS(v2);
dat1 = ARR_DATA_PTR(v1);
dat2 = ARR_DATA_PTR(v2);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bitmap1 = ARR_NULLBITMAP(v1);
bitmap2 = ARR_NULLBITMAP(v2);
nitems1 = ArrayGetNItems(ndims1, dims1);
nitems2 = ArrayGetNItems(ndims2, dims2);
ndatabytes1 = ARR_SIZE(v1) - ARR_DATA_OFFSET(v1);
ndatabytes2 = ARR_SIZE(v2) - ARR_DATA_OFFSET(v2);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
if (ndims1 == ndims2)
{
/*
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
* resulting array is made up of the elements (possibly arrays
* themselves) of the input argument arrays
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*/
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
ndims = ndims1;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
dims = (int *) palloc(ndims * sizeof(int));
lbs = (int *) palloc(ndims * sizeof(int));
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
dims[0] = dims1[0] + dims2[0];
lbs[0] = lbs1[0];
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
for (i = 1; i < ndims; i++)
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
{
if (dims1[i] != dims2[i] || lbs1[i] != lbs2[i])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot concatenate incompatible arrays"),
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
errdetail("Arrays with differing element dimensions are "
"not compatible for concatenation.")));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
dims[i] = dims1[i];
lbs[i] = lbs1[i];
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
}
}
else if (ndims1 == ndims2 - 1)
{
/*
* resulting array has the second argument as the outer array, with
Change array_push and array_cat so that they retain the lower bound of the array (for array_push) or higher-dimensional array (for array_cat) rather than decrementing it as before. This avoids generating lower bounds other than one for any array operation within the SQL spec. Per recent discussion. Interestingly, this seems to have been the original behavior, because while updating the docs I noticed that a large fraction of relevant examples were *wrong* for the old behavior and are now right. Is it worth correcting this in the back-branch docs?
2005-11-19 02:50:08 +01:00
* the first argument inserted at the front of the outer dimension
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*/
ndims = ndims2;
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
dims = (int *) palloc(ndims * sizeof(int));
lbs = (int *) palloc(ndims * sizeof(int));
memcpy(dims, dims2, ndims * sizeof(int));
memcpy(lbs, lbs2, ndims * sizeof(int));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/* increment number of elements in outer array */
dims[0] += 1;
/* make sure the added element matches our existing elements */
for (i = 0; i < ndims1; i++)
{
if (dims1[i] != dims[i + 1] || lbs1[i] != lbs[i + 1])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot concatenate incompatible arrays"),
errdetail("Arrays with differing dimensions are not "
"compatible for concatenation.")));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
}
}
else
{
/*
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
* (ndims1 == ndims2 + 1)
*
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
* resulting array has the first argument as the outer array, with the
* second argument appended to the end of the outer dimension
*/
ndims = ndims1;
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
dims = (int *) palloc(ndims * sizeof(int));
lbs = (int *) palloc(ndims * sizeof(int));
memcpy(dims, dims1, ndims * sizeof(int));
memcpy(lbs, lbs1, ndims * sizeof(int));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/* increment number of elements in outer array */
dims[0] += 1;
/* make sure the added element matches our existing elements */
for (i = 0; i < ndims2; i++)
{
if (dims2[i] != dims[i + 1] || lbs2[i] != lbs[i + 1])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot concatenate incompatible arrays"),
errdetail("Arrays with differing dimensions are not "
"compatible for concatenation.")));
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
}
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Do this mainly for overflow checking */
nitems = ArrayGetNItems(ndims, dims);
Prevent integer overflows in array subscripting calculations. While we were (mostly) careful about ensuring that the dimensions of arrays aren't large enough to cause integer overflow, the lower bound values were generally not checked. This allows situations where lower_bound + dimension overflows an integer. It seems that that's harmless so far as array reading is concerned, except that array elements with subscripts notionally exceeding INT_MAX are inaccessible. However, it confuses various array-assignment logic, resulting in a potential for memory stomps. Fix by adding checks that array lower bounds aren't large enough to cause lower_bound + dimension to overflow. (Note: this results in disallowing cases where the last subscript position would be exactly INT_MAX. In principle we could probably allow that, but there's a lot of code that computes lower_bound + dimension and would need adjustment. It seems doubtful that it's worth the trouble/risk to allow it.) Somewhat independently of that, array_set_element() was careless about possible overflow when checking the subscript of a fixed-length array, creating a different route to memory stomps. Fix that too. Security: CVE-2021-32027
2021-05-10 16:44:38 +02:00
ArrayCheckBounds(ndims, dims, lbs);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/* build the result array */
ndatabytes = ndatabytes1 + ndatabytes2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (ARR_HASNULL(v1) || ARR_HASNULL(v2))
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, nitems);
nbytes = ndatabytes + dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes = ndatabytes + ARR_OVERHEAD_NONULLS(ndims);
}
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
result = (ArrayType *) palloc0(nbytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(result, nbytes);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
result->ndim = ndims;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
result->dataoffset = dataoffset;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
result->elemtype = element_type;
memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
/* data area is arg1 then arg2 */
memcpy(ARR_DATA_PTR(result), dat1, ndatabytes1);
memcpy(ARR_DATA_PTR(result) + ndatabytes1, dat2, ndatabytes2);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* handle the null bitmap if needed */
if (ARR_HASNULL(result))
{
array_bitmap_copy(ARR_NULLBITMAP(result), 0,
bitmap1, 0,
nitems1);
array_bitmap_copy(ARR_NULLBITMAP(result), nitems1,
bitmap2, 0,
nitems2);
}
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
PG_RETURN_ARRAYTYPE_P(result);
}
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
/*
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* ARRAY_AGG(anynonarray) aggregate function
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
*/
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
Datum
array_agg_transfn(PG_FUNCTION_ARGS)
{
Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
MemoryContext aggcontext;
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
ArrayBuildState *state;
Datum elem;
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
if (arg1_typeid == InvalidOid)
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("could not determine input data type")));
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/*
* Note: we do not need a run-time check about whether arg1_typeid is a
* valid array element type, because the parser would have verified that
* while resolving the input/result types of this polymorphic aggregate.
*/
Create an official API function for C functions to use to check if they are being called as aggregates, and to get the aggregate transition state memory context if needed. Use it instead of poking directly into AggState and WindowAggState in places that shouldn't know so much. We should have done this in 8.4, probably, but better late than never. Revised version of a patch by Hitoshi Harada.
2010-02-08 21:39:52 +01:00
if (!AggCheckCallContext(fcinfo, &aggcontext))
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
{
/* cannot be called directly because of internal-type argument */
elog(ERROR, "array_agg_transfn called in non-aggregate context");
}
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
if (PG_ARGISNULL(0))
state = initArrayResult(arg1_typeid, aggcontext, false);
else
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
elem = PG_ARGISNULL(1) ? (Datum) 0 : PG_GETARG_DATUM(1);
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
state = accumArrayResult(state,
elem,
PG_ARGISNULL(1),
arg1_typeid,
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
aggcontext);
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
/*
Actually, instead of whining about how type internal might not safely store a pointer, why don't we just fix that. Every known use of "internal" really means a pointer anyway.
2008-11-14 03:09:52 +01:00
* The transition type for array_agg() is declared to be "internal", which
* is a pass-by-value type the same size as a pointer. So we can safely
* pass the ArrayBuildState pointer through nodeAgg.c's machinations.
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
*/
PG_RETURN_POINTER(state);
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
}
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
Datum
array_agg_combine(PG_FUNCTION_ARGS)
{
ArrayBuildState *state1;
ArrayBuildState *state2;
MemoryContext agg_context;
MemoryContext old_context;
if (!AggCheckCallContext(fcinfo, &agg_context))
elog(ERROR, "aggregate function called in non-aggregate context");
state1 = PG_ARGISNULL(0) ? NULL : (ArrayBuildState *) PG_GETARG_POINTER(0);
state2 = PG_ARGISNULL(1) ? NULL : (ArrayBuildState *) PG_GETARG_POINTER(1);
if (state2 == NULL)
{
/*
* NULL state2 is easy, just return state1, which we know is already
* in the agg_context
*/
if (state1 == NULL)
PG_RETURN_NULL();
PG_RETURN_POINTER(state1);
}
if (state1 == NULL)
{
/* We must copy state2's data into the agg_context */
state1 = initArrayResultWithSize(state2->element_type, agg_context,
false, state2->alen);
old_context = MemoryContextSwitchTo(agg_context);
for (int i = 0; i < state2->nelems; i++)
{
if (!state2->dnulls[i])
state1->dvalues[i] = datumCopy(state2->dvalues[i],
state1->typbyval,
state1->typlen);
else
state1->dvalues[i] = (Datum) 0;
}
MemoryContextSwitchTo(old_context);
memcpy(state1->dnulls, state2->dnulls, sizeof(bool) * state2->nelems);
state1->nelems = state2->nelems;
PG_RETURN_POINTER(state1);
}
else if (state2->nelems > 0)
{
/* We only need to combine the two states if state2 has any elements */
int reqsize = state1->nelems + state2->nelems;
MemoryContext oldContext = MemoryContextSwitchTo(state1->mcontext);
Assert(state1->element_type == state2->element_type);
/* Enlarge state1 arrays if needed */
if (state1->alen < reqsize)
{
/* Use a power of 2 size rather than allocating just reqsize */
state1->alen = pg_nextpower2_32(reqsize);
state1->dvalues = (Datum *) repalloc(state1->dvalues,
state1->alen * sizeof(Datum));
state1->dnulls = (bool *) repalloc(state1->dnulls,
state1->alen * sizeof(bool));
}
/* Copy in the state2 elements to the end of the state1 arrays */
for (int i = 0; i < state2->nelems; i++)
{
if (!state2->dnulls[i])
state1->dvalues[i + state1->nelems] =
datumCopy(state2->dvalues[i],
state1->typbyval,
state1->typlen);
else
state1->dvalues[i + state1->nelems] = (Datum) 0;
}
memcpy(&state1->dnulls[state1->nelems], state2->dnulls,
sizeof(bool) * state2->nelems);
state1->nelems = reqsize;
MemoryContextSwitchTo(oldContext);
}
PG_RETURN_POINTER(state1);
}
/*
* array_agg_serialize
* Serialize ArrayBuildState into bytea.
*/
Datum
array_agg_serialize(PG_FUNCTION_ARGS)
{
ArrayBuildState *state;
StringInfoData buf;
bytea *result;
/* cannot be called directly because of internal-type argument */
Assert(AggCheckCallContext(fcinfo, NULL));
state = (ArrayBuildState *) PG_GETARG_POINTER(0);
pq_begintypsend(&buf);
/*
* element_type. Putting this first is more convenient in deserialization
*/
pq_sendint32(&buf, state->element_type);
/*
* nelems -- send first so we know how large to make the dvalues and
* dnulls array during deserialization.
*/
pq_sendint64(&buf, state->nelems);
/* alen can be decided during deserialization */
/* typlen */
pq_sendint16(&buf, state->typlen);
/* typbyval */
pq_sendbyte(&buf, state->typbyval);
/* typalign */
pq_sendbyte(&buf, state->typalign);
/* dnulls */
Change argument type of pq_sendbytes from char * to void * This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
2023-02-14 13:32:19 +01:00
pq_sendbytes(&buf, state->dnulls, sizeof(bool) * state->nelems);
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
/*
* dvalues. By agreement with array_agg_deserialize, when the element
* type is byval, we just transmit the Datum array as-is, including any
* null elements. For by-ref types, we must invoke the element type's
* send function, and we skip null elements (which is why the nulls flags
* must be sent first).
*/
if (state->typbyval)
Change argument type of pq_sendbytes from char * to void * This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
2023-02-14 13:32:19 +01:00
pq_sendbytes(&buf, state->dvalues, sizeof(Datum) * state->nelems);
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
else
{
SerialIOData *iodata;
int i;
/* Avoid repeat catalog lookups for typsend function */
iodata = (SerialIOData *) fcinfo->flinfo->fn_extra;
if (iodata == NULL)
{
Oid typsend;
bool typisvarlena;
iodata = (SerialIOData *)
MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(SerialIOData));
getTypeBinaryOutputInfo(state->element_type, &typsend,
&typisvarlena);
fmgr_info_cxt(typsend, &iodata->typsend,
fcinfo->flinfo->fn_mcxt);
fcinfo->flinfo->fn_extra = (void *) iodata;
}
for (i = 0; i < state->nelems; i++)
{
bytea *outputbytes;
if (state->dnulls[i])
continue;
outputbytes = SendFunctionCall(&iodata->typsend,
state->dvalues[i]);
pq_sendint32(&buf, VARSIZE(outputbytes) - VARHDRSZ);
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
}
}
result = pq_endtypsend(&buf);
PG_RETURN_BYTEA_P(result);
}
Datum
array_agg_deserialize(PG_FUNCTION_ARGS)
{
bytea *sstate;
ArrayBuildState *result;
StringInfoData buf;
Oid element_type;
int64 nelems;
const char *temp;
if (!AggCheckCallContext(fcinfo, NULL))
elog(ERROR, "aggregate function called in non-aggregate context");
sstate = PG_GETARG_BYTEA_PP(0);
/*
* Copy the bytea into a StringInfo so that we can "receive" it using the
* standard recv-function infrastructure.
*/
initStringInfo(&buf);
appendBinaryStringInfo(&buf,
VARDATA_ANY(sstate), VARSIZE_ANY_EXHDR(sstate));
/* element_type */
element_type = pq_getmsgint(&buf, 4);
/* nelems */
nelems = pq_getmsgint64(&buf);
/* Create output ArrayBuildState with the needed number of elements */
result = initArrayResultWithSize(element_type, CurrentMemoryContext,
false, nelems);
result->nelems = nelems;
/* typlen */
result->typlen = pq_getmsgint(&buf, 2);
/* typbyval */
result->typbyval = pq_getmsgbyte(&buf);
/* typalign */
result->typalign = pq_getmsgbyte(&buf);
/* dnulls */
temp = pq_getmsgbytes(&buf, sizeof(bool) * nelems);
memcpy(result->dnulls, temp, sizeof(bool) * nelems);
/* dvalues --- see comment in array_agg_serialize */
if (result->typbyval)
{
temp = pq_getmsgbytes(&buf, sizeof(Datum) * nelems);
memcpy(result->dvalues, temp, sizeof(Datum) * nelems);
}
else
{
DeserialIOData *iodata;
/* Avoid repeat catalog lookups for typreceive function */
iodata = (DeserialIOData *) fcinfo->flinfo->fn_extra;
if (iodata == NULL)
{
Oid typreceive;
iodata = (DeserialIOData *)
MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(DeserialIOData));
getTypeBinaryInputInfo(element_type, &typreceive,
&iodata->typioparam);
fmgr_info_cxt(typreceive, &iodata->typreceive,
fcinfo->flinfo->fn_mcxt);
fcinfo->flinfo->fn_extra = (void *) iodata;
}
for (int i = 0; i < nelems; i++)
{
int itemlen;
StringInfoData elem_buf;
char csave;
if (result->dnulls[i])
{
result->dvalues[i] = (Datum) 0;
continue;
}
itemlen = pq_getmsgint(&buf, 4);
if (itemlen < 0 || itemlen > (buf.len - buf.cursor))
ereport(ERROR,
(errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
errmsg("insufficient data left in message")));
/*
* Rather than copying data around, we just set up a phony
* StringInfo pointing to the correct portion of the input buffer.
* We assume we can scribble on the input buffer so as to maintain
* the convention that StringInfos have a trailing null.
*/
elem_buf.data = &buf.data[buf.cursor];
elem_buf.maxlen = itemlen + 1;
elem_buf.len = itemlen;
elem_buf.cursor = 0;
buf.cursor += itemlen;
csave = buf.data[buf.cursor];
buf.data[buf.cursor] = '\0';
/* Now call the element's receiveproc */
result->dvalues[i] = ReceiveFunctionCall(&iodata->typreceive,
&elem_buf,
iodata->typioparam,
-1);
buf.data[buf.cursor] = csave;
}
}
pq_getmsgend(&buf);
pfree(buf.data);
PG_RETURN_POINTER(result);
}
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
Datum
array_agg_finalfn(PG_FUNCTION_ARGS)
{
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
Datum result;
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
ArrayBuildState *state;
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
int dims[1];
int lbs[1];
Minor code clarity improvements in array_agg functions, and add a comment about how this is playing fast and loose with the type system.
2008-11-14 01:12:08 +01:00
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
/* cannot be called directly because of internal-type argument */
Create an official API function for C functions to use to check if they are being called as aggregates, and to get the aggregate transition state memory context if needed. Use it instead of poking directly into AggState and WindowAggState in places that shouldn't know so much. We should have done this in 8.4, probably, but better late than never. Revised version of a patch by Hitoshi Harada.
2010-02-08 21:39:52 +01:00
Assert(AggCheckCallContext(fcinfo, NULL));
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
state = PG_ARGISNULL(0) ? NULL : (ArrayBuildState *) PG_GETARG_POINTER(0);
if (state == NULL)
PG_RETURN_NULL(); /* returns null iff no input values */
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
dims[0] = state->nelems;
lbs[0] = 1;
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
/*
* Make the result. We cannot release the ArrayBuildState because
In a non-hashed Agg node, reset the "aggcontext" at group boundaries, instead of individually pfree'ing pass-by-reference transition values. This should be at least as fast as the prior coding, and it has the major advantage of clearing out any working data an aggregate function may have stored in or underneath the aggcontext. This avoids memory leakage when an aggregate such as array_agg() is used in GROUP BY mode. Per report from Chris Spotts. Back-patch to 8.4. In principle the problem could arise in prior versions, but since they didn't have array_agg the issue seems not critical.
2009-07-23 22:45:27 +02:00
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
*/
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
result = makeMdArrayResult(state, 1, dims, lbs,
CurrentMemoryContext,
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
false);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
PG_RETURN_DATUM(result);
array_agg aggregate function, as per SQL:2008, but without ORDER BY clause Rearrange the documentation a bit now that array_agg and xmlagg have similar semantics and issues. best of Robert Haas, Jeff Davis, Peter Eisentraut
2008-11-13 16:59:51 +01:00
}
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/*
* ARRAY_AGG(anyarray) aggregate function
*/
Datum
array_agg_array_transfn(PG_FUNCTION_ARGS)
{
Oid arg1_typeid = get_fn_expr_argtype(fcinfo->flinfo, 1);
MemoryContext aggcontext;
ArrayBuildStateArr *state;
if (arg1_typeid == InvalidOid)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("could not determine input data type")));
/*
* Note: we do not need a run-time check about whether arg1_typeid is a
* valid array type, because the parser would have verified that while
* resolving the input/result types of this polymorphic aggregate.
*/
if (!AggCheckCallContext(fcinfo, &aggcontext))
{
/* cannot be called directly because of internal-type argument */
elog(ERROR, "array_agg_array_transfn called in non-aggregate context");
}
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
if (PG_ARGISNULL(0))
state = initArrayResultArr(arg1_typeid, InvalidOid, aggcontext, false);
else
state = (ArrayBuildStateArr *) PG_GETARG_POINTER(0);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
state = accumArrayResultArr(state,
PG_GETARG_DATUM(1),
PG_ARGISNULL(1),
arg1_typeid,
aggcontext);
/*
* The transition type for array_agg() is declared to be "internal", which
* is a pass-by-value type the same size as a pointer. So we can safely
* pass the ArrayBuildStateArr pointer through nodeAgg.c's machinations.
*/
PG_RETURN_POINTER(state);
}
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
Datum
array_agg_array_combine(PG_FUNCTION_ARGS)
{
ArrayBuildStateArr *state1;
ArrayBuildStateArr *state2;
MemoryContext agg_context;
MemoryContext old_context;
if (!AggCheckCallContext(fcinfo, &agg_context))
elog(ERROR, "aggregate function called in non-aggregate context");
state1 = PG_ARGISNULL(0) ? NULL : (ArrayBuildStateArr *) PG_GETARG_POINTER(0);
state2 = PG_ARGISNULL(1) ? NULL : (ArrayBuildStateArr *) PG_GETARG_POINTER(1);
if (state2 == NULL)
{
/*
* NULL state2 is easy, just return state1, which we know is already
* in the agg_context
*/
if (state1 == NULL)
PG_RETURN_NULL();
PG_RETURN_POINTER(state1);
}
if (state1 == NULL)
{
/* We must copy state2's data into the agg_context */
old_context = MemoryContextSwitchTo(agg_context);
state1 = initArrayResultArr(state2->array_type, InvalidOid,
agg_context, false);
state1->abytes = state2->abytes;
state1->data = (char *) palloc(state1->abytes);
if (state2->nullbitmap)
{
int size = (state2->aitems + 7) / 8;
state1->nullbitmap = (bits8 *) palloc(size);
memcpy(state1->nullbitmap, state2->nullbitmap, size);
}
memcpy(state1->data, state2->data, state2->nbytes);
state1->nbytes = state2->nbytes;
state1->aitems = state2->aitems;
state1->nitems = state2->nitems;
state1->ndims = state2->ndims;
memcpy(state1->dims, state2->dims, sizeof(state2->dims));
memcpy(state1->lbs, state2->lbs, sizeof(state2->lbs));
state1->array_type = state2->array_type;
state1->element_type = state2->element_type;
MemoryContextSwitchTo(old_context);
PG_RETURN_POINTER(state1);
}
/* We only need to combine the two states if state2 has any items */
else if (state2->nitems > 0)
{
MemoryContext oldContext;
int reqsize = state1->nbytes + state2->nbytes;
int i;
/*
* Check the states are compatible with each other. Ensure we use the
* same error messages that are listed in accumArrayResultArr so that
* the same error is shown as would have been if we'd not used the
* combine function for the aggregation.
*/
if (state1->ndims != state2->ndims)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot accumulate arrays of different dimensionality")));
/* Check dimensions match ignoring the first dimension. */
for (i = 1; i < state1->ndims; i++)
{
if (state1->dims[i] != state2->dims[i] || state1->lbs[i] != state2->lbs[i])
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot accumulate arrays of different dimensionality")));
}
oldContext = MemoryContextSwitchTo(state1->mcontext);
/*
* If there's not enough space in state1 then we'll need to reallocate
* more.
*/
if (state1->abytes < reqsize)
{
/* use a power of 2 size rather than allocating just reqsize */
state1->abytes = pg_nextpower2_32(reqsize);
state1->data = (char *) repalloc(state1->data, state1->abytes);
}
if (state2->nullbitmap)
{
int newnitems = state1->nitems + state2->nitems;
if (state1->nullbitmap == NULL)
{
/*
* First input with nulls; we must retrospectively handle any
* previous inputs by marking all their items non-null.
*/
state1->aitems = pg_nextpower2_32(Max(256, newnitems + 1));
state1->nullbitmap = (bits8 *) palloc((state1->aitems + 7) / 8);
array_bitmap_copy(state1->nullbitmap, 0,
NULL, 0,
state1->nitems);
}
else if (newnitems > state1->aitems)
{
int newaitems = state1->aitems + state2->aitems;
state1->aitems = pg_nextpower2_32(newaitems);
state1->nullbitmap = (bits8 *)
repalloc(state1->nullbitmap, (state1->aitems + 7) / 8);
}
array_bitmap_copy(state1->nullbitmap, state1->nitems,
state2->nullbitmap, 0,
state2->nitems);
}
memcpy(state1->data + state1->nbytes, state2->data, state2->nbytes);
state1->nbytes += state2->nbytes;
state1->nitems += state2->nitems;
state1->dims[0] += state2->dims[0];
Fix various typos This fixes many spelling mistakes in comments, but a few references to invalid parameter names, function names and option names too in comments and also some in string constants Also, fix an #undef that was undefining the incorrect definition Author: Alexander Lakhin Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/d5f68d19-c0fc-91a9-118d-7c6a5a3f5fad@gmail.com
2023-04-18 03:23:23 +02:00
/* remaining dims already match, per test above */
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
Assert(state1->array_type == state2->array_type);
Assert(state1->element_type == state2->element_type);
MemoryContextSwitchTo(oldContext);
}
PG_RETURN_POINTER(state1);
}
/*
* array_agg_array_serialize
* Serialize ArrayBuildStateArr into bytea.
*/
Datum
array_agg_array_serialize(PG_FUNCTION_ARGS)
{
ArrayBuildStateArr *state;
StringInfoData buf;
bytea *result;
/* cannot be called directly because of internal-type argument */
Assert(AggCheckCallContext(fcinfo, NULL));
state = (ArrayBuildStateArr *) PG_GETARG_POINTER(0);
pq_begintypsend(&buf);
/*
* element_type. Putting this first is more convenient in deserialization
* so that we can init the new state sooner.
*/
pq_sendint32(&buf, state->element_type);
/* array_type */
pq_sendint32(&buf, state->array_type);
/* nbytes */
pq_sendint32(&buf, state->nbytes);
/* data */
pq_sendbytes(&buf, state->data, state->nbytes);
/* abytes */
pq_sendint32(&buf, state->abytes);
/* aitems */
pq_sendint32(&buf, state->aitems);
/* nullbitmap */
if (state->nullbitmap)
{
Assert(state->aitems > 0);
Change argument type of pq_sendbytes from char * to void * This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
2023-02-14 13:32:19 +01:00
pq_sendbytes(&buf, state->nullbitmap, (state->aitems + 7) / 8);
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
}
/* nitems */
pq_sendint32(&buf, state->nitems);
/* ndims */
pq_sendint32(&buf, state->ndims);
/* dims: XXX should we just send ndims elements? */
Change argument type of pq_sendbytes from char * to void * This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
2023-02-14 13:32:19 +01:00
pq_sendbytes(&buf, state->dims, sizeof(state->dims));
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
/* lbs */
Change argument type of pq_sendbytes from char * to void * This is a follow-up to 1f605b82ba66ece8b421b10d41094dd2e3c0c48b. It allows getting rid of further casts at call sites. Reviewed-by: Corey Huinker <corey.huinker@gmail.com> Discussion: https://www.postgresql.org/message-id/783a4edb-84f9-6df2-7470-2ef5ccc6607a@enterprisedb.com
2023-02-14 13:32:19 +01:00
pq_sendbytes(&buf, state->lbs, sizeof(state->lbs));
Allow parallel aggregate on string_agg and array_agg This adds combine, serial and deserial functions for the array_agg() and string_agg() aggregate functions, thus allowing these aggregates to partake in partial aggregations. This allows both parallel aggregation to take place when these aggregates are present and also allows additional partition-wise aggregation plan shapes to include plans that require additional aggregation once the partially aggregated results from the partitions have been combined. Author: David Rowley Reviewed-by: Andres Freund, Tomas Vondra, Stephen Frost, Tom Lane Discussion: https://postgr.es/m/CAKJS1f9sx_6GTcvd6TMuZnNtCh0VhBzhX6FZqw17TgVFH-ga_A@mail.gmail.com
2023-01-23 05:35:01 +01:00
result = pq_endtypsend(&buf);
PG_RETURN_BYTEA_P(result);
}
Datum
array_agg_array_deserialize(PG_FUNCTION_ARGS)
{
bytea *sstate;
ArrayBuildStateArr *result;
StringInfoData buf;
Oid element_type;
Oid array_type;
int nbytes;
const char *temp;
/* cannot be called directly because of internal-type argument */
Assert(AggCheckCallContext(fcinfo, NULL));
sstate = PG_GETARG_BYTEA_PP(0);
/*
* Copy the bytea into a StringInfo so that we can "receive" it using the
* standard recv-function infrastructure.
*/
initStringInfo(&buf);
appendBinaryStringInfo(&buf,
VARDATA_ANY(sstate), VARSIZE_ANY_EXHDR(sstate));
/* element_type */
element_type = pq_getmsgint(&buf, 4);
/* array_type */
array_type = pq_getmsgint(&buf, 4);
/* nbytes */
nbytes = pq_getmsgint(&buf, 4);
result = initArrayResultArr(array_type, element_type,
CurrentMemoryContext, false);
result->abytes = 1024;
while (result->abytes < nbytes)
result->abytes *= 2;
result->data = (char *) palloc(result->abytes);
/* data */
temp = pq_getmsgbytes(&buf, nbytes);
memcpy(result->data, temp, nbytes);
result->nbytes = nbytes;
/* abytes */
result->abytes = pq_getmsgint(&buf, 4);
/* aitems: might be 0 */
result->aitems = pq_getmsgint(&buf, 4);
/* nullbitmap */
if (result->aitems > 0)
{
int size = (result->aitems + 7) / 8;
result->nullbitmap = (bits8 *) palloc(size);
temp = pq_getmsgbytes(&buf, size);
memcpy(result->nullbitmap, temp, size);
}
else
result->nullbitmap = NULL;
/* nitems */
result->nitems = pq_getmsgint(&buf, 4);
/* ndims */
result->ndims = pq_getmsgint(&buf, 4);
/* dims */
temp = pq_getmsgbytes(&buf, sizeof(result->dims));
memcpy(result->dims, temp, sizeof(result->dims));
/* lbs */
temp = pq_getmsgbytes(&buf, sizeof(result->lbs));
memcpy(result->lbs, temp, sizeof(result->lbs));
pq_getmsgend(&buf);
pfree(buf.data);
PG_RETURN_POINTER(result);
}
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
Datum
array_agg_array_finalfn(PG_FUNCTION_ARGS)
{
Datum result;
ArrayBuildStateArr *state;
/* cannot be called directly because of internal-type argument */
Assert(AggCheckCallContext(fcinfo, NULL));
state = PG_ARGISNULL(0) ? NULL : (ArrayBuildStateArr *) PG_GETARG_POINTER(0);
if (state == NULL)
PG_RETURN_NULL(); /* returns null iff no input values */
/*
* Make the result. We cannot release the ArrayBuildStateArr because
* sometimes aggregate final functions are re-executed. Rather, it is
* nodeAgg.c's responsibility to reset the aggcontext when it's safe to do
* so.
*/
result = makeArrayResultArr(state, CurrentMemoryContext, false);
PG_RETURN_DATUM(result);
}
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/*-----------------------------------------------------------------------------
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
* array_position, array_position_start :
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
* return the offset of a value in an array.
*
* IS NOT DISTINCT FROM semantics are used for comparisons. Return NULL when
* the value is not found.
*-----------------------------------------------------------------------------
*/
Datum
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
array_position(PG_FUNCTION_ARGS)
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
{
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
return array_position_common(fcinfo);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
Datum
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
array_position_start(PG_FUNCTION_ARGS)
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
{
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
return array_position_common(fcinfo);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
/*
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
* array_position_common
* Common code for array_position and array_position_start
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
*
* These are separate wrappers for the sake of opr_sanity regression test.
* They are not strict so we have to test for null inputs explicitly.
*/
static Datum
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
array_position_common(FunctionCallInfo fcinfo)
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
{
ArrayType *array;
Oid collation = PG_GET_COLLATION();
Oid element_type;
Datum searched_element,
value;
bool isnull;
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
int position,
position_min;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
bool found = false;
TypeCacheEntry *typentry;
ArrayMetaState *my_extra;
bool null_search;
ArrayIterator array_iterator;
if (PG_ARGISNULL(0))
PG_RETURN_NULL();
array = PG_GETARG_ARRAYTYPE_P(0);
element_type = ARR_ELEMTYPE(array);
/*
* We refuse to search for elements in multi-dimensional arrays, since we
* have no good way to report the element's location in the array.
*/
if (ARR_NDIM(array) > 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("searching for elements in multidimensional arrays is not supported")));
if (PG_ARGISNULL(1))
{
Remove duplicated words in comments. David Rowley
2015-04-12 09:46:17 +02:00
/* fast return when the array doesn't have nulls */
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
if (!array_contains_nulls(array))
PG_RETURN_NULL();
searched_element = (Datum) 0;
null_search = true;
}
else
{
searched_element = PG_GETARG_DATUM(1);
null_search = false;
}
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position = (ARR_LBOUND(array))[0] - 1;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/* figure out where to start */
if (PG_NARGS() == 3)
{
if (PG_ARGISNULL(2))
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
Message improvements
2015-11-17 03:16:42 +01:00
errmsg("initial position must not be null")));
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position_min = PG_GETARG_INT32(2);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
else
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position_min = (ARR_LBOUND(array))[0];
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/*
* We arrange to look up type info for array_create_iterator only once per
* series of calls, assuming the element type doesn't change underneath
* us.
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(ArrayMetaState));
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
my_extra->element_type = ~element_type;
}
if (my_extra->element_type != element_type)
{
get_typlenbyvalalign(element_type,
&my_extra->typlen,
&my_extra->typbyval,
&my_extra->typalign);
typentry = lookup_type_cache(element_type, TYPECACHE_EQ_OPR_FINFO);
if (!OidIsValid(typentry->eq_opr_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
errmsg("could not identify an equality operator for type %s",
format_type_be(element_type))));
my_extra->element_type = element_type;
Fix crasher bug in array_position(s) array_position and its cousin array_positions were caching the element type equality function's FmgrInfo without being careful enough to put it in a long-lived context. This is obviously broken but it didn't matter in most cases; only when using arrays of records (involving record_eq) it becomes a problem. The fix is to ensure that the type's equality function's FmgrInfo is cached in the array_position's flinfo->fn_mcxt rather than the current memory context. Apart from record types, the only other case that seems complex enough to possibly cause the same problem are range types. I didn't find a way to reproduce the problem with those, so I only include the test case submitted with the bug report as regression test. Bug report and patch: Junseok Yang Discussion: https://postgr.es/m/CAE+byMupUURYiZ6bKYgMZb9pgV1CYAijJGqWj-90W=nS7uEOeA@mail.gmail.com Backpatch to 9.5, where array_position appeared.
2016-12-09 16:42:17 +01:00
fmgr_info_cxt(typentry->eq_opr_finfo.fn_oid, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
/* Examine each array element until we find a match. */
array_iterator = array_create_iterator(array, 0, my_extra);
while (array_iterate(array_iterator, &value, &isnull))
{
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position++;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/* skip initial elements if caller requested so */
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
if (position < position_min)
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
continue;
/*
* Can't look at the array element's value if it's null; but if we
* search for null, we have a hit and are done.
*/
if (isnull || null_search)
{
if (isnull && null_search)
{
found = true;
break;
}
else
continue;
}
/* not nulls, so run the operator */
if (DatumGetBool(FunctionCall2Coll(&my_extra->proc, collation,
searched_element, value)))
{
found = true;
break;
}
}
array_free_iterator(array_iterator);
/* Avoid leaking memory when handed toasted input */
PG_FREE_IF_COPY(array, 0);
if (!found)
PG_RETURN_NULL();
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
PG_RETURN_INT32(position);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
/*-----------------------------------------------------------------------------
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
* array_positions :
* return an array of positions of a value in an array.
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
*
* IS NOT DISTINCT FROM semantics are used for comparisons. Returns NULL when
* the input array is NULL. When the value is not found in the array, returns
* an empty array.
*
* This is not strict so we have to test for null inputs explicitly.
*-----------------------------------------------------------------------------
*/
Datum
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
array_positions(PG_FUNCTION_ARGS)
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
{
ArrayType *array;
Oid collation = PG_GET_COLLATION();
Oid element_type;
Datum searched_element,
value;
bool isnull;
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
int position;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
TypeCacheEntry *typentry;
ArrayMetaState *my_extra;
bool null_search;
ArrayIterator array_iterator;
ArrayBuildState *astate = NULL;
if (PG_ARGISNULL(0))
PG_RETURN_NULL();
array = PG_GETARG_ARRAYTYPE_P(0);
element_type = ARR_ELEMTYPE(array);
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position = (ARR_LBOUND(array))[0] - 1;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/*
* We refuse to search for elements in multi-dimensional arrays, since we
* have no good way to report the element's location in the array.
*/
if (ARR_NDIM(array) > 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("searching for elements in multidimensional arrays is not supported")));
astate = initArrayResult(INT4OID, CurrentMemoryContext, false);
if (PG_ARGISNULL(1))
{
Remove duplicated words in comments. David Rowley
2015-04-12 09:46:17 +02:00
/* fast return when the array doesn't have nulls */
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
if (!array_contains_nulls(array))
PG_RETURN_DATUM(makeArrayResult(astate, CurrentMemoryContext));
searched_element = (Datum) 0;
null_search = true;
}
else
{
searched_element = PG_GETARG_DATUM(1);
null_search = false;
}
/*
* We arrange to look up type info for array_create_iterator only once per
* series of calls, assuming the element type doesn't change underneath
* us.
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(ArrayMetaState));
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
my_extra->element_type = ~element_type;
}
if (my_extra->element_type != element_type)
{
get_typlenbyvalalign(element_type,
&my_extra->typlen,
&my_extra->typbyval,
&my_extra->typalign);
typentry = lookup_type_cache(element_type, TYPECACHE_EQ_OPR_FINFO);
if (!OidIsValid(typentry->eq_opr_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
errmsg("could not identify an equality operator for type %s",
format_type_be(element_type))));
my_extra->element_type = element_type;
Fix crasher bug in array_position(s) array_position and its cousin array_positions were caching the element type equality function's FmgrInfo without being careful enough to put it in a long-lived context. This is obviously broken but it didn't matter in most cases; only when using arrays of records (involving record_eq) it becomes a problem. The fix is to ensure that the type's equality function's FmgrInfo is cached in the array_position's flinfo->fn_mcxt rather than the current memory context. Apart from record types, the only other case that seems complex enough to possibly cause the same problem are range types. I didn't find a way to reproduce the problem with those, so I only include the test case submitted with the bug report as regression test. Bug report and patch: Junseok Yang Discussion: https://postgr.es/m/CAE+byMupUURYiZ6bKYgMZb9pgV1CYAijJGqWj-90W=nS7uEOeA@mail.gmail.com Backpatch to 9.5, where array_position appeared.
2016-12-09 16:42:17 +01:00
fmgr_info_cxt(typentry->eq_opr_finfo.fn_oid, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
}
/*
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
* Accumulate each array position iff the element matches the given
* element.
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
*/
array_iterator = array_create_iterator(array, 0, my_extra);
while (array_iterate(array_iterator, &value, &isnull))
{
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
position += 1;
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
/*
* Can't look at the array element's value if it's null; but if we
* search for null, we have a hit.
*/
if (isnull || null_search)
{
if (isnull && null_search)
astate =
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
accumArrayResult(astate, Int32GetDatum(position), false,
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
INT4OID, CurrentMemoryContext);
continue;
}
/* not nulls, so run the operator */
if (DatumGetBool(FunctionCall2Coll(&my_extra->proc, collation,
searched_element, value)))
astate =
Change array_offset to return subscripts, not offsets ... and rename it and its sibling array_offsets to array_position and array_positions, to account for the changed behavior. Having the functions return subscripts better matches existing practice, and is better suited to using the result value as a subscript into the array directly. For one-based arrays, the new definition is identical to what was originally committed. (We use the term "subscript" in the documentation, which is what we use whenever we talk about arrays; but the functions themselves are named using the word "position" to match the standard-defined POSITION() functions.) Author: Pavel Stěhule Behavioral problem noted by Dean Rasheed.
2015-03-30 21:13:21 +02:00
accumArrayResult(astate, Int32GetDatum(position), false,
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
INT4OID, CurrentMemoryContext);
}
array_free_iterator(array_iterator);
/* Avoid leaking memory when handed toasted input */
PG_FREE_IF_COPY(array, 0);
PG_RETURN_DATUM(makeArrayResult(astate, CurrentMemoryContext));
}
Add array_sample() and array_shuffle() functions. These are useful in Monte Carlo applications. Martin Kalcher, reviewed/adjusted by Daniel Gustafsson and myself Discussion: https://postgr.es/m/9d160a44-7675-51e8-60cf-6d64b76db831@aboutsource.net
2023-04-07 17:47:07 +02:00
/*
* array_shuffle_n
* Return a copy of array with n randomly chosen items.
*
* The number of items must not exceed the size of the first dimension of the
* array. We preserve the first dimension's lower bound if keep_lb,
* else it's set to 1. Lower-order dimensions are preserved in any case.
*
* NOTE: it would be cleaner to look up the elmlen/elmbval/elmalign info
* from the system catalogs, given only the elmtyp. However, the caller is
* in a better position to cache this info across multiple calls.
*/
static ArrayType *
array_shuffle_n(ArrayType *array, int n, bool keep_lb,
Oid elmtyp, TypeCacheEntry *typentry)
{
ArrayType *result;
int ndim,
*dims,
*lbs,
nelm,
nitem,
rdims[MAXDIM],
rlbs[MAXDIM];
int16 elmlen;
bool elmbyval;
char elmalign;
Datum *elms,
*ielms;
bool *nuls,
*inuls;
ndim = ARR_NDIM(array);
dims = ARR_DIMS(array);
lbs = ARR_LBOUND(array);
elmlen = typentry->typlen;
elmbyval = typentry->typbyval;
elmalign = typentry->typalign;
/* If the target array is empty, exit fast */
if (ndim < 1 || dims[0] < 1 || n < 1)
return construct_empty_array(elmtyp);
deconstruct_array(array, elmtyp, elmlen, elmbyval, elmalign,
&elms, &nuls, &nelm);
nitem = dims[0]; /* total number of items */
nelm /= nitem; /* number of elements per item */
Assert(n <= nitem); /* else it's caller error */
/*
* Shuffle array using Fisher-Yates algorithm. Scan the array and swap
* current item (nelm datums starting at ielms) with a randomly chosen
* later item (nelm datums starting at jelms) in each iteration. We can
* stop once we've done n iterations; then first n items are the result.
*/
ielms = elms;
inuls = nuls;
for (int i = 0; i < n; i++)
{
int j = (int) pg_prng_uint64_range(&pg_global_prng_state, i, nitem - 1) * nelm;
Datum *jelms = elms + j;
bool *jnuls = nuls + j;
/* Swap i'th and j'th items; advance ielms/inuls to next item */
for (int k = 0; k < nelm; k++)
{
Datum elm = *ielms;
bool nul = *inuls;
*ielms++ = *jelms;
*inuls++ = *jnuls;
*jelms++ = elm;
*jnuls++ = nul;
}
}
/* Set up dimensions of the result */
memcpy(rdims, dims, ndim * sizeof(int));
memcpy(rlbs, lbs, ndim * sizeof(int));
rdims[0] = n;
if (!keep_lb)
rlbs[0] = 1;
result = construct_md_array(elms, nuls, ndim, rdims, rlbs,
elmtyp, elmlen, elmbyval, elmalign);
pfree(elms);
pfree(nuls);
return result;
}
/*
* array_shuffle
*
* Returns an array with the same dimensions as the input array, with its
* first-dimension elements in random order.
*/
Datum
array_shuffle(PG_FUNCTION_ARGS)
{
ArrayType *array = PG_GETARG_ARRAYTYPE_P(0);
ArrayType *result;
Oid elmtyp;
TypeCacheEntry *typentry;
/*
* There is no point in shuffling empty arrays or arrays with less than
* two items.
*/
if (ARR_NDIM(array) < 1 || ARR_DIMS(array)[0] < 2)
PG_RETURN_ARRAYTYPE_P(array);
elmtyp = ARR_ELEMTYPE(array);
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL || typentry->type_id != elmtyp)
{
typentry = lookup_type_cache(elmtyp, 0);
fcinfo->flinfo->fn_extra = (void *) typentry;
}
result = array_shuffle_n(array, ARR_DIMS(array)[0], true, elmtyp, typentry);
PG_RETURN_ARRAYTYPE_P(result);
}
/*
* array_sample
*
* Returns an array of n randomly chosen first-dimension elements
* from the input array.
*/
Datum
array_sample(PG_FUNCTION_ARGS)
{
ArrayType *array = PG_GETARG_ARRAYTYPE_P(0);
int n = PG_GETARG_INT32(1);
ArrayType *result;
Oid elmtyp;
TypeCacheEntry *typentry;
int nitem;
nitem = (ARR_NDIM(array) < 1) ? 0 : ARR_DIMS(array)[0];
if (n < 0 || n > nitem)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("sample size must be between 0 and %d", nitem)));
elmtyp = ARR_ELEMTYPE(array);
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL || typentry->type_id != elmtyp)
{
typentry = lookup_type_cache(elmtyp, 0);
fcinfo->flinfo->fn_extra = (void *) typentry;
}
result = array_shuffle_n(array, n, false, elmtyp, typentry);
PG_RETURN_ARRAYTYPE_P(result);
}