rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/utils/adt/arrayfuncs.c

6608 lines
172 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* arrayfuncs.c
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* Support functions for arrays.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Update copyrights for 2020 Backpatch-through: update all files in master, backpatch legal files through 9.4
2020-01-01 18:21:45 +01:00
* Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/utils/adt/arrayfuncs.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*-------------------------------------------------------------------------
*/
Fix portability problems recently exposed by regression tests on Alphas. 1. Distinguish cases where a Datum representing a tuple datatype is an OID from cases where it is a pointer to TupleTableSlot, and make sure we use the right typlen in each case. 2. Make fetchatt() and related code support 8-byte by-value datatypes on machines where Datum is 8 bytes. Centralize knowledge of the available by-value datatype sizes in two macros in tupmacs.h, so that this will be easier if we ever have to do it again.
2000-12-28 00:59:14 +01:00
#include "postgres.h"
At this rate, maybe next year sometime I'll get this done... Goals: reduce the difficulty of porting from platform to platform, release to release, but moving as much as possible into config.h
1996-08-27 09:32:33 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include <ctype.h>
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
#include <math.h>
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Split tuple struct defs from htup.h to htup_details.h This reduces unnecessary exposure of other headers through htup.h, which is very widely included by many files. I have chosen to move the function prototypes to the new file as well, because that means htup.h no longer needs to include tupdesc.h. In itself this doesn't have much effect in indirect inclusion of tupdesc.h throughout the tree, because it's also required by execnodes.h; but it's something to explore in the future, and it seemed best to do the htup.h change now while I'm busy with it.
2012-08-30 22:15:44 +02:00
#include "access/htup_details.h"
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
#include "catalog/pg_type.h"
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
#include "funcapi.h"
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
#include "libpq/pqformat.h"
Build out the planner support function infrastructure. Add support function requests for estimating the selectivity, cost, and number of result rows (if a SRF) of the target function. The lack of a way to estimate selectivity of a boolean-returning function in WHERE has been a recognized deficiency of the planner since Berkeley days. This commit finally fixes it. In addition, non-constant estimates of cost and number of output rows are now possible. We still fall back to looking at procost and prorows if the support function doesn't service the request, of course. To make concrete use of the possibility of estimating output rowcount for SRFs, this commit adds support functions for array_unnest(anyarray) and the integer variants of generate_series; the lack of plausible rowcount estimates for those, even when it's obvious to a human, has been a repeated subject of complaints. Obviously, much more could now be done in this line, but I'm mostly just trying to get the infrastructure in place. Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:32:23 +01:00
#include "nodes/nodeFuncs.h"
#include "nodes/supportnodes.h"
#include "optimizer/optimizer.h"
Modify additional power 2 calculations to use new helper functions 2nd pass of modifying various places which obtain the next power of 2 of a number and make them use the new functions added in f0705bb62. In passing, also modify num_combinations(). This can be implemented using simple bitshifting rather than looping. Reviewed-by: John Naylor Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org
2020-04-08 08:29:51 +02:00
#include "port/pg_bitutils.h"
Clean up the #include mess a little. walsender.h should depend on xlog.h, not vice versa. (Actually, the inclusion was circular until a couple hours ago, which was even sillier; but Bruce broke it in the expedient rather than logically correct direction.) Because of that poor decision, plus blind application of pgrminclude, we had a situation where half the system was depending on xlog.h to include such unrelated stuff as array.h and guc.h. Clean up the header inclusion, and manually revert a lot of what pgrminclude had done so things build again. This episode reinforces my feeling that pgrminclude should not be run without adult supervision. Inclusion changes in header files in particular need to be reviewed with great care. More generally, it'd be good if we had a clearer notion of module layering to dictate which headers can sanely include which others ... but that's a big task for another day.
2011-09-04 07:13:16 +02:00
#include "utils/array.h"
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
#include "utils/arrayaccess.h"
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
#include "utils/builtins.h"
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
#include "utils/datum.h"
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
#include "utils/lsyscache.h"
Sort reference of include files, "A" - "F".
2006-07-11 18:35:33 +02:00
#include "utils/memutils.h"
Build out the planner support function infrastructure. Add support function requests for estimating the selectivity, cost, and number of result rows (if a SRF) of the target function. The lack of a way to estimate selectivity of a boolean-returning function in WHERE has been a recognized deficiency of the planner since Berkeley days. This commit finally fixes it. In addition, non-constant estimates of cost and number of output rows are now possible. We still fall back to looking at procost and prorows if the support function doesn't service the request, of course. To make concrete use of the possibility of estimating output rowcount for SRFs, this commit adds support functions for array_unnest(anyarray) and the integer variants of generate_series; the lack of plausible rowcount estimates for those, even when it's obvious to a human, has been a repeated subject of complaints. Obviously, much more could now be done in this line, but I'm mostly just trying to get the infrastructure in place. Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:32:23 +01:00
#include "utils/selfuncs.h"
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
#include "utils/typcache.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* GUC parameter
Add debugging statements.
1997-07-29 17:51:33 +02:00
*/
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
bool Array_nulls = true;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
* Local definitions
*/
#define ASSGN "="
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
#define AARR_FREE_IF_COPY(array,n) \
do { \
if (!VARATT_IS_EXPANDED_HEADER(array)) \
PG_FREE_IF_COPY(array, n); \
} while (0)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
typedef enum
{
ARRAY_NO_LEVEL,
ARRAY_LEVEL_STARTED,
ARRAY_ELEM_STARTED,
ARRAY_ELEM_COMPLETED,
ARRAY_QUOTED_ELEM_STARTED,
ARRAY_QUOTED_ELEM_COMPLETED,
ARRAY_ELEM_DELIMITED,
ARRAY_LEVEL_COMPLETED,
ARRAY_LEVEL_DELIMITED
} ArrayParseState;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
Add FOREACH IN ARRAY looping to plpgsql. (I'm not entirely sure that we've finished bikeshedding the syntax details, but the functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost and Tom Lane
2011-02-16 07:52:04 +01:00
/* Working state for array_iterate() */
typedef struct ArrayIteratorData
{
/* basic info about the array, set up during array_create_iterator() */
ArrayType *arr; /* array we're iterating through */
bits8 *nullbitmap; /* its null bitmap, if any */
int nitems; /* total number of elements in array */
int16 typlen; /* element type's length */
bool typbyval; /* element type's byval property */
char typalign; /* element type's align property */
/* information about the requested slice size */
int slice_ndim; /* slice dimension, or 0 if not slicing */
int slice_len; /* number of elements per slice */
int *slice_dims; /* slice dims array */
int *slice_lbound; /* slice lbound array */
Datum *slice_values; /* workspace of length slice_len */
bool *slice_nulls; /* workspace of length slice_len */
/* current position information, updated on each iteration */
char *data_ptr; /* our current position in the array */
int current_item; /* the item # we're at in the array */
Initial pgindent run with pg_bsd_indent version 2.0. The new indent version includes numerous fixes thanks to Piotr Stefaniak. The main changes visible in this commit are: * Nicer formatting of function-pointer declarations. * No longer unexpectedly removes spaces in expressions using casts, sizeof, or offsetof. * No longer wants to add a space in "struct structname *varname", as well as some similar cases for const- or volatile-qualified pointers. * Declarations using PG_USED_FOR_ASSERTS_ONLY are formatted more nicely. * Fixes bug where comments following declarations were sometimes placed with no space separating them from the code. * Fixes some odd decisions for comments following case labels. * Fixes some cases where comments following code were indented to less than the expected column 33. On the less good side, it now tends to put more whitespace around typedef names that are not listed in typedefs.list. This might encourage us to put more effort into typedef name collection; it's not really a bug in indent itself. There are more changes coming after this round, having to do with comment indentation and alignment of lines appearing within parentheses. I wanted to limit the size of the diffs to something that could be reviewed without one's eyes completely glazing over, so it seemed better to split up the changes as much as practical. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 20:39:04 +02:00
} ArrayIteratorData;
Add FOREACH IN ARRAY looping to plpgsql. (I'm not entirely sure that we've finished bikeshedding the syntax details, but the functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost and Tom Lane
2011-02-16 07:52:04 +01:00
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
static bool array_isspace(char ch);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static int ArrayCount(const char *str, int *dim, char typdelim);
static void ReadArrayStr(char *arrayStr, const char *origStr,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int nitems, int ndim, int *dim,
FmgrInfo *inputproc, Oid typioparam, int32 typmod,
char typdelim,
int typlen, bool typbyval, char typalign,
Datum *values, bool *nulls,
bool *hasnulls, int32 *nbytes);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static void ReadArrayBinary(StringInfo buf, int nitems,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
FmgrInfo *receiveproc, Oid typioparam, int32 typmod,
int typlen, bool typbyval, char typalign,
Datum *values, bool *nulls,
bool *hasnulls, int32 *nbytes);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
static Datum array_get_element_expanded(Datum arraydatum,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int nSubscripts, int *indx,
int arraytyplen,
int elmlen, bool elmbyval, char elmalign,
bool *isNull);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
static Datum array_set_element_expanded(Datum arraydatum,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int nSubscripts, int *indx,
Datum dataValue, bool isNull,
int arraytyplen,
int elmlen, bool elmbyval, char elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static bool array_get_isnull(const bits8 *nullbitmap, int offset);
static void array_set_isnull(bits8 *nullbitmap, int offset, bool isNull);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
static Datum ArrayCast(char *value, bool byval, int len);
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
static int ArrayCastAndSet(Datum src,
int typlen, bool typbyval, char typalign,
char *dest);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static char *array_seek(char *ptr, int offset, bits8 *nullbitmap, int nitems,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int typlen, bool typbyval, char typalign);
static int array_nelems_size(char *ptr, int offset, bits8 *nullbitmap,
int nitems, int typlen, bool typbyval, char typalign);
static int array_copy(char *destptr, int nitems,
char *srcptr, int offset, bits8 *nullbitmap,
int typlen, bool typbyval, char typalign);
static int array_slice_size(char *arraydataptr, bits8 *arraynullsptr,
int ndim, int *dim, int *lb,
int *st, int *endp,
int typlen, bool typbyval, char typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static void array_extract_slice(ArrayType *newarray,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
int ndim, int *dim, int *lb,
char *arraydataptr, bits8 *arraynullsptr,
int *st, int *endp,
int typlen, bool typbyval, char typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static void array_insert_slice(ArrayType *destArray, ArrayType *origArray,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
ArrayType *srcArray,
int ndim, int *dim, int *lb,
int *st, int *endp,
int typlen, bool typbyval, char typalign);
pgindent run.
2003-08-04 02:43:34 +02:00
static int array_cmp(FunctionCallInfo fcinfo);
Fix inconsistencies and typos in the tree This is numbered take 8, and addresses again a set of issues with code comments, variable names and unreferenced variables. Author: Alexander Lakhin Discussion: https://postgr.es/m/b137b5eb-9c95-9c2f-586e-38aba7d59788@gmail.com
2019-07-29 05:28:30 +02:00
static ArrayType *create_array_envelope(int ndims, int *dimv, int *lbsv, int nbytes,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
Oid elmtype, int dataoffset);
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
static ArrayType *array_fill_internal(ArrayType *dims, ArrayType *lbs,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
Datum value, bool isnull, Oid elmtype,
FunctionCallInfo fcinfo);
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
static ArrayType *array_replace_internal(ArrayType *array,
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
Datum search, bool search_isnull,
Datum replace, bool replace_isnull,
bool remove, Oid collation,
FunctionCallInfo fcinfo);
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
static int width_bucket_array_float8(Datum operand, ArrayType *thresholds);
Phase 2 pgindent run for v12. Switch to 2.1 version of pg_bsd_indent. This formats multiline function declarations "correctly", that is with additional lines of parameter declarations indented to match where the first line's left parenthesis is. Discussion: https://postgr.es/m/CAEepm=0P3FeTXRcU5B2W3jv3PgRVZ-kGUXLGfd42FFhUROO3ug@mail.gmail.com
2019-05-22 19:04:48 +02:00
static int width_bucket_array_fixed(Datum operand,
ArrayType *thresholds,
Oid collation,
TypeCacheEntry *typentry);
static int width_bucket_array_variable(Datum operand,
ArrayType *thresholds,
Oid collation,
TypeCacheEntry *typentry);
Represent type-specific length coercion functions as pg_cast entries, eliminating the former hard-wired convention about their names. Allow pg_cast entries to represent both type coercion and length coercion in a single step --- this is represented by a function that takes an extra typmod argument, just like a length coercion function. This nicely merges the type and length coercion mechanisms into something at least a little cleaner than we had before. Make use of the single- coercion-step behavior to fix integer-to-bit coercion so that coercing to bit(n) yields the rightmost n bits of the integer instead of the leftmost n bits. This should fix recurrent complaints about the odd behavior of this coercion. Clean up the documentation of the bit string functions, and try to put it where people might actually find it. Also, get rid of the unreliable heuristics in ruleutils.c about whether to display nested coercion steps; instead require parse_coerce.c to label them properly in the first place.
2004-06-16 03:27:00 +02:00
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* array_in :
* converts an array from the external format in "string" to
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
* its internal format.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* return value :
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* the internal representation of the input array
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Datum
array_in(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
char *string = PG_GETARG_CSTRING(0); /* external form */
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
Oid element_type = PG_GETARG_OID(1); /* type of an array
* element */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
int32 typmod = PG_GETARG_INT32(2); /* typmod for array elements */
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int typlen;
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
bool typbyval;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
char typalign;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
char typdelim;
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
Oid typioparam;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
char *string_save,
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
*p;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
Thank god for searchable mail archives. Patch by: wieck@sapserv.debis.de (Jan Wieck) One of the design rules of PostgreSQL is extensibility. And to follow this rule means (at least for me) that there should not only be a builtin PL. Instead I would prefer a defined interface for PL implemetations.
1998-01-15 20:46:37 +01:00
nitems;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Datum *dataPtr;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nullsPtr;
bool hasnulls;
int32 nbytes;
int32 dataoffset;
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
ArrayType *retval;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int ndim,
dim[MAXDIM],
lBound[MAXDIM];
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *my_extra;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* We arrange to look up info about element type, including its input
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* conversion proc, only once per series of calls, assuming the element
* type doesn't change underneath us.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
sizeof(ArrayMetaState));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
array_in() and array_recv() need to be more paranoid about validating their OID parameter. It was possible to crash the backend with select array_in('{123}',0,0); because that would bypass the needed step of initializing the workspace. These seem to be the only two places with a problem, though (record_in and record_recv don't have the issue, and the other array functions aren't depending on user-supplied input). Back-patch as far as 7.4; 7.3 does not have the bug.
2005-08-15 21:40:20 +02:00
my_extra->element_type = ~element_type;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
if (my_extra->element_type != element_type)
{
pgindent run.
2003-08-04 02:43:34 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Get info about element type, including its input conversion proc
pgindent run.
2003-08-04 02:43:34 +02:00
*/
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
get_type_io_data(element_type, IOFunc_input,
&my_extra->typlen, &my_extra->typbyval,
&my_extra->typalign, &my_extra->typdelim,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
&my_extra->typioparam, &my_extra->typiofunc);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
fmgr_info_cxt(my_extra->typiofunc, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
my_extra->element_type = element_type;
}
typlen = my_extra->typlen;
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
typdelim = my_extra->typdelim;
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
typioparam = my_extra->typioparam;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
/* Make a modifiable copy of the input */
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
string_save = pstrdup(string);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
/*
* If the input string starts with dimension info, read and use that.
* Otherwise, we require the input to be in curly-brace style, and we
* prescan the input to determine dimensions.
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Dimension info takes the form of one or more [n] or [m:n] items. The
* outer loop iterates once per dimension item.
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
*/
p = string_save;
ndim = 0;
for (;;)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
char *q;
int ub;
/*
* Note: we currently allow whitespace between, but not within,
* dimension items.
*/
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
while (array_isspace(*p))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
p++;
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
if (*p != '[')
break; /* no more dimension items */
p++;
if (ndim >= MAXDIM)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
Fix wrong error reports in 'number of array dimensions exceeds the maximum allowed' messages, that have reported one-less dimensions. Alexey Klyukin
2011-02-01 07:21:32 +01:00
ndim + 1, MAXDIM)));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
for (q = p; isdigit((unsigned char) *q) || (*q == '-') || (*q == '+'); q++)
/* skip */ ;
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
if (q == p) /* no digits? */
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("\"[\" must introduce explicitly-specified array dimensions.")));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
if (*q == ':')
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
/* [m:n] format */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*q = '\0';
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
lBound[ndim] = atoi(p);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
p = q + 1;
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
for (q = p; isdigit((unsigned char) *q) || (*q == '-') || (*q == '+'); q++)
/* skip */ ;
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
if (q == p) /* no digits? */
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Missing array dimension value.")));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
{
/* [n] format */
lBound[ndim] = 1;
}
if (*q != ']')
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Missing \"%s\" after array dimensions.",
"]")));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
*q = '\0';
ub = atoi(p);
p = q + 1;
if (ub < lBound[ndim])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("upper bound cannot be less than lower bound")));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
dim[ndim] = ub - lBound[ndim] + 1;
ndim++;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
if (ndim == 0)
{
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
/* No array dimensions, so intuit dimensions from brace structure */
if (*p != '{')
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Array value must start with \"{\" or dimension information.")));
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
ndim = ArrayCount(p, dim, typdelim);
for (i = 0; i < ndim; i++)
lBound[i] = 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
int ndim_braces,
dim_braces[MAXDIM];
Clean up array-dimensions parser a bit. This code still needs a lot of love, however ...
2000-11-15 00:28:13 +01:00
/* If array dimensions are given, expect '=' operator */
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
if (strncmp(p, ASSGN, strlen(ASSGN)) != 0)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Missing \"%s\" after array dimensions.",
ASSGN)));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
p += strlen(ASSGN);
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
while (array_isspace(*p))
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
p++;
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* intuit dimensions from brace structure -- it better match what we
* were given
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
*/
if (*p != '{')
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Array contents must start with \"{\".")));
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
ndim_braces = ArrayCount(p, dim_braces, typdelim);
if (ndim_braces != ndim)
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Specified array dimensions do not match array contents.")));
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
for (i = 0; i < ndim; ++i)
{
if (dim[i] != dim_braces[i])
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", string),
errdetail("Specified array dimensions do not match array contents.")));
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Add debugging statements.
1997-07-29 17:51:33 +02:00
#ifdef ARRAYDEBUG
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
printf("array_in- ndim %d (", ndim);
for (i = 0; i < ndim; i++)
{
printf(" %d", dim[i]);
};
printf(") for %s\n", string);
Add debugging statements.
1997-07-29 17:51:33 +02:00
#endif
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* This checks for overflow of the array dimensions */
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
nitems = ArrayGetNItems(ndim, dim);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Empty array? */
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (nitems == 0)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
PG_RETURN_ARRAYTYPE_P(construct_empty_array(element_type));
dataPtr = (Datum *) palloc(nitems * sizeof(Datum));
nullsPtr = (bool *) palloc(nitems * sizeof(bool));
ReadArrayStr(p, string,
nitems, ndim, dim,
&my_extra->proc, typioparam, typmod,
typdelim,
typlen, typbyval, typalign,
dataPtr, nullsPtr,
&hasnulls, &nbytes);
if (hasnulls)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dataoffset = ARR_OVERHEAD_WITHNULLS(ndim, nitems);
nbytes += dataoffset;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes += ARR_OVERHEAD_NONULLS(ndim);
}
Fix several datatype input functions that were allowing unused bytes in their results to contain uninitialized, unpredictable values. While this was okay as far as the datatypes themselves were concerned, it's a problem for the parser because occurrences of the "same" literal might not be recognized as equal by datumIsEqual (and hence not by equal()). It seems sufficient to fix this in the input functions since the only critical use of equal() is in the parser's comparisons of ORDER BY and DISTINCT expressions. Per a trouble report from Marc Cousin. Patch all the way back. Interestingly, array_in did not have the bug before 8.2, which may explain why the issue went unnoticed for so long.
2008-04-12 00:52:05 +02:00
retval = (ArrayType *) palloc0(nbytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(retval, nbytes);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
retval->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
retval->dataoffset = dataoffset;
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
binary migration: pg_migrator Add comments about places where system oids have to be preserved for binary migration.
2009-12-19 01:47:57 +01:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* This comes from the array's pg_type.typelem (which points to the base
* data type's pg_type.oid) and stores system oids in user tables. This
* oid must be preserved by binary upgrades.
binary migration: pg_migrator Add comments about places where system oids have to be preserved for binary migration.
2009-12-19 01:47:57 +01:00
*/
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
retval->elemtype = element_type;
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
memcpy(ARR_DIMS(retval), dim, ndim * sizeof(int));
memcpy(ARR_LBOUND(retval), lBound, ndim * sizeof(int));
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
CopyArrayEls(retval,
dataPtr, nullsPtr, nitems,
typlen, typbyval, typalign,
true);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
pfree(dataPtr);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
pfree(nullsPtr);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pfree(string_save);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
PG_RETURN_ARRAYTYPE_P(retval);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
/*
* array_isspace() --- a non-locale-dependent isspace()
*
* We used to use isspace() for parsing array values, but that has
* undesirable results: an array value might be silently interpreted
* differently depending on the locale setting. Now we just hard-wire
* the traditional ASCII definition of isspace().
*/
static bool
array_isspace(char ch)
{
if (ch == ' ' ||
ch == '\t' ||
ch == '\n' ||
ch == '\r' ||
ch == '\v' ||
ch == '\f')
return true;
return false;
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* ArrayCount
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Determines the dimensions for an array string.
*
* Returns number of dimensions as function result. The axis lengths are
* returned in dim[], which must be of size MAXDIM.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static int
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ArrayCount(const char *str, int *dim, char typdelim)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
int nest_level = 0,
i;
int ndim = 1,
temp[MAXDIM],
nelems[MAXDIM],
nelems_last[MAXDIM];
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
bool in_quotes = false;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
bool eoArray = false;
bool empty_array = true;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
const char *ptr;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
ArrayParseState parse_state = ARRAY_NO_LEVEL;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (i = 0; i < MAXDIM; ++i)
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
{
arrays: tighten checks for multi-dimensional input Previously an input array string that started with a single-element array dimension would then later accept a multi-dimensional segment. BACKWARD INCOMPATIBILITY
2014-02-01 16:49:17 +01:00
temp[i] = dim[i] = nelems_last[i] = 0;
nelems[i] = 1;
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
ptr = str;
while (!eoArray)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
bool itemdone = false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
while (!itemdone)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
if (parse_state == ARRAY_ELEM_STARTED ||
parse_state == ARRAY_QUOTED_ELEM_STARTED)
empty_array = false;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
switch (*ptr)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case '\0':
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
/* Signal a premature end of the string */
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unexpected end of input.")));
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
case '\\':
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* An escape must be after a level start, after an element
* start, or after an element delimiter. In any case we
* now must be past an element start.
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
*/
if (parse_state != ARRAY_LEVEL_STARTED &&
parse_state != ARRAY_ELEM_STARTED &&
parse_state != ARRAY_QUOTED_ELEM_STARTED &&
parse_state != ARRAY_ELEM_DELIMITED)
ereport(ERROR,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("malformed array literal: \"%s\"", str),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errdetail("Unexpected \"%c\" character.",
'\\')));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
if (parse_state != ARRAY_QUOTED_ELEM_STARTED)
parse_state = ARRAY_ELEM_STARTED;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
/* skip the escaped character */
if (*(ptr + 1))
ptr++;
else
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("malformed array literal: \"%s\"", str),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errdetail("Unexpected end of input.")));
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
break;
Remove unnecessary escaping in C character literals '\"' is more commonly written simply as '"'.
2015-12-23 04:43:46 +01:00
case '"':
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
/*
* A quote must be after a level start, after a quoted
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* element start, or after an element delimiter. In any
* case we now must be past an element start.
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
*/
if (parse_state != ARRAY_LEVEL_STARTED &&
parse_state != ARRAY_QUOTED_ELEM_STARTED &&
parse_state != ARRAY_ELEM_DELIMITED)
ereport(ERROR,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("malformed array literal: \"%s\"", str),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errdetail("Unexpected array element.")));
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
in_quotes = !in_quotes;
if (in_quotes)
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
parse_state = ARRAY_QUOTED_ELEM_STARTED;
else
parse_state = ARRAY_QUOTED_ELEM_COMPLETED;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
case '{':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (!in_quotes)
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
{
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* A left brace can occur if no nesting has occurred
* yet, after a level start, or after a level
* delimiter.
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
*/
if (parse_state != ARRAY_NO_LEVEL &&
parse_state != ARRAY_LEVEL_STARTED &&
parse_state != ARRAY_LEVEL_DELIMITED)
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unexpected \"%c\" character.",
'{')));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
parse_state = ARRAY_LEVEL_STARTED;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (nest_level >= MAXDIM)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
Fix wrong error reports in 'number of array dimensions exceeds the maximum allowed' messages, that have reported one-less dimensions. Alexey Klyukin
2011-02-01 07:21:32 +01:00
nest_level + 1, MAXDIM)));
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
temp[nest_level] = 0;
nest_level++;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (ndim < nest_level)
ndim = nest_level;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
}
break;
case '}':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (!in_quotes)
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
{
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* A right brace can occur after an element start, an
* element completion, a quoted element completion, or
* a level completion.
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
*/
if (parse_state != ARRAY_ELEM_STARTED &&
parse_state != ARRAY_ELEM_COMPLETED &&
parse_state != ARRAY_QUOTED_ELEM_COMPLETED &&
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
parse_state != ARRAY_LEVEL_COMPLETED &&
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
!(nest_level == 1 && parse_state == ARRAY_LEVEL_STARTED))
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unexpected \"%c\" character.",
'}')));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
parse_state = ARRAY_LEVEL_COMPLETED;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (nest_level == 0)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unmatched \"%c\" character.", '}')));
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
nest_level--;
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
arrays: tighten checks for multi-dimensional input Previously an input array string that started with a single-element array dimension would then later accept a multi-dimensional segment. BACKWARD INCOMPATIBILITY
2014-02-01 16:49:17 +01:00
if (nelems_last[nest_level] != 0 &&
nelems[nest_level] != nelems_last[nest_level])
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Multidimensional arrays must have "
"sub-arrays with matching "
"dimensions.")));
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
nelems_last[nest_level] = nelems[nest_level];
nelems[nest_level] = 1;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
if (nest_level == 0)
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
eoArray = itemdone = true;
else
{
/*
* We don't set itemdone here; see comments in
* ReadArrayStr
*/
temp[nest_level - 1]++;
}
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
}
break;
default:
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (!in_quotes)
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
{
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
if (*ptr == typdelim)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Delimiters can occur after an element start, an
* element completion, a quoted element
* completion, or a level completion.
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
*/
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
if (parse_state != ARRAY_ELEM_STARTED &&
parse_state != ARRAY_ELEM_COMPLETED &&
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
parse_state != ARRAY_QUOTED_ELEM_COMPLETED &&
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
parse_state != ARRAY_LEVEL_COMPLETED)
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unexpected \"%c\" character.",
typdelim)));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
if (parse_state == ARRAY_LEVEL_COMPLETED)
parse_state = ARRAY_LEVEL_DELIMITED;
else
parse_state = ARRAY_ELEM_DELIMITED;
itemdone = true;
nelems[nest_level - 1]++;
}
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
else if (!array_isspace(*ptr))
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
{
/*
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
* Other non-space characters must be after a
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* level start, after an element start, or after
* an element delimiter. In any case we now must
* be past an element start.
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
*/
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
if (parse_state != ARRAY_LEVEL_STARTED &&
parse_state != ARRAY_ELEM_STARTED &&
parse_state != ARRAY_ELEM_DELIMITED)
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"", str),
errdetail("Unexpected array element.")));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
parse_state = ARRAY_ELEM_STARTED;
}
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
}
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (!itemdone)
ptr++;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
temp[ndim - 1]++;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
ptr++;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
/* only whitespace is allowed after the closing brace */
while (*ptr)
{
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
if (!array_isspace(*ptr++))
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
ereport(ERROR,
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
errmsg("malformed array literal: \"%s\"", str),
errdetail("Junk after closing right brace.")));
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
}
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Further tightening of the array literal parser. Prevent junk from being accepted after the outer right brace. Per report from Markus Bertheau. Also add regression test cases for this change, and for previous recent array literal parser changes.
2004-08-28 21:31:29 +02:00
/* special case for an empty array */
if (empty_array)
return 0;
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (i = 0; i < ndim; ++i)
dim[i] = temp[i];
Renaming cleanup, no pgindent yet.
1998-09-01 05:29:17 +02:00
return ndim;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* ReadArrayStr :
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* parses the array string pointed to by "arrayStr" and converts the values
* to internal format. Unspecified elements are initialized to nulls.
* The array dimensions must already have been determined.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Inputs:
* arrayStr: the string to parse.
* CAUTION: the contents of "arrayStr" will be modified!
* origStr: the unmodified input string, used only in error messages.
* nitems: total number of array elements, as already determined.
* ndim: number of array dimensions
* dim[]: array axis lengths
* inputproc: type-specific input procedure for element datatype.
* typioparam, typmod: auxiliary values to pass to inputproc.
* typdelim: the value delimiter (type-specific).
* typlen, typbyval, typalign: storage parameters of element datatype.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Outputs:
* values[]: filled with converted data values.
* nulls[]: filled with is-null markers.
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 06:22:32 +02:00
* *hasnulls: set true iff there are any null elements.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* *nbytes: set to total size of data area needed (including alignment
* padding but not including array header overhead).
*
* Note that values[] and nulls[] are allocated by the caller, and must have
* nitems elements.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static void
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
ReadArrayStr(char *arrayStr,
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
const char *origStr,
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int nitems,
int ndim,
int *dim,
FmgrInfo *inputproc,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
Oid typioparam,
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int32 typmod,
char typdelim,
int typlen,
bool typbyval,
char typalign,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Datum *values,
bool *nulls,
bool *hasnulls,
int32 *nbytes)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
nest_level = 0;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
char *srcptr;
bool in_quotes = false;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
bool eoArray = false;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool hasnull;
int32 totbytes;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int indx[MAXDIM],
prod[MAXDIM];
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
mda_get_prod(ndim, dim, prod);
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
MemSet(indx, 0, sizeof(indx));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Initialize is-null markers to true */
memset(nulls, true, nitems * sizeof(bool));
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We have to remove " and \ characters to create a clean item value to
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* pass to the datatype input routine. We overwrite each item value
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* in-place within arrayStr to do this. srcptr is the current scan point,
* and dstptr is where we are copying to.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* We also want to suppress leading and trailing unquoted whitespace. We
* use the leadingspace flag to suppress leading space. Trailing space is
* tracked by using dstendptr to point to the last significant output
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* character.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* The error checking in this routine is mostly pro-forma, since we expect
Improve error messages for malformed array input strings. Make the error messages issued by array_in() uniformly follow the style ERROR: malformed array literal: "actual input string" DETAIL: specific complaint here and rewrite many of the specific complaints to be clearer. The immediate motivation for doing this is a complaint from Josh Berkus that json_to_record() produced an unintelligible error message when dealing with an array item, because it tries to feed the JSON-format array value to array_in(). Really it ought to be smart enough to perform JSON-to-Postgres array conversion, but that's a future feature not a bug fix. In the meantime, this change is something we agreed we could back-patch into 9.4, and it should help de-confuse things a bit.
2014-12-03 00:23:16 +01:00
* that ArrayCount() already validated the string. So we don't bother
* with errdetail messages.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*/
srcptr = arrayStr;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
while (!eoArray)
{
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
bool itemdone = false;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
bool leadingspace = true;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool hasquoting = false;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
char *itemstart;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
char *dstptr;
char *dstendptr;
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
i = -1;
itemstart = dstptr = dstendptr = srcptr;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
while (!itemdone)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
switch (*srcptr)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
case '\0':
/* Signal a premature end of the string */
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
errmsg("malformed array literal: \"%s\"",
origStr)));
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
break;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case '\\':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
/* Skip backslash, copy next character as-is. */
srcptr++;
if (*srcptr == '\0')
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"",
origStr)));
*dstptr++ = *srcptr++;
/* Treat the escaped character as non-whitespace */
leadingspace = false;
dstendptr = dstptr;
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
hasquoting = true; /* can't be a NULL marker */
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
break;
Remove unnecessary escaping in C character literals '\"' is more commonly written simply as '"'.
2015-12-23 04:43:46 +01:00
case '"':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
in_quotes = !in_quotes;
if (in_quotes)
leadingspace = false;
else
pgindent run.
2002-09-04 22:31:48 +02:00
{
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
/*
* Advance dstendptr when we exit in_quotes; this
* saves having to do it in all the other in_quotes
* cases.
*/
dstendptr = dstptr;
pgindent run.
2002-09-04 22:31:48 +02:00
}
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
hasquoting = true; /* can't be a NULL marker */
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
srcptr++;
break;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
case '{':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (!in_quotes)
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
{
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (nest_level >= ndim)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"",
origStr)));
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
nest_level++;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
indx[nest_level - 1] = 0;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
srcptr++;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
else
*dstptr++ = *srcptr++;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
case '}':
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (!in_quotes)
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
{
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (nest_level == 0)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
errmsg("malformed array literal: \"%s\"",
origStr)));
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
if (i == -1)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
i = ArrayGetOffset0(ndim, indx, prod);
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
indx[nest_level - 1] = 0;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
nest_level--;
if (nest_level == 0)
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
eoArray = itemdone = true;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
else
indx[nest_level - 1]++;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
srcptr++;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
}
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
else
*dstptr++ = *srcptr++;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
break;
default:
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
if (in_quotes)
*dstptr++ = *srcptr++;
else if (*srcptr == typdelim)
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
{
if (i == -1)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
i = ArrayGetOffset0(ndim, indx, prod);
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
itemdone = true;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
indx[ndim - 1]++;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
srcptr++;
}
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
else if (array_isspace(*srcptr))
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* If leading space, drop it immediately. Else, copy
* but don't advance dstendptr.
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*/
if (leadingspace)
srcptr++;
else
*dstptr++ = *srcptr++;
}
else
{
*dstptr++ = *srcptr++;
leadingspace = false;
dstendptr = dstptr;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
}
break;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
Assert(dstptr < srcptr);
*dstendptr = '\0';
Try to make array_in's behavior a tad less bizarre. Leading whitespace before a data item is now always skipped, rather than only sometimes. Backslashes not within double-quoted text are treated reasonably, as are multiple sequences of quoted text in a single data item. But it still seems rather prone to misbehavior if the input is not completely syntactically correct --- in particular, garbage following a right brace will be ignored.
2002-03-16 23:47:13 +01:00
if (i < 0 || i >= nitems)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_TEXT_REPRESENTATION),
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
errmsg("malformed array literal: \"%s\"",
origStr)));
Tighened up syntax checking of array input processing considerably. Junk that was previously allowed in odd places with odd results now causes an ERROR. Also changed behavior with respect to whitespace -- trailing whitespace is now ignored as well as leading whitespace (which has always been ignored). Documentation updated to reflect change in whitespace handling. Also some refactoring to what I believe is a more sensible order of several paragraphs.
2004-08-08 07:01:55 +02:00
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
if (Array_nulls && !hasquoting &&
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
pg_strcasecmp(itemstart, "NULL") == 0)
{
/* it's a NULL item */
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
values[i] = InputFunctionCall(inputproc, NULL,
typioparam, typmod);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nulls[i] = true;
}
else
{
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
values[i] = InputFunctionCall(inputproc, itemstart,
typioparam, typmod);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nulls[i] = false;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Check for nulls, compute total data space needed
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
hasnull = false;
totbytes = 0;
for (i = 0; i < nitems; i++)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nulls[i])
hasnull = true;
else
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* let's just make sure data is not toasted */
if (typlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM(values[i]));
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
totbytes = att_addlength_datum(totbytes, typlen, values[i]);
totbytes = att_align_nominal(totbytes, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* check for overflow of total request */
if (!AllocSizeIsValid(totbytes))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*hasnulls = hasnull;
Rewrite ReadArrayStr() to avoid O(N^2) behavior on large strings, and hopefully improve code clarity while at it. One intentional semantics change: a backslashed space will not be treated as removable trailing whitespace, as the prior coding would do. ISTM that if it wouldn't be considered removable leading whitespace, it shouldn't be stripped at the end either.
2004-09-27 03:39:02 +02:00
*nbytes = totbytes;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* Copy data into an array object from a temporary array of Datums.
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* array: array object (with header fields already filled in)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* values: array of Datums to be copied
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nulls: array of is-null flags (can be NULL if no nulls)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* nitems: number of Datums to be copied
* typbyval, typlen, typalign: info about element datatype
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 06:22:32 +02:00
* freedata: if true and element type is pass-by-ref, pfree data values
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* referenced by Datums after copying them.
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
*
* If the input data is of varlena type, the caller must have ensured that
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* the values are not toasted. (Doing it here doesn't work since the
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
* caller has already allocated space for the array...)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*/
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
void
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
CopyArrayEls(ArrayType *array,
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Datum *values,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nulls,
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int nitems,
int typlen,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
bool typbyval,
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
char typalign,
bool freedata)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char *p = ARR_DATA_PTR(array);
bits8 *bitmap = ARR_NULLBITMAP(array);
int bitval = 0;
int bitmask = 1;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
if (typbyval)
freedata = false;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (i = 0; i < nitems; i++)
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nulls && nulls[i])
{
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
if (!bitmap) /* shouldn't happen */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
elog(ERROR, "null array element where not supported");
/* bitmap bit stays 0 */
}
else
{
bitval |= bitmask;
p += ArrayCastAndSet(values[i], typlen, typbyval, typalign, p);
if (freedata)
pfree(DatumGetPointer(values[i]));
}
if (bitmap)
{
bitmask <<= 1;
if (bitmask == 0x100)
{
*bitmap++ = bitval;
bitval = 0;
bitmask = 1;
}
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (bitmap && bitmask != 1)
*bitmap = bitval;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* array_out :
* takes the internal representation of an array and returns a string
* containing the array in its external format.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Datum
array_out(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Oid element_type = AARR_ELEMTYPE(v);
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int typlen;
bool typbyval;
char typalign;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
char typdelim;
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
char *p,
*tmp,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*retval,
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
**values,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dims_str[(MAXDIM * 33) + 2];
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
/*
* 33 per dim since we assume 15 digits per number + ':' +'[]'
*
* +2 allows for assignment operator + trailing null
*/
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
bool *needquotes,
needdims = false;
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
size_t overall_length;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int nitems,
i,
j,
k,
indx[MAXDIM];
int ndim,
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
*dims,
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
*lb;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter iter;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *my_extra;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* We arrange to look up info about element type, including its output
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* conversion proc, only once per series of calls, assuming the element
* type doesn't change underneath us.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
sizeof(ArrayMetaState));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
my_extra->element_type = ~element_type;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
if (my_extra->element_type != element_type)
{
pgindent run.
2003-08-04 02:43:34 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Get info about element type, including its output conversion proc
pgindent run.
2003-08-04 02:43:34 +02:00
*/
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
get_type_io_data(element_type, IOFunc_output,
&my_extra->typlen, &my_extra->typbyval,
&my_extra->typalign, &my_extra->typdelim,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
&my_extra->typioparam, &my_extra->typiofunc);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
fmgr_info_cxt(my_extra->typiofunc, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
my_extra->element_type = element_type;
}
typlen = my_extra->typlen;
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
typdelim = my_extra->typdelim;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
ndim = AARR_NDIM(v);
dims = AARR_DIMS(v);
lb = AARR_LBOUND(v);
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
nitems = ArrayGetNItems(ndim, dims);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (nitems == 0)
{
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
retval = pstrdup("{}");
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
PG_RETURN_CSTRING(retval);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* we will need to add explicit dimensions if any dimension has a lower
* bound other than one
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
*/
for (i = 0; i < ndim; i++)
{
if (lb[i] != 1)
{
needdims = true;
break;
}
}
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Convert all values to string form, count total space needed (including
* any overhead such as escaping backslashes), and detect whether each
* item needs double quotes.
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
*/
values = (char **) palloc(nitems * sizeof(char *));
needquotes = (bool *) palloc(nitems * sizeof(bool));
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
overall_length = 0;
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&iter, v);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
for (i = 0; i < nitems; i++)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Datum itemvalue;
bool isnull;
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
bool needquote;
Fix portability problems recently exposed by regression tests on Alphas. 1. Distinguish cases where a Datum representing a tuple datatype is an OID from cases where it is a pointer to TupleTableSlot, and make sure we use the right typlen in each case. 2. Make fetchatt() and related code support 8-byte by-value datatypes on machines where Datum is 8 bytes. Centralize knowledge of the available by-value datatype sizes in two macros in tupmacs.h, so that this will be easier if we ever have to do it again.
2000-12-28 00:59:14 +01:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get source element, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
itemvalue = array_iter_next(&iter, &isnull, i,
typlen, typbyval, typalign);
if (isnull)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
values[i] = pstrdup("NULL");
overall_length += 4;
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
needquote = false;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
else
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
{
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
values[i] = OutputFunctionCall(&my_extra->proc, itemvalue);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* count data plus backslashes; detect chars needing quotes */
if (values[i][0] == '\0')
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
needquote = true; /* force quotes for empty string */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else if (pg_strcasecmp(values[i], "NULL") == 0)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
needquote = true; /* force quotes for literal NULL */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else
needquote = false;
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
for (tmp = values[i]; *tmp != '\0'; tmp++)
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char ch = *tmp;
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
overall_length += 1;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (ch == '"' || ch == '\\')
{
needquote = true;
overall_length += 1;
}
else if (ch == '{' || ch == '}' || ch == typdelim ||
Use a non-locale-dependent definition of isspace() in array_in/array_out. array_in discards unquoted leading and trailing whitespace in array values, while array_out is careful to quote array elements that contain whitespace. This is problematic when the definition of "whitespace" varies between locales: array_in could drop characters that were meant to be part of the value. To avoid that, lock down "whitespace" to mean only the traditional six ASCII space characters. This change also works around a bug in OS X and some older BSD systems, in which isspace() could return true for character fragments in UTF8 locales. (There may be other places in PG where that bug could cause problems, but this is the only one complained of so far; see recent report from Steven Schlansker.) Back-patch to 9.0, but not further. Given the lack of previous reports of trouble, changing this behavior in stable branches seems to offer more risk of breaking applications than reward of avoiding problems.
2010-08-21 18:55:51 +02:00
array_isspace(ch))
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
needquote = true;
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
}
> > I'm using text[] arrays. Some of my array elements have '"' > > characters in them. Dumping and reloading using pg_dumpall > > doesn't work with this and dumping the entire array and > > then trying to parse it is hopeless. Doug Gibson
1998-03-20 04:44:19 +01:00
}
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
needquotes[i] = needquote;
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
/* Count the pair of double quotes, if needed */
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
if (needquote)
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
overall_length += 2;
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
/* and the comma (or other typdelim delimiter) */
> > I'm using text[] arrays. Some of my array elements have '"' > > characters in them. Dumping and reloading using pg_dumpall > > doesn't work with this and dumping the entire array and > > then trying to parse it is hopeless. Doug Gibson
1998-03-20 04:44:19 +01:00
overall_length += 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
/*
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
* The very last array element doesn't have a typdelim delimiter after it,
* but that's OK; that space is needed for the trailing '\0'.
*
* Now count total number of curly brace pairs in output string.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
for (i = j = 0, k = 1; i < ndim; i++)
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
{
j += k, k *= dims[i];
}
overall_length += 2 * j;
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
/* Format explicit dimensions if required */
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
dims_str[0] = '\0';
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
if (needdims)
{
Pgindent run for 8.0.
2004-08-29 07:07:03 +02:00
char *ptr = dims_str;
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
for (i = 0; i < ndim; i++)
{
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
sprintf(ptr, "[%d:%d]", lb[i], lb[i] + dims[i] - 1);
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
ptr += strlen(ptr);
}
*ptr++ = *ASSGN;
*ptr = '\0';
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
overall_length += ptr - dims_str;
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
}
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
/* Now construct the output string */
retval = (char *) palloc(overall_length);
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
p = retval;
#define APPENDSTR(str) (strcpy(p, (str)), p += strlen(p))
#define APPENDCHAR(ch) (*p++ = (ch), *p = '\0')
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Require that array literals produce "rectangular" arrays, i.e. all the subarrays of a given dimension have the same number of elements/subarrays. Also repair a longstanding undocumented (as far as I can see) ability to explicitly set array bounds in the array literal syntax. It now can deal properly with negative array indicies. Modify array_out so that arrays with non-standard lower bounds (i.e. not 1) are output with the expicit dimension syntax. This fixes a longstanding issue whereby arrays with non-default lower bounds had them changed to default after a dump/reload cycle. Modify regression tests and docs to suit, and add some minimal documentation regarding the explicit dimension syntax.
2004-08-05 05:30:44 +02:00
if (needdims)
APPENDSTR(dims_str);
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDCHAR('{');
Fix a read of uninitialized memory in array_out(). Perform some minor cosmetic code cleanup at the same time.
2004-09-16 05:15:54 +02:00
for (i = 0; i < ndim; i++)
indx[i] = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
j = 0;
k = 0;
do
{
for (i = j; i < ndim - 1; i++)
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDCHAR('{');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
if (needquotes[k])
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDCHAR('"');
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
for (tmp = values[k]; *tmp; tmp++)
{
pgindent run.
2002-09-04 22:31:48 +02:00
char ch = *tmp;
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
if (ch == '"' || ch == '\\')
*p++ = '\\';
*p++ = ch;
OK, folks, here is the pgindent output.
1998-09-01 06:40:42 +02:00
}
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
*p = '\0';
APPENDCHAR('"');
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDSTR(values[k]);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pfree(values[k++]);
for (i = ndim - 1; i >= 0; i--)
{
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
if (++(indx[i]) < dims[i])
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDCHAR(typdelim);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
break;
}
else
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
{
indx[i] = 0;
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
APPENDCHAR('}');
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
j = i;
} while (j != -1);
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
#undef APPENDSTR
#undef APPENDCHAR
Fix over-allocation of space for array_out()'s result string. array_out overestimated the space needed for its output, possibly by a very substantial amount if the array is multi-dimensional, because of wrong order of operations in the loop that counts the number of curly-brace pairs needed. While the output string is normally short-lived, this could still cause problems in extreme cases. An additional minor error was that it counted one more delimiter than is actually needed. Repair those errors, add an Assert that the space is now correctly calculated, and make some minor improvements in the comments. I also failed to resist the temptation to get rid of an integer modulus operation per array element; a simple comparison is sufficient. This bug dates clear back to Berkeley days, so back-patch to all supported versions. Keiichi Hirobe, minor additional work by me Discussion: https://postgr.es/m/CAH=EFxE9W0tRvQkixR2XJRRCToUYUEDkJZk6tnADXugPBRdcdg@mail.gmail.com
2018-09-24 17:30:51 +02:00
/* Assert that we calculated the string length accurately */
Assert(overall_length == (p - retval + 1));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
pfree(values);
Fix array_out's failure to backslash backslashes, per bug# 524. Also, remove brain-dead rule that double quotes are needed if and only if the datatype is pass-by-reference; neither direction of the implication holds water. Instead, examine the actual data string to see if it contains any characters that force us to quote it. Add some documentation about quoting of array values, which was previously explained nowhere AFAICT.
2001-11-29 22:02:41 +01:00
pfree(needquotes);
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
PG_RETURN_CSTRING(retval);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
* array_recv :
* converts an array from the external binary format to
* its internal format.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
* return value :
* the internal representation of the input array
*/
Datum
array_recv(PG_FUNCTION_ARGS)
{
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
StringInfo buf = (StringInfo) PG_GETARG_POINTER(0);
Oid spec_element_type = PG_GETARG_OID(1); /* type of an array
* element */
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
int32 typmod = PG_GETARG_INT32(2); /* typmod for array elements */
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Oid element_type;
int typlen;
bool typbyval;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
char typalign;
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
Oid typioparam;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
int i,
nitems;
Datum *dataPtr;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nullsPtr;
bool hasnulls;
int32 nbytes;
int32 dataoffset;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
ArrayType *retval;
int ndim,
flags,
dim[MAXDIM],
lBound[MAXDIM];
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *my_extra;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
/* Get the array header information */
ndim = pq_getmsgint(buf, 4);
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
if (ndim < 0) /* we do allow zero-dimension arrays */
ereport(ERROR,
(errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
errmsg("invalid number of dimensions: %d", ndim)));
if (ndim > MAXDIM)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
ndim, MAXDIM)));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
flags = pq_getmsgint(buf, 4);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (flags != 0 && flags != 1)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
errmsg("invalid array flags")));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
element_type = pq_getmsgint(buf, sizeof(Oid));
if (element_type != spec_element_type)
{
/* XXX Can we allow taking the input element type in any cases? */
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("wrong element type")));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
for (i = 0; i < ndim; i++)
{
dim[i] = pq_getmsgint(buf, 4);
lBound[i] = pq_getmsgint(buf, 4);
Tigthen binary receive functions so that they reject values that the text input functions don't accept either. While the backend can handle such values fine, they can cause trouble in clients and in pg_dump/restore. This is followup to the original issue on time datatype reported by Andrew McNamara a while ago. Like that one, none of these seem worth back-patching.
2009-09-04 13:20:23 +02:00
The sanity check added to array_recv() wa a bit too tight; we must continue to accept an empty array with dimension information. array_send() can output such arrays. Per report from Vladimir Shakhov.
2010-08-11 21:12:27 +02:00
/*
Fix many typos and inconsistencies Author: Alexander Lakhin Discussion: https://postgr.es/m/af27d1b3-a128-9d62-46e0-88f424397f44@gmail.com
2019-07-01 03:00:23 +02:00
* Check overflow of upper bound. (ArrayGetNItems() below checks that
The sanity check added to array_recv() wa a bit too tight; we must continue to accept an empty array with dimension information. array_send() can output such arrays. Per report from Vladimir Shakhov.
2010-08-11 21:12:27 +02:00
* dim[i] >= 0)
*/
if (dim[i] != 0)
{
pgindent run before PG 9.1 beta 1.
2011-04-10 17:42:00 +02:00
int ub = lBound[i] + dim[i] - 1;
The sanity check added to array_recv() wa a bit too tight; we must continue to accept an empty array with dimension information. array_send() can output such arrays. Per report from Vladimir Shakhov.
2010-08-11 21:12:27 +02:00
if (lBound[i] > ub)
ereport(ERROR,
(errcode(ERRCODE_NUMERIC_VALUE_OUT_OF_RANGE),
errmsg("integer out of range")));
}
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* This checks for overflow of array dimensions */
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
nitems = ArrayGetNItems(ndim, dim);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We arrange to look up info about element type, including its receive
* conversion proc, only once per series of calls, assuming the element
* type doesn't change underneath us.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
sizeof(ArrayMetaState));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
array_in() and array_recv() need to be more paranoid about validating their OID parameter. It was possible to crash the backend with select array_in('{123}',0,0); because that would bypass the needed step of initializing the workspace. These seem to be the only two places with a problem, though (record_in and record_recv don't have the issue, and the other array functions aren't depending on user-supplied input). Back-patch as far as 7.4; 7.3 does not have the bug.
2005-08-15 21:40:20 +02:00
my_extra->element_type = ~element_type;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
if (my_extra->element_type != element_type)
{
/* Get info about element type, including its receive proc */
get_type_io_data(element_type, IOFunc_receive,
&my_extra->typlen, &my_extra->typbyval,
&my_extra->typalign, &my_extra->typdelim,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
&my_extra->typioparam, &my_extra->typiofunc);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
if (!OidIsValid(my_extra->typiofunc))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("no binary input function available for type %s",
format_type_be(element_type))));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
fmgr_info_cxt(my_extra->typiofunc, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
my_extra->element_type = element_type;
}
array_in() and array_recv() need to be more paranoid about validating their OID parameter. It was possible to crash the backend with select array_in('{123}',0,0); because that would bypass the needed step of initializing the workspace. These seem to be the only two places with a problem, though (record_in and record_recv don't have the issue, and the other array functions aren't depending on user-supplied input). Back-patch as far as 7.4; 7.3 does not have the bug.
2005-08-15 21:40:20 +02:00
if (nitems == 0)
{
/* Return empty array ... but not till we've validated element_type */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
PG_RETURN_ARRAYTYPE_P(construct_empty_array(element_type));
array_in() and array_recv() need to be more paranoid about validating their OID parameter. It was possible to crash the backend with select array_in('{123}',0,0); because that would bypass the needed step of initializing the workspace. These seem to be the only two places with a problem, though (record_in and record_recv don't have the issue, and the other array functions aren't depending on user-supplied input). Back-patch as far as 7.4; 7.3 does not have the bug.
2005-08-15 21:40:20 +02:00
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
typlen = my_extra->typlen;
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
typioparam = my_extra->typioparam;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dataPtr = (Datum *) palloc(nitems * sizeof(Datum));
nullsPtr = (bool *) palloc(nitems * sizeof(bool));
ReadArrayBinary(buf, nitems,
&my_extra->proc, typioparam, typmod,
typlen, typbyval, typalign,
dataPtr, nullsPtr,
&hasnulls, &nbytes);
if (hasnulls)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndim, nitems);
nbytes += dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes += ARR_OVERHEAD_NONULLS(ndim);
}
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
retval = (ArrayType *) palloc0(nbytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(retval, nbytes);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
retval->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
retval->dataoffset = dataoffset;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
retval->elemtype = element_type;
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
memcpy(ARR_DIMS(retval), dim, ndim * sizeof(int));
memcpy(ARR_LBOUND(retval), lBound, ndim * sizeof(int));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
CopyArrayEls(retval,
dataPtr, nullsPtr, nitems,
typlen, typbyval, typalign,
true);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
pfree(dataPtr);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
pfree(nullsPtr);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
PG_RETURN_ARRAYTYPE_P(retval);
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
* ReadArrayBinary:
* collect the data elements of an array being read in binary style.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
* Inputs:
* buf: the data buffer to read from.
* nitems: total number of array elements (already read).
* receiveproc: type-specific receive procedure for element datatype.
* typioparam, typmod: auxiliary values to pass to receiveproc.
* typlen, typbyval, typalign: storage parameters of element datatype.
*
* Outputs:
* values[]: filled with converted data values.
* nulls[]: filled with is-null markers.
Change TRUE/FALSE to true/false The lower case spellings are C and C++ standard and are used in most parts of the PostgreSQL sources. The upper case spellings are only used in some files/modules. So standardize on the standard spellings. The APIs for ICU, Perl, and Windows define their own TRUE and FALSE, so those are left as is when using those APIs. In code comments, we use the lower-case spelling for the C concepts and keep the upper-case spelling for the SQL concepts. Reviewed-by: Michael Paquier <michael.paquier@gmail.com>
2017-08-16 06:22:32 +02:00
* *hasnulls: set true iff there are any null elements.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* *nbytes: set to total size of data area needed (including alignment
* padding but not including array header overhead).
*
* Note that values[] and nulls[] are allocated by the caller, and must have
* nitems elements.
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static void
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
ReadArrayBinary(StringInfo buf,
int nitems,
FmgrInfo *receiveproc,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
Oid typioparam,
Change typreceive function API so that receive functions get the same optional arguments as text input functions, ie, typioparam OID and atttypmod. Make all the datatypes that use typmod enforce it the same way in typreceive as they do in typinput. This fixes a problem with failure to enforce length restrictions during COPY FROM BINARY.
2005-07-10 23:14:00 +02:00
int32 typmod,
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
int typlen,
bool typbyval,
char typalign,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Datum *values,
bool *nulls,
bool *hasnulls,
int32 *nbytes)
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
{
int i;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool hasnull;
int32 totbytes;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
for (i = 0; i < nitems; i++)
{
pgindent run.
2003-08-04 02:43:34 +02:00
int itemlen;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
StringInfoData elem_buf;
pgindent run.
2003-08-04 02:43:34 +02:00
char csave;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
/* Get and check the item length */
itemlen = pq_getmsgint(buf, 4);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (itemlen < -1 || itemlen > (buf->len - buf->cursor))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
errmsg("insufficient data left in message")));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (itemlen == -1)
{
/* -1 length means NULL */
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
values[i] = ReceiveFunctionCall(receiveproc, NULL,
typioparam, typmod);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nulls[i] = true;
continue;
}
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Rather than copying data around, we just set up a phony StringInfo
* pointing to the correct portion of the input buffer. We assume we
* can scribble on the input buffer so as to maintain the convention
* that StringInfos have a trailing null.
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
*/
elem_buf.data = &buf->data[buf->cursor];
elem_buf.maxlen = itemlen + 1;
elem_buf.len = itemlen;
elem_buf.cursor = 0;
buf->cursor += itemlen;
csave = buf->data[buf->cursor];
buf->data[buf->cursor] = '\0';
/* Now call the element's receiveproc */
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
values[i] = ReceiveFunctionCall(receiveproc, &elem_buf,
typioparam, typmod);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nulls[i] = false;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
/* Trouble if it didn't eat the whole buffer */
if (elem_buf.cursor != itemlen)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_INVALID_BINARY_REPRESENTATION),
errmsg("improper binary format in array element %d",
pgindent run.
2003-08-04 02:43:34 +02:00
i + 1)));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
buf->data[buf->cursor] = csave;
}
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Check for nulls, compute total data space needed
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
hasnull = false;
totbytes = 0;
for (i = 0; i < nitems; i++)
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nulls[i])
hasnull = true;
else
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
{
/* let's just make sure data is not toasted */
if (typlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM(values[i]));
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
totbytes = att_addlength_datum(totbytes, typlen, values[i]);
totbytes = att_align_nominal(totbytes, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* check for overflow of total request */
if (!AllocSizeIsValid(totbytes))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*hasnulls = hasnull;
*nbytes = totbytes;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
* array_send :
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* takes the internal representation of an array and returns a bytea
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
* containing the array in its external binary format.
*/
Datum
array_send(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Oid element_type = AARR_ELEMTYPE(v);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
int typlen;
bool typbyval;
char typalign;
int nitems,
i;
int ndim,
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
*dim,
*lb;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
StringInfoData buf;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter iter;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *my_extra;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* We arrange to look up info about element type, including its send
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* conversion proc, only once per series of calls, assuming the element
* type doesn't change underneath us.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
sizeof(ArrayMetaState));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
my_extra->element_type = ~element_type;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
if (my_extra->element_type != element_type)
{
/* Get info about element type, including its send proc */
get_type_io_data(element_type, IOFunc_send,
&my_extra->typlen, &my_extra->typbyval,
&my_extra->typalign, &my_extra->typdelim,
Infrastructure for I/O of composite types: arrange for the I/O routines of a composite type to get that type's OID as their second parameter, in place of typelem which is useless. The actual changes are mostly centralized in getTypeInputInfo and siblings, but I had to fix a few places that were fetching pg_type.typelem for themselves instead of using the lsyscache.c routines. Also, I renamed all the related variables from 'typelem' to 'typioparam' to discourage people from assuming that they necessarily contain array element types.
2004-06-06 02:41:28 +02:00
&my_extra->typioparam, &my_extra->typiofunc);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
if (!OidIsValid(my_extra->typiofunc))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("no binary output function available for type %s",
format_type_be(element_type))));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
fmgr_info_cxt(my_extra->typiofunc, &my_extra->proc,
fcinfo->flinfo->fn_mcxt);
my_extra->element_type = element_type;
}
typlen = my_extra->typlen;
typbyval = my_extra->typbyval;
typalign = my_extra->typalign;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
ndim = AARR_NDIM(v);
dim = AARR_DIMS(v);
lb = AARR_LBOUND(v);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
nitems = ArrayGetNItems(ndim, dim);
pq_begintypsend(&buf);
/* Send the array header information */
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(&buf, ndim);
pq_sendint32(&buf, AARR_HASNULL(v) ? 1 : 0);
pq_sendint32(&buf, element_type);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
for (i = 0; i < ndim; i++)
{
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(&buf, dim[i]);
pq_sendint32(&buf, lb[i]);
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
/* Send the array elements using the element's own sendproc */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&iter, v);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
for (i = 0; i < nitems; i++)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Datum itemvalue;
bool isnull;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get source element, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
itemvalue = array_iter_next(&iter, &isnull, i,
typlen, typbyval, typalign);
if (isnull)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
/* -1 length means a NULL */
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(&buf, -1);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
else
{
bytea *outputbytes;
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
Modify all callers of datatype input and receive functions so that if these functions are not strict, they will be called (passing a NULL first parameter) during any attempt to input a NULL value of their datatype. Currently, all our input functions are strict and so this commit does not change any behavior. However, this will make it possible to build domain input functions that centralize checking of domain constraints, thereby closing numerous holes in our domain support, as per previous discussion. While at it, I took the opportunity to introduce convenience functions InputFunctionCall, OutputFunctionCall, etc to use in code that calls I/O functions. This eliminates a lot of grotty-looking casts, but the main motivation is to make it easier to grep for these places if we ever need to touch them again.
2006-04-04 21:35:37 +02:00
outputbytes = SendFunctionCall(&my_extra->proc, itemvalue);
Replace remaining uses of pq_sendint with pq_sendint{8,16,32}. pq_sendint() remains, so extension code doesn't unnecessarily break. Author: Andres Freund Discussion: https://postgr.es/m/20170914063418.sckdzgjfrsbekae4@alap3.anarazel.de
2017-10-12 06:00:46 +02:00
pq_sendint32(&buf, VARSIZE(outputbytes) - VARHDRSZ);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
pq_sendbytes(&buf, VARDATA(outputbytes),
VARSIZE(outputbytes) - VARHDRSZ);
pfree(outputbytes);
}
Implement array_send/array_recv (binary I/O for arrays). This exposed the folly of not passing element type to typsend/typreceive, so fix that.
2003-05-10 01:01:45 +02:00
}
PG_RETURN_BYTEA_P(pq_endtypsend(&buf));
Reinstate pg_type's typsend and typreceive columns. They don't do much yet, but they're there. Also some editorial work on CREATE TYPE reference page.
2003-05-09 00:19:58 +02:00
}
ADD array_ndims function Author: Robert Haas <robertmhaas@gmail.com>
2008-11-04 15:49:12 +01:00
/*
* array_ndims :
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* returns the number of dimensions of the array pointed to by "v"
ADD array_ndims function Author: Robert Haas <robertmhaas@gmail.com>
2008-11-04 15:49:12 +01:00
*/
Datum
array_ndims(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
ADD array_ndims function Author: Robert Haas <robertmhaas@gmail.com>
2008-11-04 15:49:12 +01:00
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
ADD array_ndims function Author: Robert Haas <robertmhaas@gmail.com>
2008-11-04 15:49:12 +01:00
PG_RETURN_NULL();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
PG_RETURN_INT32(AARR_NDIM(v));
ADD array_ndims function Author: Robert Haas <robertmhaas@gmail.com>
2008-11-04 15:49:12 +01:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* array_dims :
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
* returns the dimensions of the array pointed to by "v", as a "text"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Datum
array_dims(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
char *p;
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
int i;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int *dimv,
*lb;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
/*
* 33 since we assume 15 digits per number + ':' +'[]'
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
*
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
* +1 for trailing null
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
char buf[MAXDIM * 33 + 1];
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
PG_RETURN_NULL();
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
dimv = AARR_DIMS(v);
lb = AARR_LBOUND(v);
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
p = buf;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
for (i = 0; i < AARR_NDIM(v); i++)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
sprintf(p, "[%d:%d]", lb[i], dimv[i] + lb[i] - 1);
p += strlen(p);
}
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Simplify and standardize conversions between TEXT datums and ordinary C strings. This patch introduces four support functions cstring_to_text, cstring_to_text_with_len, text_to_cstring, and text_to_cstring_buffer, and two macros CStringGetTextDatum and TextDatumGetCString. A number of existing macros that provided variants on these themes were removed. Most of the places that need to make such conversions now require just one function or macro call, in place of the multiple notational layers that used to be needed. There are no longer any direct calls of textout or textin, and we got most of the places that were using handmade conversions via memcpy (there may be a few still lurking, though). This commit doesn't make any serious effort to eliminate transient memory leaks caused by detoasting toasted text objects before they reach text_to_cstring. We changed PG_GETARG_TEXT_P to PG_GETARG_TEXT_PP in a few places where it was easy, but much more could be done. Brendan Jurd and Tom Lane
2008-03-25 23:42:46 +01:00
PG_RETURN_TEXT_P(cstring_to_text(buf));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
* array_lower :
* returns the lower dimension, of the DIM requested, for
* the array pointed to by "v", as an int4
*/
Datum
array_lower(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
int reqdim = PG_GETARG_INT32(1);
int *lb;
int result;
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (reqdim <= 0 || reqdim > AARR_NDIM(v))
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
PG_RETURN_NULL();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb = AARR_LBOUND(v);
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
result = lb[reqdim - 1];
PG_RETURN_INT32(result);
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
* array_upper :
* returns the upper dimension, of the DIM requested, for
* the array pointed to by "v", as an int4
*/
Datum
array_upper(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
int reqdim = PG_GETARG_INT32(1);
int *dimv,
*lb;
int result;
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (reqdim <= 0 || reqdim > AARR_NDIM(v))
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
PG_RETURN_NULL();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb = AARR_LBOUND(v);
dimv = AARR_DIMS(v);
The "Allow easy display of usernames in a group (pg_hba.conf uses groups now)" item on the open items, and subsequent plpgsql function I sent in, made me realize it was too hard to get the upper and lower bound of an array. The attached creates two functions that I think will be very useful when combined with the ability of plpgsql to return sets. array_lower(array, dim_num) - and - array_upper(array, dim_num) They return the value (as an int) of the upper and lower bound of the requested dim in the provided array. Joe Conway
2002-11-08 18:27:03 +01:00
result = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
PG_RETURN_INT32(result);
}
array_length() function, and for SQL compatibility also cardinality() function as a special case. This version still has the suspicious behavior of returning null for an empty array (rather than zero), but this may need a wholesale revision of empty array behavior, currently under discussion. Jim Nasby, Robert Haas, Peter Eisentraut
2008-11-12 14:09:28 +01:00
/*
* array_length :
* returns the length, of the dimension requested, for
* the array pointed to by "v", as an int4
*/
Datum
array_length(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
array_length() function, and for SQL compatibility also cardinality() function as a special case. This version still has the suspicious behavior of returning null for an empty array (rather than zero), but this may need a wholesale revision of empty array behavior, currently under discussion. Jim Nasby, Robert Haas, Peter Eisentraut
2008-11-12 14:09:28 +01:00
int reqdim = PG_GETARG_INT32(1);
int *dimv;
int result;
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
array_length() function, and for SQL compatibility also cardinality() function as a special case. This version still has the suspicious behavior of returning null for an empty array (rather than zero), but this may need a wholesale revision of empty array behavior, currently under discussion. Jim Nasby, Robert Haas, Peter Eisentraut
2008-11-12 14:09:28 +01:00
PG_RETURN_NULL();
/* Sanity check: was the requested dim valid */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (reqdim <= 0 || reqdim > AARR_NDIM(v))
array_length() function, and for SQL compatibility also cardinality() function as a special case. This version still has the suspicious behavior of returning null for an empty array (rather than zero), but this may need a wholesale revision of empty array behavior, currently under discussion. Jim Nasby, Robert Haas, Peter Eisentraut
2008-11-12 14:09:28 +01:00
PG_RETURN_NULL();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
dimv = AARR_DIMS(v);
array_length() function, and for SQL compatibility also cardinality() function as a special case. This version still has the suspicious behavior of returning null for an empty array (rather than zero), but this may need a wholesale revision of empty array behavior, currently under discussion. Jim Nasby, Robert Haas, Peter Eisentraut
2008-11-12 14:09:28 +01:00
result = dimv[reqdim - 1];
PG_RETURN_INT32(result);
}
Add a cardinality function for arrays. Unlike our other array functions, this considers the total number of elements across all dimensions, and returns 0 rather than NULL when the array has no elements. But it seems that both of those behaviors are almost universally disliked, so hopefully that's OK. Marko Tiikkaja, reviewed by Dean Rasheed and Pavel Stehule
2014-01-21 18:38:53 +01:00
/*
* array_cardinality:
* returns the total number of elements in an array
*/
Datum
array_cardinality(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
PG_RETURN_INT32(ArrayGetNItems(AARR_NDIM(v), AARR_DIMS(v)));
Add a cardinality function for arrays. Unlike our other array functions, this considers the total number of elements across all dimensions, and returns 0 rather than NULL when the array has no elements. But it seems that both of those behaviors are almost universally disliked, so hopefully that's OK. Marko Tiikkaja, reviewed by Dean Rasheed and Pavel Stehule
2014-01-21 18:38:53 +01:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* array_get_element :
* This routine takes an array datum and a subscript array and returns
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* the referenced item as a Datum. Note that for a pass-by-reference
* datatype, the returned Datum is a pointer into the array object.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
* This handles both ordinary varlena arrays and fixed-length arrays.
*
* Inputs:
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* arraydatum: the array object (mustn't be NULL)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nSubscripts: number of subscripts supplied
* indx[]: the subscript values
* arraytyplen: pg_type.typlen for the array type
* elmlen: pg_type.typlen for the array's element type
* elmbyval: pg_type.typbyval for the array's element type
* elmalign: pg_type.typalign for the array's element type
*
* Outputs:
* The return value is the element Datum.
* *isNull is set to indicate whether the element is NULL.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Datum
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
array_get_element(Datum arraydatum,
int nSubscripts,
int *indx,
int arraytyplen,
int elmlen,
bool elmbyval,
char elmalign,
bool *isNull)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int i,
ndim,
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
*dim,
*lb,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
offset,
fixedDim[1],
fixedLb[1];
char *arraydataptr,
*retptr;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *arraynullsptr;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (arraytyplen > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
/*
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* fixed-length arrays -- these are assumed to be 1-d, 0-based
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
ndim = 1;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
fixedDim[0] = arraytyplen / elmlen;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
fixedLb[0] = 0;
dim = fixedDim;
lb = fixedLb;
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
arraydataptr = (char *) DatumGetPointer(arraydatum);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
arraynullsptr = NULL;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
else if (VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(arraydatum)))
{
/* expanded array: let's do this in a separate function */
return array_get_element_expanded(arraydatum,
nSubscripts,
indx,
arraytyplen,
elmlen,
elmbyval,
elmalign,
isNull);
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
else
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/* detoast array if necessary, producing normal varlena input */
ArrayType *array = DatumGetArrayTypeP(arraydatum);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
ndim = ARR_NDIM(array);
dim = ARR_DIMS(array);
lb = ARR_LBOUND(array);
arraydataptr = ARR_DATA_PTR(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
arraynullsptr = ARR_NULLBITMAP(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Return NULL for invalid subscript
*/
if (ndim != nSubscripts || ndim <= 0 || ndim > MAXDIM)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
*isNull = true;
return (Datum) 0;
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
for (i = 0; i < ndim; i++)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
if (indx[i] < lb[i] || indx[i] >= (dim[i] + lb[i]))
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
*isNull = true;
return (Datum) 0;
}
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Calculate the element number
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Check for NULL array element
*/
if (array_get_isnull(arraynullsptr, offset))
{
*isNull = true;
return (Datum) 0;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* OK, get the element
*/
array_ref() should set isNull to false explicitly if it's not going to return NULL.
2002-03-01 23:17:10 +01:00
*isNull = false;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
retptr = array_seek(arraydataptr, 0, arraynullsptr, offset,
elmlen, elmbyval, elmalign);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
return ArrayCast(retptr, elmbyval, elmlen);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/*
* Implementation of array_get_element() for an expanded array
*/
static Datum
array_get_element_expanded(Datum arraydatum,
int nSubscripts, int *indx,
int arraytyplen,
int elmlen, bool elmbyval, char elmalign,
bool *isNull)
{
ExpandedArrayHeader *eah;
int i,
ndim,
*dim,
*lb,
offset;
Datum *dvalues;
bool *dnulls;
eah = (ExpandedArrayHeader *) DatumGetEOHP(arraydatum);
Assert(eah->ea_magic == EA_MAGIC);
/* sanity-check caller's info against object */
Assert(arraytyplen == -1);
Assert(elmlen == eah->typlen);
Assert(elmbyval == eah->typbyval);
Assert(elmalign == eah->typalign);
ndim = eah->ndims;
dim = eah->dims;
lb = eah->lbound;
/*
* Return NULL for invalid subscript
*/
if (ndim != nSubscripts || ndim <= 0 || ndim > MAXDIM)
{
*isNull = true;
return (Datum) 0;
}
for (i = 0; i < ndim; i++)
{
if (indx[i] < lb[i] || indx[i] >= (dim[i] + lb[i]))
{
*isNull = true;
return (Datum) 0;
}
}
/*
* Calculate the element number
*/
offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
/*
* Deconstruct array if we didn't already. Note that we apply this even
* if the input is nominally read-only: it should be safe enough.
*/
deconstruct_expanded_array(eah);
dvalues = eah->dvalues;
dnulls = eah->dnulls;
/*
* Check for NULL array element
*/
if (dnulls && dnulls[offset])
{
*isNull = true;
return (Datum) 0;
}
/*
* OK, get the element. It's OK to return a pass-by-ref value as a
* pointer into the expanded array, for the same reason that regular
* array_get_element can return a pointer into flat arrays: the value is
* assumed not to change for as long as the Datum reference can exist.
*/
*isNull = false;
return dvalues[offset];
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* array_get_slice :
Fix inconsistencies and typos in the tree, take 11 This fixes various typos in docs and comments, and removes some orphaned definitions. Author: Alexander Lakhin Discussion: https://postgr.es/m/5da8e325-c665-da95-21e0-c8a99ea61fbf@gmail.com
2019-08-19 09:21:39 +02:00
* This routine takes an array and a range of indices (upperIndx and
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* lowerIndx), creates a new array structure for the referred elements
* and returns a pointer to it.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* This handles both ordinary varlena arrays and fixed-length arrays.
*
* Inputs:
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* arraydatum: the array object (mustn't be NULL)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nSubscripts: number of subscripts supplied (must be same for upper/lower)
* upperIndx[]: the upper subscript values
* lowerIndx[]: the lower subscript values
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
* upperProvided[]: true for provided upper subscript values
* lowerProvided[]: true for provided lower subscript values
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* arraytyplen: pg_type.typlen for the array type
* elmlen: pg_type.typlen for the array's element type
* elmbyval: pg_type.typbyval for the array's element type
* elmalign: pg_type.typalign for the array's element type
*
* Outputs:
* The return value is the new array Datum (it's never NULL)
*
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
* Omitted upper and lower subscript values are replaced by the corresponding
* array bound.
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* NOTE: we assume it is OK to scribble on the provided subscript arrays
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
* lowerIndx[] and upperIndx[]. These are generally just temporaries.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
Datum
array_get_slice(Datum arraydatum,
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
int nSubscripts,
int *upperIndx,
int *lowerIndx,
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
bool *upperProvided,
bool *lowerProvided,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int arraytyplen,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int elmlen,
bool elmbyval,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char elmalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
ArrayType *array;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ArrayType *newarray;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
ndim,
*dim,
Array slice extraction should produce a result array with index lower bounds of 1, not the lower bound subscripts of the original slice. Per bug report from Andre Holzner, 1-Feb-02.
2002-03-02 01:34:24 +01:00
*lb,
*newlb;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int fixedDim[1],
fixedLb[1];
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Oid elemtype;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
char *arraydataptr;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *arraynullsptr;
int32 dataoffset;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int bytes,
span[MAXDIM];
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (arraytyplen > 0)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* fixed-length arrays -- currently, cannot slice these because parser
* labels output as being of the fixed-length array type! Code below
* shows how we could support it if the parser were changed to label
* output as a suitable varlena array type.
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
*/
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("slices of fixed-length arrays not implemented")));
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* fixed-length arrays -- these are assumed to be 1-d, 0-based
*
* XXX where would we get the correct ELEMTYPE from?
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
ndim = 1;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
fixedDim[0] = arraytyplen / elmlen;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
fixedLb[0] = 0;
dim = fixedDim;
lb = fixedLb;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
elemtype = InvalidOid; /* XXX */
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
arraydataptr = (char *) DatumGetPointer(arraydatum);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
arraynullsptr = NULL;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
else
{
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
/* detoast input array if necessary */
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
array = DatumGetArrayTypeP(arraydatum);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
ndim = ARR_NDIM(array);
dim = ARR_DIMS(array);
lb = ARR_LBOUND(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
elemtype = ARR_ELEMTYPE(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
arraydataptr = ARR_DATA_PTR(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
arraynullsptr = ARR_NULLBITMAP(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Check provided subscripts. A slice exceeding the current array limits
* is silently truncated to the array limits. If we end up with an empty
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* slice, return an empty array.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
if (ndim < nSubscripts || ndim <= 0 || ndim > MAXDIM)
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(construct_empty_array(elemtype));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
for (i = 0; i < nSubscripts; i++)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
if (!lowerProvided[i] || lowerIndx[i] < lb[i])
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lowerIndx[i] = lb[i];
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
if (!upperProvided[i] || upperIndx[i] >= (dim[i] + lb[i]))
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
upperIndx[i] = dim[i] + lb[i] - 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (lowerIndx[i] > upperIndx[i])
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(construct_empty_array(elemtype));
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
/* fill any missing subscript positions with full array range */
for (; i < ndim; i++)
{
lowerIndx[i] = lb[i];
upperIndx[i] = dim[i] + lb[i] - 1;
if (lowerIndx[i] > upperIndx[i])
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(construct_empty_array(elemtype));
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
mda_get_range(ndim, span, lowerIndx, upperIndx);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bytes = array_slice_size(arraydataptr, arraynullsptr,
ndim, dim, lb,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
lowerIndx, upperIndx,
elmlen, elmbyval, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Currently, we put a null bitmap in the result if the source has one;
* could be smarter ...
*/
if (arraynullsptr)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndim, ArrayGetNItems(ndim, span));
bytes += dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
bytes += ARR_OVERHEAD_NONULLS(ndim);
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
newarray = (ArrayType *) palloc0(bytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(newarray, bytes);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
newarray->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
newarray->dataoffset = dataoffset;
newarray->elemtype = elemtype;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy(ARR_DIMS(newarray), span, ndim * sizeof(int));
pgindent run.
2002-09-04 22:31:48 +02:00
Array slice extraction should produce a result array with index lower bounds of 1, not the lower bound subscripts of the original slice. Per bug report from Andre Holzner, 1-Feb-02.
2002-03-02 01:34:24 +01:00
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Lower bounds of the new array are set to 1. Formerly (before 7.3) we
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* copied the given lowerIndx values ... but that seems confusing.
Array slice extraction should produce a result array with index lower bounds of 1, not the lower bound subscripts of the original slice. Per bug report from Andre Holzner, 1-Feb-02.
2002-03-02 01:34:24 +01:00
*/
newlb = ARR_LBOUND(newarray);
for (i = 0; i < ndim; i++)
newlb[i] = 1;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_extract_slice(newarray,
ndim, dim, lb,
arraydataptr, arraynullsptr,
lowerIndx, upperIndx,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(newarray);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* array_set_element :
* This routine sets the value of one array element (specified by
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* a subscript array) to a new value specified by "dataValue".
*
* This handles both ordinary varlena arrays and fixed-length arrays.
*
* Inputs:
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* arraydatum: the initial array object (mustn't be NULL)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nSubscripts: number of subscripts supplied
* indx[]: the subscript values
* dataValue: the datum to be inserted at the given position
* isNull: whether dataValue is NULL
* arraytyplen: pg_type.typlen for the array type
* elmlen: pg_type.typlen for the array's element type
* elmbyval: pg_type.typbyval for the array's element type
* elmalign: pg_type.typalign for the array's element type
*
* Result:
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* A new array is returned, just like the old except for the one
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
* modified entry. The original array object is not changed,
* unless what is passed is a read-write reference to an expanded
* array object; in that case the expanded array is updated in-place.
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
*
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
* For one-dimensional arrays only, we allow the array to be extended
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
* by assigning to a position outside the existing subscript range; any
* positions between the existing elements and the new one are set to NULLs.
* (XXX TODO: allow a corresponding behavior for multidimensional arrays)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*
* NOTE: For assignments, we throw an error for invalid subscripts etc,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* rather than returning a NULL as the fetch operations do.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
Datum
array_set_element(Datum arraydatum,
int nSubscripts,
int *indx,
Datum dataValue,
bool isNull,
int arraytyplen,
int elmlen,
bool elmbyval,
char elmalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
ArrayType *array;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ArrayType *newarray;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int i,
ndim,
dim[MAXDIM],
lb[MAXDIM],
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
offset;
char *elt_ptr;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool newhasnulls;
bits8 *oldnullbitmap;
int oldnitems,
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
newnitems,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
olddatasize,
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
newsize,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
olditemlen,
newitemlen,
overheadlen,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldoverheadlen,
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedbefore,
addedafter,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore,
lenafter;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (arraytyplen > 0)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
/*
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
* fixed-length arrays -- these are assumed to be 1-d, 0-based. We
* cannot extend them, either.
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
*/
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
char *resultarray;
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
if (nSubscripts != 1)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("wrong number of array subscripts")));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (indx[0] < 0 || indx[0] * elmlen >= arraytyplen)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("array subscript out of range")));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (isNull)
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
Message style improvements
2006-10-06 19:14:01 +02:00
errmsg("cannot assign null value to an element of a fixed-length array")));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
resultarray = (char *) palloc(arraytyplen);
memcpy(resultarray, DatumGetPointer(arraydatum), arraytyplen);
elt_ptr = (char *) resultarray + indx[0] * elmlen;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
ArrayCastAndSet(dataValue, elmlen, elmbyval, elmalign, elt_ptr);
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(resultarray);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nSubscripts <= 0 || nSubscripts > MAXDIM)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("wrong number of array subscripts")));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
/* make sure item to be inserted is not toasted */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (elmlen == -1 && !isNull)
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
dataValue = PointerGetDatum(PG_DETOAST_DATUM(dataValue));
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (VARATT_IS_EXTERNAL_EXPANDED(DatumGetPointer(arraydatum)))
{
/* expanded array: let's do this in a separate function */
return array_set_element_expanded(arraydatum,
nSubscripts,
indx,
dataValue,
isNull,
arraytyplen,
elmlen,
elmbyval,
elmalign);
}
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
/* detoast input array if necessary */
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
array = DatumGetArrayTypeP(arraydatum);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
ndim = ARR_NDIM(array);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* if number of dims is zero, i.e. an empty array, create an array with
* nSubscripts dimensions, and set the lower bounds to the supplied
* subscripts
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
if (ndim == 0)
{
pgindent run.
2003-08-04 02:43:34 +02:00
Oid elmtype = ARR_ELEMTYPE(array);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
for (i = 0; i < nSubscripts; i++)
{
dim[i] = 1;
lb[i] = indx[i];
}
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(construct_md_array(&dataValue, &isNull,
nSubscripts, dim, lb,
elmtype,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
elmlen, elmbyval, elmalign));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (ndim != nSubscripts)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("wrong number of array subscripts")));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* copy dim/lb since we may modify them */
memcpy(dim, ARR_DIMS(array), ndim * sizeof(int));
memcpy(lb, ARR_LBOUND(array), ndim * sizeof(int));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
newhasnulls = (ARR_HASNULL(array) || isNull);
addedbefore = addedafter = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Check subscripts
*/
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (ndim == 1)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (indx[0] < lb[0])
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedbefore = lb[0] - indx[0];
dim[0] += addedbefore;
lb[0] = indx[0];
if (addedbefore > 1)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (indx[0] >= (dim[0] + lb[0]))
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedafter = indx[0] - (dim[0] + lb[0]) + 1;
dim[0] += addedafter;
if (addedafter > 1)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
}
}
else
{
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* XXX currently we do not support extending multi-dimensional arrays
* during assignment
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
*/
for (i = 0; i < ndim; i++)
{
if (indx[i] < lb[i] ||
indx[i] >= (dim[i] + lb[i]))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("array subscript out of range")));
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Compute sizes of items and areas to copy
*/
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
newnitems = ArrayGetNItems(ndim, dim);
if (newhasnulls)
overheadlen = ARR_OVERHEAD_WITHNULLS(ndim, newnitems);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else
overheadlen = ARR_OVERHEAD_NONULLS(ndim);
oldnitems = ArrayGetNItems(ndim, ARR_DIMS(array));
oldnullbitmap = ARR_NULLBITMAP(array);
oldoverheadlen = ARR_DATA_OFFSET(array);
olddatasize = ARR_SIZE(array) - oldoverheadlen;
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (addedbefore)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
offset = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore = 0;
olditemlen = 0;
lenafter = olddatasize;
}
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
else if (addedafter)
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
offset = oldnitems;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore = olddatasize;
olditemlen = 0;
lenafter = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
elt_ptr = array_seek(ARR_DATA_PTR(array), 0, oldnullbitmap, offset,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore = (int) (elt_ptr - ARR_DATA_PTR(array));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (array_get_isnull(oldnullbitmap, offset))
olditemlen = 0;
else
{
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
olditemlen = att_addlength_pointer(0, elmlen, elt_ptr);
olditemlen = att_align_nominal(olditemlen, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenafter = (int) (olddatasize - lenbefore - olditemlen);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (isNull)
newitemlen = 0;
else
{
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
newitemlen = att_addlength_datum(0, elmlen, dataValue);
newitemlen = att_align_nominal(newitemlen, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
newsize = overheadlen + lenbefore + newitemlen + lenafter;
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* OK, create the new array and fill in header/dimensions
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
newarray = (ArrayType *) palloc0(newsize);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(newarray, newsize);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
newarray->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
newarray->dataoffset = newhasnulls ? overheadlen : 0;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
newarray->elemtype = ARR_ELEMTYPE(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy(ARR_DIMS(newarray), dim, ndim * sizeof(int));
memcpy(ARR_LBOUND(newarray), lb, ndim * sizeof(int));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Fill in data
*/
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy((char *) newarray + overheadlen,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
(char *) array + oldoverheadlen,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (!isNull)
ArrayCastAndSet(dataValue, elmlen, elmbyval, elmalign,
(char *) newarray + overheadlen + lenbefore);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy((char *) newarray + overheadlen + lenbefore + newitemlen,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
(char *) array + oldoverheadlen + lenbefore + olditemlen,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenafter);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Fill in nulls bitmap if needed
*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* Note: it's possible we just replaced the last NULL with a non-NULL, and
* could get rid of the bitmap. Seems not worth testing for though.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*/
if (newhasnulls)
{
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
bits8 *newnullbitmap = ARR_NULLBITMAP(newarray);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
/* Zero the bitmap to take care of marking inserted positions null */
MemSet(newnullbitmap, 0, (newnitems + 7) / 8);
/* Fix the inserted value */
if (addedafter)
array_set_isnull(newnullbitmap, newnitems - 1, isNull);
else
array_set_isnull(newnullbitmap, offset, isNull);
/* Fix the copied range(s) */
if (addedbefore)
array_bitmap_copy(newnullbitmap, addedbefore,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldnullbitmap, 0,
oldnitems);
else
{
array_bitmap_copy(newnullbitmap, 0,
oldnullbitmap, 0,
offset);
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (addedafter == 0)
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
array_bitmap_copy(newnullbitmap, offset + 1,
oldnullbitmap, offset + 1,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldnitems - offset - 1);
}
}
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(newarray);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/*
* Implementation of array_set_element() for an expanded array
*
* Note: as with any operation on a read/write expanded object, we must
* take pains not to leave the object in a corrupt state if we fail partway
* through.
*/
static Datum
array_set_element_expanded(Datum arraydatum,
int nSubscripts, int *indx,
Datum dataValue, bool isNull,
int arraytyplen,
int elmlen, bool elmbyval, char elmalign)
{
ExpandedArrayHeader *eah;
Datum *dvalues;
bool *dnulls;
int i,
ndim,
dim[MAXDIM],
lb[MAXDIM],
offset;
bool dimschanged,
newhasnulls;
int addedbefore,
addedafter;
char *oldValue;
/* Convert to R/W object if not so already */
eah = DatumGetExpandedArray(arraydatum);
/* Sanity-check caller's info against object; we don't use it otherwise */
Assert(arraytyplen == -1);
Assert(elmlen == eah->typlen);
Assert(elmbyval == eah->typbyval);
Assert(elmalign == eah->typalign);
/*
* Copy dimension info into local storage. This allows us to modify the
* dimensions if needed, while not messing up the expanded value if we
* fail partway through.
*/
ndim = eah->ndims;
Assert(ndim >= 0 && ndim <= MAXDIM);
memcpy(dim, eah->dims, ndim * sizeof(int));
memcpy(lb, eah->lbound, ndim * sizeof(int));
dimschanged = false;
/*
* if number of dims is zero, i.e. an empty array, create an array with
* nSubscripts dimensions, and set the lower bounds to the supplied
* subscripts.
*/
if (ndim == 0)
{
/*
* Allocate adequate space for new dimension info. This is harmless
* if we fail later.
*/
Assert(nSubscripts > 0 && nSubscripts <= MAXDIM);
eah->dims = (int *) MemoryContextAllocZero(eah->hdr.eoh_context,
nSubscripts * sizeof(int));
eah->lbound = (int *) MemoryContextAllocZero(eah->hdr.eoh_context,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
nSubscripts * sizeof(int));
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/* Update local copies of dimension info */
ndim = nSubscripts;
for (i = 0; i < nSubscripts; i++)
{
dim[i] = 0;
lb[i] = indx[i];
}
dimschanged = true;
}
else if (ndim != nSubscripts)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("wrong number of array subscripts")));
/*
* Deconstruct array if we didn't already. (Someday maybe add a special
* case path for fixed-length, no-nulls cases, where we can overwrite an
* element in place without ever deconstructing. But today is not that
* day.)
*/
deconstruct_expanded_array(eah);
/*
* Copy new element into array's context, if needed (we assume it's
* already detoasted, so no junk should be created). If we fail further
* down, this memory is leaked, but that's reasonably harmless.
*/
if (!eah->typbyval && !isNull)
{
MemoryContext oldcxt = MemoryContextSwitchTo(eah->hdr.eoh_context);
dataValue = datumCopy(dataValue, false, eah->typlen);
MemoryContextSwitchTo(oldcxt);
}
dvalues = eah->dvalues;
dnulls = eah->dnulls;
newhasnulls = ((dnulls != NULL) || isNull);
addedbefore = addedafter = 0;
/*
* Check subscripts (this logic matches original array_set_element)
*/
if (ndim == 1)
{
if (indx[0] < lb[0])
{
addedbefore = lb[0] - indx[0];
dim[0] += addedbefore;
lb[0] = indx[0];
dimschanged = true;
if (addedbefore > 1)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
}
if (indx[0] >= (dim[0] + lb[0]))
{
addedafter = indx[0] - (dim[0] + lb[0]) + 1;
dim[0] += addedafter;
dimschanged = true;
if (addedafter > 1)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
}
}
else
{
/*
* XXX currently we do not support extending multi-dimensional arrays
* during assignment
*/
for (i = 0; i < ndim; i++)
{
if (indx[i] < lb[i] ||
indx[i] >= (dim[i] + lb[i]))
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("array subscript out of range")));
}
}
/* Now we can calculate linear offset of target item in array */
offset = ArrayGetOffset(nSubscripts, dim, lb, indx);
/* Physically enlarge existing dvalues/dnulls arrays if needed */
if (dim[0] > eah->dvalueslen)
{
/* We want some extra space if we're enlarging */
int newlen = dim[0] + dim[0] / 8;
newlen = Max(newlen, dim[0]); /* integer overflow guard */
eah->dvalues = dvalues = (Datum *)
repalloc(dvalues, newlen * sizeof(Datum));
if (dnulls)
eah->dnulls = dnulls = (bool *)
repalloc(dnulls, newlen * sizeof(bool));
eah->dvalueslen = newlen;
}
/*
* If we need a nulls bitmap and don't already have one, create it, being
* sure to mark all existing entries as not null.
*/
if (newhasnulls && dnulls == NULL)
eah->dnulls = dnulls = (bool *)
MemoryContextAllocZero(eah->hdr.eoh_context,
eah->dvalueslen * sizeof(bool));
/*
* We now have all the needed space allocated, so we're ready to make
* irreversible changes. Be very wary of allowing failure below here.
*/
/* Flattened value will no longer represent array accurately */
eah->fvalue = NULL;
/* And we don't know the flattened size either */
eah->flat_size = 0;
/* Update dimensionality info if needed */
if (dimschanged)
{
eah->ndims = ndim;
memcpy(eah->dims, dim, ndim * sizeof(int));
memcpy(eah->lbound, lb, ndim * sizeof(int));
}
/* Reposition items if needed, and fill addedbefore items with nulls */
if (addedbefore > 0)
{
memmove(dvalues + addedbefore, dvalues, eah->nelems * sizeof(Datum));
for (i = 0; i < addedbefore; i++)
dvalues[i] = (Datum) 0;
if (dnulls)
{
memmove(dnulls + addedbefore, dnulls, eah->nelems * sizeof(bool));
for (i = 0; i < addedbefore; i++)
dnulls[i] = true;
}
eah->nelems += addedbefore;
}
/* fill addedafter items with nulls */
if (addedafter > 0)
{
for (i = 0; i < addedafter; i++)
dvalues[eah->nelems + i] = (Datum) 0;
if (dnulls)
{
for (i = 0; i < addedafter; i++)
dnulls[eah->nelems + i] = true;
}
eah->nelems += addedafter;
}
/* Grab old element value for pfree'ing, if needed. */
if (!eah->typbyval && (dnulls == NULL || !dnulls[offset]))
oldValue = (char *) DatumGetPointer(dvalues[offset]);
else
oldValue = NULL;
/* And finally we can insert the new element. */
dvalues[offset] = dataValue;
if (dnulls)
dnulls[offset] = isNull;
/*
* Free old element if needed; this keeps repeated element replacements
* from bloating the array's storage. If the pfree somehow fails, it
* won't corrupt the array.
*/
if (oldValue)
{
/* Don't try to pfree a part of the original flat array */
if (oldValue < eah->fstartptr || oldValue >= eah->fendptr)
pfree(oldValue);
}
/* Done, return standard TOAST pointer for object */
return EOHPGetRWDatum(&eah->hdr);
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* array_set_slice :
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* This routine sets the value of a range of array locations (specified
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* by upper and lower subscript values) to new values passed as
* another array.
*
* This handles both ordinary varlena arrays and fixed-length arrays.
*
* Inputs:
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* arraydatum: the initial array object (mustn't be NULL)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nSubscripts: number of subscripts supplied (must be same for upper/lower)
* upperIndx[]: the upper subscript values
* lowerIndx[]: the lower subscript values
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
* upperProvided[]: true for provided upper subscript values
* lowerProvided[]: true for provided lower subscript values
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
* srcArrayDatum: the source for the inserted values
* isNull: indicates whether srcArrayDatum is NULL
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* arraytyplen: pg_type.typlen for the array type
* elmlen: pg_type.typlen for the array's element type
* elmbyval: pg_type.typbyval for the array's element type
* elmalign: pg_type.typalign for the array's element type
*
* Result:
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* A new array is returned, just like the old except for the
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* modified range. The original array object is not changed.
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
*
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
* Omitted upper and lower subscript values are replaced by the corresponding
* array bound.
*
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
* For one-dimensional arrays only, we allow the array to be extended
* by assigning to positions outside the existing subscript range; any
* positions between the existing elements and the new ones are set to NULLs.
* (XXX TODO: allow a corresponding behavior for multidimensional arrays)
*
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
* NOTE: we assume it is OK to scribble on the provided index arrays
* lowerIndx[] and upperIndx[]. These are generally just temporaries.
*
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
* NOTE: For assignments, we throw an error for silly subscripts etc,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* rather than returning a NULL or empty array as the fetch operations do.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
Datum
array_set_slice(Datum arraydatum,
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
int nSubscripts,
int *upperIndx,
int *lowerIndx,
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
bool *upperProvided,
bool *lowerProvided,
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
Datum srcArrayDatum,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool isNull,
int arraytyplen,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int elmlen,
bool elmbyval,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char elmalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
ArrayType *array;
ArrayType *srcArray;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ArrayType *newarray;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
ndim,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
dim[MAXDIM],
lb[MAXDIM],
span[MAXDIM];
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool newhasnulls;
int nitems,
nsrcitems,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
olddatasize,
newsize,
olditemsize,
newitemsize,
overheadlen,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldoverheadlen,
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedbefore,
addedafter,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
lenafter,
itemsbefore,
itemsafter,
nolditems;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Currently, assignment from a NULL source array is a no-op */
if (isNull)
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return arraydatum;
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (arraytyplen > 0)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
{
/*
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
* fixed-length arrays -- not got round to doing this...
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
*/
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("updates on slices of fixed-length arrays not implemented")));
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* detoast arrays if necessary */
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
array = DatumGetArrayTypeP(arraydatum);
srcArray = DatumGetArrayTypeP(srcArrayDatum);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
/* note: we assume srcArray contains no toasted elements */
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
ndim = ARR_NDIM(array);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* if number of dims is zero, i.e. an empty array, create an array with
* nSubscripts dimensions, and set the upper and lower bounds to the
* supplied subscripts
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
if (ndim == 0)
{
pgindent run.
2003-08-04 02:43:34 +02:00
Datum *dvalues;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *dnulls;
pgindent run.
2003-08-04 02:43:34 +02:00
int nelems;
Oid elmtype = ARR_ELEMTYPE(array);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
deconstruct_array(srcArray, elmtype, elmlen, elmbyval, elmalign,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
&dvalues, &dnulls, &nelems);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
for (i = 0; i < nSubscripts; i++)
{
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
if (!upperProvided[i] || !lowerProvided[i])
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("array slice subscript must provide both boundaries"),
errdetail("When assigning to a slice of an empty array value,"
" slice boundaries must be fully specified.")));
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
dim[i] = 1 + upperIndx[i] - lowerIndx[i];
lb[i] = lowerIndx[i];
}
Add missing check for too-few-inputs when replacing a zero-dimensional array.
2004-06-08 22:28:21 +02:00
/* complain if too few source items; we ignore extras, however */
if (nelems < ArrayGetNItems(nSubscripts, dim))
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("source array too small")));
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(construct_md_array(dvalues, dnulls, nSubscripts,
dim, lb, elmtype,
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
elmlen, elmbyval, elmalign));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
Tweak behavior of array slicing operations: seems like it ought to be okay to omit low-order dimensions when accessing an array slice.
2002-03-20 20:41:47 +01:00
if (ndim < nSubscripts || ndim <= 0 || ndim > MAXDIM)
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("wrong number of array subscripts")));
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* copy dim/lb since we may modify them */
memcpy(dim, ARR_DIMS(array), ndim * sizeof(int));
memcpy(lb, ARR_LBOUND(array), ndim * sizeof(int));
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
newhasnulls = (ARR_HASNULL(array) || ARR_HASNULL(srcArray));
addedbefore = addedafter = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
* Check subscripts
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (ndim == 1)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
Assert(nSubscripts == 1);
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
if (!lowerProvided[0])
lowerIndx[0] = lb[0];
if (!upperProvided[0])
upperIndx[0] = dim[0] + lb[0] - 1;
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (lowerIndx[0] > upperIndx[0])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("upper bound cannot be less than lower bound")));
if (lowerIndx[0] < lb[0])
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (upperIndx[0] < lb[0] - 1)
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedbefore = lb[0] - lowerIndx[0];
dim[0] += addedbefore;
lb[0] = lowerIndx[0];
}
if (upperIndx[0] >= (dim[0] + lb[0]))
{
if (lowerIndx[0] > (dim[0] + lb[0]))
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
newhasnulls = true; /* will insert nulls */
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
addedafter = upperIndx[0] - (dim[0] + lb[0]) + 1;
dim[0] += addedafter;
}
}
else
{
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* XXX currently we do not support extending multi-dimensional arrays
* during assignment
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
*/
for (i = 0; i < nSubscripts; i++)
{
Allow omitting one or both boundaries in an array slice specifier. Omitted boundaries represent the upper or lower limit of the corresponding array subscript. This allows simpler specification of many common use-cases. (Revised version of commit 9246af6799819847faa33baf441251003acbb8fe) YUriy Zhuravlev
2015-12-23 03:05:16 +01:00
if (!lowerProvided[i])
lowerIndx[i] = lb[i];
if (!upperProvided[i])
upperIndx[i] = dim[i] + lb[i] - 1;
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (lowerIndx[i] > upperIndx[i])
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("upper bound cannot be less than lower bound")));
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (lowerIndx[i] < lb[i] ||
upperIndx[i] >= (dim[i] + lb[i]))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
errmsg("array subscript out of range")));
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
/* fill any missing subscript positions with full array range */
for (; i < ndim; i++)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
lowerIndx[i] = lb[i];
upperIndx[i] = dim[i] + lb[i] - 1;
if (lowerIndx[i] > upperIndx[i])
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("upper bound cannot be less than lower bound")));
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Do this mainly to check for overflow */
nitems = ArrayGetNItems(ndim, dim);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* Make sure source array has enough entries. Note we ignore the shape of
* the source array and just read entries serially.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
mda_get_range(ndim, span, lowerIndx, upperIndx);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
nsrcitems = ArrayGetNItems(ndim, span);
if (nsrcitems > ArrayGetNItems(ARR_NDIM(srcArray), ARR_DIMS(srcArray)))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("source array too small")));
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Compute space occupied by new entries, space occupied by replaced
* entries, and required space for new array.
*/
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
if (newhasnulls)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
overheadlen = ARR_OVERHEAD_WITHNULLS(ndim, nitems);
else
overheadlen = ARR_OVERHEAD_NONULLS(ndim);
newitemsize = array_nelems_size(ARR_DATA_PTR(srcArray), 0,
ARR_NULLBITMAP(srcArray), nsrcitems,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldoverheadlen = ARR_DATA_OFFSET(array);
olddatasize = ARR_SIZE(array) - oldoverheadlen;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
if (ndim > 1)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* here we do not need to cope with extension of the array; it would
* be a lot more complicated if we had to do so...
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
olditemsize = array_slice_size(ARR_DATA_PTR(array),
ARR_NULLBITMAP(array),
ndim, dim, lb,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
lowerIndx, upperIndx,
elmlen, elmbyval, elmalign);
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
lenbefore = lenafter = 0; /* keep compiler quiet */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
itemsbefore = itemsafter = nolditems = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
else
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* here we must allow for possibility of slice larger than orig array
Fix miscalculation of itemsafter in array_set_slice(). If the slice to be assigned to was before the existing array lower bound (requiring at least one null element to spring into existence to fill the gap), the code miscalculated how many entries needed to be copied from the old array's null bitmap. This could result in trashing the array's data area (as seen in bug #5840 from Karsten Loesing), or worse. This has been broken since we first allowed the behavior of assigning to non-adjacent slices, in 8.2. Back-patch to all affected versions.
2011-01-17 18:38:52 +01:00
* and/or not adjacent to orig array subscripts
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
int oldlb = ARR_LBOUND(array)[0];
int oldub = oldlb + ARR_DIMS(array)[0] - 1;
Remove MAX/MIN() macros, use c.h Max/Min() instead.
2002-02-18 15:25:40 +01:00
int slicelb = Max(oldlb, lowerIndx[0]);
int sliceub = Min(oldub, upperIndx[0]);
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
char *oldarraydata = ARR_DATA_PTR(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *oldarraybitmap = ARR_NULLBITMAP(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
Fix miscalculation of itemsafter in array_set_slice(). If the slice to be assigned to was before the existing array lower bound (requiring at least one null element to spring into existence to fill the gap), the code miscalculated how many entries needed to be copied from the old array's null bitmap. This could result in trashing the array's data area (as seen in bug #5840 from Karsten Loesing), or worse. This has been broken since we first allowed the behavior of assigning to non-adjacent slices, in 8.2. Back-patch to all affected versions.
2011-01-17 18:38:52 +01:00
/* count/size of old array entries that will go before the slice */
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
itemsbefore = Min(slicelb, oldub + 1) - oldlb;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
lenbefore = array_nelems_size(oldarraydata, 0, oldarraybitmap,
itemsbefore,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Fix miscalculation of itemsafter in array_set_slice(). If the slice to be assigned to was before the existing array lower bound (requiring at least one null element to spring into existence to fill the gap), the code miscalculated how many entries needed to be copied from the old array's null bitmap. This could result in trashing the array's data area (as seen in bug #5840 from Karsten Loesing), or worse. This has been broken since we first allowed the behavior of assigning to non-adjacent slices, in 8.2. Back-patch to all affected versions.
2011-01-17 18:38:52 +01:00
/* count/size of old array entries that will be replaced by slice */
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
if (slicelb > sliceub)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
nolditems = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
olditemsize = 0;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
else
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
{
nolditems = sliceub - slicelb + 1;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
olditemsize = array_nelems_size(oldarraydata + lenbefore,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
itemsbefore, oldarraybitmap,
nolditems,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
Fix miscalculation of itemsafter in array_set_slice(). If the slice to be assigned to was before the existing array lower bound (requiring at least one null element to spring into existence to fill the gap), the code miscalculated how many entries needed to be copied from the old array's null bitmap. This could result in trashing the array's data area (as seen in bug #5840 from Karsten Loesing), or worse. This has been broken since we first allowed the behavior of assigning to non-adjacent slices, in 8.2. Back-patch to all affected versions.
2011-01-17 18:38:52 +01:00
/* count/size of old array entries that will go after the slice */
itemsafter = oldub + 1 - Max(sliceub + 1, oldlb);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenafter = olddatasize - lenbefore - olditemsize;
}
newsize = overheadlen + olddatasize - olditemsize + newitemsize;
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
newarray = (ArrayType *) palloc0(newsize);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(newarray, newsize);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
newarray->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
newarray->dataoffset = newhasnulls ? overheadlen : 0;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
newarray->elemtype = ARR_ELEMTYPE(array);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy(ARR_DIMS(newarray), dim, ndim * sizeof(int));
memcpy(ARR_LBOUND(newarray), lb, ndim * sizeof(int));
if (ndim > 1)
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* here we do not need to cope with extension of the array; it would
* be a lot more complicated if we had to do so...
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_insert_slice(newarray, array, srcArray,
ndim, dim, lb,
lowerIndx, upperIndx,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
elmlen, elmbyval, elmalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
else
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* fill in data */
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
memcpy((char *) newarray + overheadlen,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
(char *) array + oldoverheadlen,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenbefore);
memcpy((char *) newarray + overheadlen + lenbefore,
ARR_DATA_PTR(srcArray),
newitemsize);
memcpy((char *) newarray + overheadlen + lenbefore + newitemsize,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
(char *) array + oldoverheadlen + lenbefore + olditemsize,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
lenafter);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* fill in nulls bitmap if needed */
if (newhasnulls)
{
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
bits8 *newnullbitmap = ARR_NULLBITMAP(newarray);
bits8 *oldnullbitmap = ARR_NULLBITMAP(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
/* Zero the bitmap to handle marking inserted positions null */
MemSet(newnullbitmap, 0, (nitems + 7) / 8);
array_bitmap_copy(newnullbitmap, addedbefore,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
oldnullbitmap, 0,
itemsbefore);
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
array_bitmap_copy(newnullbitmap, lowerIndx[0] - lb[0],
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ARR_NULLBITMAP(srcArray), 0,
nsrcitems);
Allow assignment to array elements not contiguous with those already present; intervening positions are filled with nulls. This behavior is required by SQL99 but was not implementable before 8.2 due to lack of support for nulls in arrays. I have only made it work for the one-dimensional case, which is all that SQL99 requires. It seems quite complex to get it right in higher dimensions, and since we never allowed extension at all in higher dimensions, I think that must count as a future feature addition not a bug fix.
2006-09-29 23:22:21 +02:00
array_bitmap_copy(newnullbitmap, addedbefore + itemsbefore + nolditems,
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
oldnullbitmap, itemsbefore + nolditems,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
itemsafter);
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Another directory that compiles with no errors, and few warnings
1996-11-06 11:32:10 +01:00
Rationalize the APIs of array element/slice access functions. The four functions array_ref, array_set, array_get_slice, array_set_slice have traditionally declared their array inputs and results as being of type "ArrayType *". This is a lie, and has been since Berkeley days, because they actually also support "fixed-length array" types such as "name" and "point"; not to mention that the inputs could be toasted. These values should be declared Datum instead to avoid confusion. The current coding already risks possible misoptimization by compilers, and it'll get worse when "expanded" array representations become a valid alternative. However, there's a fair amount of code using array_ref and array_set with arrays that *are* known to be ArrayType structures, and there might be more such places in third-party code. Rather than cluttering those call sites with PointerGetDatum/DatumGetArrayTypeP cruft, what I did was to rename the existing functions to array_get_element/array_set_element, fix their signatures, then reincarnate array_ref/array_set as backwards compatibility wrappers. array_get_slice/array_set_slice have no such constituency in the core code, and probably not in third-party code either, so I just changed their APIs.
2015-02-16 18:23:58 +01:00
return PointerGetDatum(newarray);
}
/*
* array_ref : backwards compatibility wrapper for array_get_element
*
* This only works for detoasted/flattened varlena arrays, since the array
* argument is declared as "ArrayType *". However there's enough code like
* that to justify preserving this API.
*/
Datum
array_ref(ArrayType *array, int nSubscripts, int *indx,
int arraytyplen, int elmlen, bool elmbyval, char elmalign,
bool *isNull)
{
return array_get_element(PointerGetDatum(array), nSubscripts, indx,
arraytyplen, elmlen, elmbyval, elmalign,
isNull);
}
/*
* array_set : backwards compatibility wrapper for array_set_element
*
* This only works for detoasted/flattened varlena arrays, since the array
* argument and result are declared as "ArrayType *". However there's enough
* code like that to justify preserving this API.
*/
ArrayType *
array_set(ArrayType *array, int nSubscripts, int *indx,
Datum dataValue, bool isNull,
int arraytyplen, int elmlen, bool elmbyval, char elmalign)
{
return DatumGetArrayTypeP(array_set_element(PointerGetDatum(array),
nSubscripts, indx,
dataValue, isNull,
arraytyplen,
elmlen, elmbyval, elmalign));
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
/*
* array_map()
*
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
* Map an array through an arbitrary expression. Return a new array with
* the same dimensions and each source element transformed by the given,
* already-compiled expression. Each source element is placed in the
* innermost_caseval/innermost_casenull fields of the ExprState.
Convert array_map to use new fmgr interface.
2000-05-29 23:02:32 +02:00
*
* Parameters are:
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
* * arrayd: Datum representing array argument.
* * exprstate: ExprState representing the per-element transformation.
* * econtext: context for expression evaluation.
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* * retType: OID of element type of output array. This must be the same as,
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
* or binary-compatible with, the result type of the expression. It might
* be different from the input array's element type.
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* * amstate: workspace for array_map. Must be zeroed by caller before
array_map can't use the fn_extra field of the provided fcinfo struct as its private storage, because that belongs to the function that it is supposed to call. Per report from Ezequiel Tolnay.
2005-03-24 22:50:38 +01:00
* first call, and not touched after that.
*
* It is legitimate to pass a freshly-zeroed ArrayMapState on each call,
* but better performance can be had if the state can be preserved across
* a series of calls.
Convert array_map to use new fmgr interface.
2000-05-29 23:02:32 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* NB: caller must assure that input array is not NULL. NULL elements in
* the array are OK however.
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
* NB: caller should be running in econtext's per-tuple memory context.
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
*/
Convert array_map to use new fmgr interface.
2000-05-29 23:02:32 +02:00
Datum
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
array_map(Datum arrayd,
ExprState *exprstate, ExprContext *econtext,
Oid retType, ArrayMapState *amstate)
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
{
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
AnyArrayType *v = DatumGetAnyArrayP(arrayd);
pgindent run over code.
1999-05-25 18:15:34 +02:00
ArrayType *result;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Datum *values;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nulls;
pgindent run over code.
1999-05-25 18:15:34 +02:00
int *dim;
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
int ndim;
int nitems;
int i;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int32 nbytes = 0;
int32 dataoffset;
bool hasnulls;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Oid inpType;
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
int inp_typlen;
bool inp_typbyval;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
char inp_typalign;
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
int typlen;
bool typbyval;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
char typalign;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter iter;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ArrayMetaState *inp_extra;
ArrayMetaState *ret_extra;
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
Datum *transform_source = exprstate->innermost_caseval;
bool *transform_source_isnull = exprstate->innermost_casenull;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
inpType = AARR_ELEMTYPE(v);
ndim = AARR_NDIM(v);
dim = AARR_DIMS(v);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
nitems = ArrayGetNItems(ndim, dim);
I have two patches for 6.5.0: arrayfuncs.patch fixes a small bug in my previous patches for arrays array-regress.patch adds _bpchar and _varchar to regression tests -- Massimo Dal Zotto
1999-05-05 23:38:40 +02:00
/* Check for empty array */
pgindent run over code.
1999-05-25 18:15:34 +02:00
if (nitems <= 0)
array_map failed to insert correct result type in an empty array. Per example from Florian Pflug.
2004-12-17 21:58:26 +01:00
{
/* Return empty array */
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
return PointerGetDatum(construct_empty_array(retType));
array_map failed to insert correct result type in an empty array. Per example from Florian Pflug.
2004-12-17 21:58:26 +01:00
}
Convert array_map to use new fmgr interface.
2000-05-29 23:02:32 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We arrange to look up info about input and return element types only
* once per series of calls, assuming the element type doesn't change
* underneath us.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
array_map can't use the fn_extra field of the provided fcinfo struct as its private storage, because that belongs to the function that it is supposed to call. Per report from Ezequiel Tolnay.
2005-03-24 22:50:38 +01:00
inp_extra = &amstate->inp_extra;
ret_extra = &amstate->ret_extra;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
if (inp_extra->element_type != inpType)
{
get_typlenbyvalalign(inpType,
&inp_extra->typlen,
&inp_extra->typbyval,
&inp_extra->typalign);
inp_extra->element_type = inpType;
}
inp_typlen = inp_extra->typlen;
inp_typbyval = inp_extra->typbyval;
inp_typalign = inp_extra->typalign;
if (ret_extra->element_type != retType)
{
get_typlenbyvalalign(retType,
&ret_extra->typlen,
&ret_extra->typbyval,
&ret_extra->typalign);
ret_extra->element_type = retType;
}
typlen = ret_extra->typlen;
typbyval = ret_extra->typbyval;
typalign = ret_extra->typalign;
I have two patches for 6.5.0: arrayfuncs.patch fixes a small bug in my previous patches for arrays array-regress.patch adds _bpchar and _varchar to regression tests -- Massimo Dal Zotto
1999-05-05 23:38:40 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Allocate temporary arrays for new values */
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
values = (Datum *) palloc(nitems * sizeof(Datum));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nulls = (bool *) palloc(nitems * sizeof(bool));
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
/* Loop over source data */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&iter, v);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
hasnulls = false;
pgindent run over code.
1999-05-25 18:15:34 +02:00
for (i = 0; i < nitems; i++)
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get source element, checking for NULL */
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
*transform_source =
array_iter_next(&iter, transform_source_isnull, i,
inp_typlen, inp_typbyval, inp_typalign);
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
/* Apply the given expression to source element */
values[i] = ExecEvalExpr(exprstate, econtext, &nulls[i]);
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
if (nulls[i])
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
hasnulls = true;
else
{
/* Ensure data is not toasted */
if (typlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM(values[i]));
/* Update total result size */
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
nbytes = att_addlength_datum(nbytes, typlen, values[i]);
nbytes = att_align_nominal(nbytes, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* check for overflow of total request */
if (!AllocSizeIsValid(nbytes))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
}
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
}
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
/* Allocate and fill the result array */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (hasnulls)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndim, nitems);
nbytes += dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes += ARR_OVERHEAD_NONULLS(ndim);
}
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
result = (ArrayType *) palloc0(nbytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(result, nbytes);
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
result->ndim = ndim;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
result->dataoffset = dataoffset;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
result->elemtype = retType;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
memcpy(ARR_DIMS(result), AARR_DIMS(v), ndim * sizeof(int));
memcpy(ARR_LBOUND(result), AARR_LBOUND(v), ndim * sizeof(int));
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
CopyArrayEls(result,
values, nulls, nitems,
typlen, typbyval, typalign,
false);
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
/*
* Note: do not risk trying to pfree the results of the called expression
*/
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
pfree(values);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
pfree(nulls);
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
return PointerGetDatum(result);
here are some patches for 6.5.0 which I already submitted but have never been applied. The patches are in the .tar.gz attachment at the end: varchar-array.patch this patch adds support for arrays of bpchar() and varchar(), which where always missing from postgres. These datatypes can be used to replace the _char4, _char8, etc., which were dropped some time ago. block-size.patch this patch fixes many errors in the parser and other program which happen with very large query statements (> 8K) when using a page size larger than 8192. This patch is needed if you want to submit queries larger than 8K. Postgres supports tuples up to 32K but you can't insert them because you can't submit queries larger than 8K. My patch fixes this problem. The patch also replaces all the occurrences of `8192' and `1<<13' in the sources with the proper constants defined in include files. You should now never find 8192 hardwired in C code, just to make code clearer. -- Massimo Dal Zotto
1999-05-03 21:10:48 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
pgindent run. Make it all clean.
2001-03-22 05:01:46 +01:00
* construct_array --- simple method for constructing an array object
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*
* elems: array of Datum items to become the array contents
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* (NULL element values are not supported).
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* nelems: number of items
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
* elmtype, elmlen, elmbyval, elmalign: info for the datatype of the items
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*
* A palloc'd 1-D array object is constructed and returned. Note that
* elem values will be copied into the object even if pass-by-ref type.
Make construct_[md_]array return a valid empty array for zero-size input. If construct_array() or construct_md_array() were given a dimension of zero, they'd produce an array that contains no elements but has positive dimension. This violates a general expectation that empty arrays should have ndims = 0; in particular, while arrays like this print as empty, they don't compare equal to other empty arrays. Up to now we've expected callers to avoid making such calls and instead be careful to call construct_empty_array() if there would be no elements. But this has always been an easily missed case, and we've repeatedly had to fix callers to do it right. In bug #14826, Erwin Brandstetter pointed out yet another such oversight, in ts_lexize(); and a bit of examination of other call sites found at least two more with similar issues. So let's fix the problem centrally and permanently by changing these two functions to construct a proper zero-D empty array whenever the array would be empty. This renders a few explicit calls of construct_empty_array() redundant, but the only such place I found that really seemed worth changing was in ExecEvalArrayExpr(). Although this fixes some very old bugs, no back-patch: the problem is pretty minor and the risk of changing behavior seems to outweigh the benefit in stable branches. Discussion: https://postgr.es/m/20170923125723.1448.39412@wrigleys.postgresql.org Discussion: https://postgr.es/m/20570.1506198383@sss.pgh.pa.us
2017-09-25 17:55:24 +02:00
* Also note the result will be 0-D not 1-D if nelems = 0.
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
*
* NOTE: it would be cleaner to look up the elmlen/elmbval/elmalign info
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
* from the system catalogs, given the elmtype. However, the caller is
* in a better position to cache this info across multiple uses, or even
* to hard-wire values if the element type is hard-wired.
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*/
pgindent run on all C files. Java run to follow. initdb/regression tests pass.
2001-10-25 07:50:21 +02:00
ArrayType *
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
construct_array(Datum *elems, int nelems,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Oid elmtype,
int elmlen, bool elmbyval, char elmalign)
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
int dims[1];
int lbs[1];
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
dims[0] = nelems;
lbs[0] = 1;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
return construct_md_array(elems, NULL, 1, dims, lbs,
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
elmtype, elmlen, elmbyval, elmalign);
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
* construct_md_array --- simple method for constructing an array object
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* with arbitrary dimensions and possible NULLs
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*
* elems: array of Datum items to become the array contents
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nulls: array of is-null flags (can be NULL if no nulls)
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
* ndims: number of dimensions
* dims: integer array with size of each dimension
* lbs: integer array with lower bound of each dimension
* elmtype, elmlen, elmbyval, elmalign: info for the datatype of the items
*
* A palloc'd ndims-D array object is constructed and returned. Note that
* elem values will be copied into the object even if pass-by-ref type.
Make construct_[md_]array return a valid empty array for zero-size input. If construct_array() or construct_md_array() were given a dimension of zero, they'd produce an array that contains no elements but has positive dimension. This violates a general expectation that empty arrays should have ndims = 0; in particular, while arrays like this print as empty, they don't compare equal to other empty arrays. Up to now we've expected callers to avoid making such calls and instead be careful to call construct_empty_array() if there would be no elements. But this has always been an easily missed case, and we've repeatedly had to fix callers to do it right. In bug #14826, Erwin Brandstetter pointed out yet another such oversight, in ts_lexize(); and a bit of examination of other call sites found at least two more with similar issues. So let's fix the problem centrally and permanently by changing these two functions to construct a proper zero-D empty array whenever the array would be empty. This renders a few explicit calls of construct_empty_array() redundant, but the only such place I found that really seemed worth changing was in ExecEvalArrayExpr(). Although this fixes some very old bugs, no back-patch: the problem is pretty minor and the risk of changing behavior seems to outweigh the benefit in stable branches. Discussion: https://postgr.es/m/20170923125723.1448.39412@wrigleys.postgresql.org Discussion: https://postgr.es/m/20570.1506198383@sss.pgh.pa.us
2017-09-25 17:55:24 +02:00
* Also note the result will be 0-D not ndims-D if any dims[i] = 0.
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
*
* NOTE: it would be cleaner to look up the elmlen/elmbval/elmalign info
* from the system catalogs, given the elmtype. However, the caller is
* in a better position to cache this info across multiple uses, or even
* to hard-wire values if the element type is hard-wired.
*/
ArrayType *
construct_md_array(Datum *elems,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nulls,
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
int ndims,
int *dims,
int *lbs,
Oid elmtype, int elmlen, bool elmbyval, char elmalign)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
{
ArrayType *result;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool hasnulls;
int32 nbytes;
int32 dataoffset;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int i;
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
int nelems;
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
if (ndims < 0) /* we do allow zero-dimension arrays */
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("invalid number of dimensions: %d", ndims)));
if (ndims > MAXDIM)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
Message editing: remove gratuitous variations in message wording, standardize terms, add some clarifications, fix some untranslatable attempts at dynamic message building.
2003-09-25 08:58:07 +02:00
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
ndims, MAXDIM)));
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
nelems = ArrayGetNItems(ndims, dims);
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Make construct_[md_]array return a valid empty array for zero-size input. If construct_array() or construct_md_array() were given a dimension of zero, they'd produce an array that contains no elements but has positive dimension. This violates a general expectation that empty arrays should have ndims = 0; in particular, while arrays like this print as empty, they don't compare equal to other empty arrays. Up to now we've expected callers to avoid making such calls and instead be careful to call construct_empty_array() if there would be no elements. But this has always been an easily missed case, and we've repeatedly had to fix callers to do it right. In bug #14826, Erwin Brandstetter pointed out yet another such oversight, in ts_lexize(); and a bit of examination of other call sites found at least two more with similar issues. So let's fix the problem centrally and permanently by changing these two functions to construct a proper zero-D empty array whenever the array would be empty. This renders a few explicit calls of construct_empty_array() redundant, but the only such place I found that really seemed worth changing was in ExecEvalArrayExpr(). Although this fixes some very old bugs, no back-patch: the problem is pretty minor and the risk of changing behavior seems to outweigh the benefit in stable branches. Discussion: https://postgr.es/m/20170923125723.1448.39412@wrigleys.postgresql.org Discussion: https://postgr.es/m/20570.1506198383@sss.pgh.pa.us
2017-09-25 17:55:24 +02:00
/* if ndims <= 0 or any dims[i] == 0, return empty array */
if (nelems <= 0)
return construct_empty_array(elmtype);
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
/* compute required space */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nbytes = 0;
hasnulls = false;
for (i = 0; i < nelems; i++)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nulls && nulls[i])
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
hasnulls = true;
continue;
Ensure that values stored within arrays are not toasted, per discussion on pghackers a few days ago.
2000-07-27 05:50:52 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* make sure data is not toasted */
if (elmlen == -1)
elems[i] = PointerGetDatum(PG_DETOAST_DATUM(elems[i]));
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
nbytes = att_addlength_datum(nbytes, elmlen, elems[i]);
nbytes = att_align_nominal(nbytes, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* check for overflow of total request */
if (!AllocSizeIsValid(nbytes))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Allocate and initialize result array */
if (hasnulls)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, nelems);
nbytes += dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes += ARR_OVERHEAD_NONULLS(ndims);
}
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
result = (ArrayType *) palloc0(nbytes);
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(result, nbytes);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
result->ndim = ndims;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
result->dataoffset = dataoffset;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
result->elemtype = elmtype;
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
memcpy(ARR_DIMS(result), dims, ndims * sizeof(int));
memcpy(ARR_LBOUND(result), lbs, ndims * sizeof(int));
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
CopyArrayEls(result,
elems, nulls, nelems,
elmlen, elmbyval, elmalign,
false);
return result;
}
/*
* construct_empty_array --- make a zero-dimensional array of given type
*/
ArrayType *
construct_empty_array(Oid elmtype)
{
ArrayType *result;
Fix array- and path-creating functions to ensure padding bytes are zeroes. Per recent discussion, it's important for all computed datums (not only the results of input functions) to not contain any ill-defined (uninitialized) bits. Failing to ensure that can result in equal() reporting that semantically indistinguishable Consts are not equal, which in turn leads to bizarre and undesirable planner behavior, such as in a recent example from David Johnston. We might eventually try to fix this in a general manner by allowing datatypes to define identity-testing functions, but for now the path of least resistance is to expect datatypes to force all unused bits into consistent states. Per some testing by Noah Misch, array and path functions seem to be the only ones presenting risks at the moment, so I looked through all the functions in adt/array*.c and geo_ops.c and fixed them as necessary. In the array functions, the easiest/safest fix is to allocate result arrays with palloc0 instead of palloc. Possibly in future someone will want to look into whether we can just zero the padding bytes, but that looks too complex for a back-patchable fix. In the path functions, we already had a precedent in path_in for just zeroing the one known pad field, so duplicate that code as needed. Back-patch to all supported branches.
2011-04-27 19:58:36 +02:00
result = (ArrayType *) palloc0(sizeof(ArrayType));
Replace direct assignments to VARATT_SIZEP(x) with SET_VARSIZE(x, len). Get rid of VARATT_SIZE and VARATT_DATA, which were simply redundant with VARSIZE and VARDATA, and as a consequence almost no code was using the longer names. Rename the length fields of struct varlena and various derived structures to catch anyplace that was accessing them directly; and clean up various places so caught. In itself this patch doesn't change any behavior at all, but it is necessary infrastructure if we hope to play any games with the representation of varlena headers. Greg Stark and Tom Lane
2007-02-28 00:48:10 +01:00
SET_VARSIZE(result, sizeof(ArrayType));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
result->ndim = 0;
result->dataoffset = 0;
result->elemtype = elmtype;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
return result;
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
/*
* construct_empty_expanded_array: make an empty expanded array
* given only type information. (metacache can be NULL if not needed.)
*/
ExpandedArrayHeader *
construct_empty_expanded_array(Oid element_type,
MemoryContext parentcontext,
ArrayMetaState *metacache)
{
ArrayType *array = construct_empty_array(element_type);
Datum d;
d = expand_array(PointerGetDatum(array), parentcontext, metacache);
pfree(array);
return (ExpandedArrayHeader *) DatumGetEOHP(d);
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* deconstruct_array --- simple method for extracting data from an array
*
* array: array object to examine (must not be NULL)
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
* elmtype, elmlen, elmbyval, elmalign: info for the datatype of the items
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* elemsp: return value, set to point to palloc'd array of Datum values
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* nullsp: return value, set to point to palloc'd array of isnull markers
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* nelemsp: return value, set to number of extracted values
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* The caller may pass nullsp == NULL if it does not support NULLs in the
* array. Note that this produces a very uninformative error message,
* so do it only in cases where a NULL is really not expected.
*
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
* If array elements are pass-by-ref data type, the returned Datums will
* be pointers into the array object.
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
*
* NOTE: it would be cleaner to look up the elmlen/elmbval/elmalign info
* from the system catalogs, given the elmtype. However, in most current
* uses the type is hard-wired into the caller and so we can save a lookup
* cycle by hard-wiring the type info as well.
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*/
void
deconstruct_array(ArrayType *array,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Oid elmtype,
int elmlen, bool elmbyval, char elmalign,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Datum **elemsp, bool **nullsp, int *nelemsp)
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
{
Datum *elems;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool *nulls;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int nelems;
char *p;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *bitmap;
int bitmask;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
int i;
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Assert(ARR_ELEMTYPE(array) == elmtype);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
nelems = ArrayGetNItems(ARR_NDIM(array), ARR_DIMS(array));
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*elemsp = elems = (Datum *) palloc(nelems * sizeof(Datum));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (nullsp)
Add array_contains_nulls() function in arrayfuncs.c. This will support fixing contrib/intarray (and probably other places) so that they don't have to fail on arrays that contain a null bitmap but no live null entries.
2011-01-09 02:24:08 +01:00
*nullsp = nulls = (bool *) palloc0(nelems * sizeof(bool));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else
nulls = NULL;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*nelemsp = nelems;
p = ARR_DATA_PTR(array);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bitmap = ARR_NULLBITMAP(array);
bitmask = 1;
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
for (i = 0; i < nelems; i++)
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get source element, checking for NULL */
if (bitmap && (*bitmap & bitmask) == 0)
{
elems[i] = (Datum) 0;
if (nulls)
nulls[i] = true;
else
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("null array element not allowed in this context")));
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
else
{
elems[i] = fetch_att(p, elmbyval, elmlen);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
p = att_addlength_pointer(p, elmlen, p);
p = (char *) att_align_nominal(p, elmalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
/* advance bitmap pointer if any */
if (bitmap)
{
bitmask <<= 1;
if (bitmask == 0x100)
{
bitmap++;
bitmask = 1;
}
}
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
}
}
Add array_contains_nulls() function in arrayfuncs.c. This will support fixing contrib/intarray (and probably other places) so that they don't have to fail on arrays that contain a null bitmap but no live null entries.
2011-01-09 02:24:08 +01:00
/*
* array_contains_nulls --- detect whether an array has any null elements
*
* This gives an accurate answer, whereas testing ARR_HASNULL only tells
* if the array *might* contain a null.
*/
bool
array_contains_nulls(ArrayType *array)
{
int nelems;
bits8 *bitmap;
int bitmask;
/* Easy answer if there's no null bitmap */
if (!ARR_HASNULL(array))
return false;
nelems = ArrayGetNItems(ARR_NDIM(array), ARR_DIMS(array));
bitmap = ARR_NULLBITMAP(array);
/* check whole bytes of the bitmap byte-at-a-time */
while (nelems >= 8)
{
if (*bitmap != 0xFF)
return true;
bitmap++;
nelems -= 8;
}
/* check last partial byte */
bitmask = 1;
while (nelems > 0)
{
if ((*bitmap & bitmask) == 0)
return true;
bitmask <<= 1;
nelems--;
}
return false;
}
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* array_eq :
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
* compares two arrays for equality
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* result :
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
* returns true if the arrays are equal, false otherwise.
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
*
* Note: we do not use array_cmp here, since equality may be meaningful in
* datatypes that don't have a total ordering (and hence no btree support).
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Datum
array_eq(PG_FUNCTION_ARGS)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array1 = PG_GETARG_ANY_ARRAY_P(0);
AnyArrayType *array2 = PG_GETARG_ANY_ARRAY_P(1);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
Oid collation = PG_GET_COLLATION();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
int ndims1 = AARR_NDIM(array1);
int ndims2 = AARR_NDIM(array2);
int *dims1 = AARR_DIMS(array1);
int *dims2 = AARR_DIMS(array2);
int *lbs1 = AARR_LBOUND(array1);
int *lbs2 = AARR_LBOUND(array2);
Oid element_type = AARR_ELEMTYPE(array1);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
bool result = true;
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
int nitems;
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
TypeCacheEntry *typentry;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int typlen;
bool typbyval;
char typalign;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter it1;
array_iter it2;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int i;
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (element_type != AARR_ELEMTYPE(array2))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("cannot compare arrays of different element types")));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
/* fast path if the arrays do not have the same dimensionality */
if (ndims1 != ndims2 ||
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
memcmp(dims1, dims2, ndims1 * sizeof(int)) != 0 ||
memcmp(lbs1, lbs2, ndims1 * sizeof(int)) != 0)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
result = false;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
else
{
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We arrange to look up the equality function only once per series of
* calls, assuming the element type doesn't change underneath us. The
* typcache is used so that we have no memory leakage when being used
* as an index support function.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typentry = lookup_type_cache(element_type,
TYPECACHE_EQ_OPR_FINFO);
if (!OidIsValid(typentry->eq_opr_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("could not identify an equality operator for type %s",
format_type_be(element_type))));
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
fcinfo->flinfo->fn_extra = (void *) typentry;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* apply the operator to each pair of array elements.
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
collation, NULL, NULL);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/* Loop over source data */
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
nitems = ArrayGetNItems(ndims1, dims1);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&it1, array1);
array_iter_setup(&it2, array2);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
for (i = 0; i < nitems; i++)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
pgindent run.
2003-08-04 02:43:34 +02:00
Datum elt1;
Datum elt2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool isnull1;
bool isnull2;
pgindent run.
2003-08-04 02:43:34 +02:00
bool oprresult;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get elements, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
elt1 = array_iter_next(&it1, &isnull1, i,
typlen, typbyval, typalign);
elt2 = array_iter_next(&it2, &isnull2, i,
typlen, typbyval, typalign);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* We consider two NULLs equal; NULL and not-NULL are unequal.
*/
if (isnull1 && isnull2)
continue;
if (isnull1 || isnull2)
{
result = false;
break;
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
* Apply the operator to the element pair; treat NULL as false
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt1;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = elt2;
locfcinfo->args[1].isnull = false;
locfcinfo->isnull = false;
oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo));
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
if (locfcinfo->isnull || !oprresult)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
result = false;
break;
}
}
}
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array1, 0);
AARR_FREE_IF_COPY(array2, 1);
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
PG_RETURN_BOOL(result);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*-----------------------------------------------------------------------------
* array-array bool operators:
* Given two arrays, iterate comparison operators
* over the array. Uses logic similar to text comparison
* functions, except element-by-element instead of
* character-by-character.
*----------------------------------------------------------------------------
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Datum
array_ne(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(!DatumGetBool(array_eq(fcinfo)));
}
Array mega-patch. Joe Conway
2003-06-25 01:14:49 +02:00
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Datum
array_lt(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(array_cmp(fcinfo) < 0);
}
Datum
array_gt(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(array_cmp(fcinfo) > 0);
}
Datum
array_le(PG_FUNCTION_ARGS)
{
PG_RETURN_BOOL(array_cmp(fcinfo) <= 0);
}
Datum
array_ge(PG_FUNCTION_ARGS)
Back out array mega-patch. Joe Conway
2003-06-25 23:30:34 +02:00
{
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
PG_RETURN_BOOL(array_cmp(fcinfo) >= 0);
}
Datum
btarraycmp(PG_FUNCTION_ARGS)
{
PG_RETURN_INT32(array_cmp(fcinfo));
Back out array mega-patch. Joe Conway
2003-06-25 23:30:34 +02:00
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* array_cmp()
* Internal comparison function for arrays.
*
* Returns -1, 0 or 1
*/
static int
array_cmp(FunctionCallInfo fcinfo)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array1 = PG_GETARG_ANY_ARRAY_P(0);
AnyArrayType *array2 = PG_GETARG_ANY_ARRAY_P(1);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
Oid collation = PG_GET_COLLATION();
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
int ndims1 = AARR_NDIM(array1);
int ndims2 = AARR_NDIM(array2);
int *dims1 = AARR_DIMS(array1);
int *dims2 = AARR_DIMS(array2);
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
int nitems1 = ArrayGetNItems(ndims1, dims1);
int nitems2 = ArrayGetNItems(ndims2, dims2);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Oid element_type = AARR_ELEMTYPE(array1);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int result = 0;
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
TypeCacheEntry *typentry;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int typlen;
bool typbyval;
char typalign;
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
int min_nitems;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter it1;
array_iter it2;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int i;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (element_type != AARR_ELEMTYPE(array2))
Error message editing in utils/adt. Again thanks to Joe Conway for doing the bulk of the heavy lifting ...
2003-07-27 06:53:12 +02:00
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
errmsg("cannot compare arrays of different element types")));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
Standard pgindent run for 8.1.
2005-10-15 04:49:52 +02:00
* We arrange to look up the comparison function only once per series of
* calls, assuming the element type doesn't change underneath us. The
* typcache is used so that we have no memory leakage when being used as
* an index support function.
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
typentry->type_id != element_type)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typentry = lookup_type_cache(element_type,
TYPECACHE_CMP_PROC_FINFO);
if (!OidIsValid(typentry->cmp_proc_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("could not identify a comparison function for type %s",
format_type_be(element_type))));
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
fcinfo->flinfo->fn_extra = (void *) typentry;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
/*
* apply the operator to each pair of array elements.
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->cmp_proc_finfo, 2,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
collation, NULL, NULL);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
/* Loop over source data */
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
min_nitems = Min(nitems1, nitems2);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&it1, array1);
array_iter_setup(&it2, array2);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
for (i = 0; i < min_nitems; i++)
{
Datum elt1;
Datum elt2;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bool isnull1;
bool isnull2;
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
int32 cmpresult;
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Get elements, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
elt1 = array_iter_next(&it1, &isnull1, i, typlen, typbyval, typalign);
elt2 = array_iter_next(&it2, &isnull2, i, typlen, typbyval, typalign);
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* We consider two NULLs equal; NULL > not-NULL.
*/
if (isnull1 && isnull2)
continue;
if (isnull1)
{
/* arg1 is greater than arg2 */
result = 1;
break;
}
if (isnull2)
{
/* arg1 is less than arg2 */
result = -1;
break;
}
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
/* Compare the pair of elements */
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt1;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = elt2;
locfcinfo->args[1].isnull = false;
cmpresult = DatumGetInt32(FunctionCallInvoke(locfcinfo));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
/* We don't expect comparison support functions to return null */
Assert(!locfcinfo->isnull);
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
if (cmpresult == 0)
continue; /* equal */
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Create a 'type cache' that keeps track of the data needed for any particular datatype by array_eq and array_cmp; use this to solve problems with memory leaks in array indexing support. The parser's equality_oper and ordering_oper routines also use the cache. Change the operator search algorithms to look for appropriate btree or hash index opclasses, instead of assuming operators named '<' or '=' have the right semantics. (ORDER BY ASC/DESC now also look at opclasses, instead of assuming '<' and '>' are the right things.) Add several more index opclasses so that there is no regression in functionality for base datatypes. initdb forced due to catalog additions.
2003-08-17 21:58:06 +02:00
if (cmpresult < 0)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
Rewrite array_cmp to not depend on deconstruct_array. Should be a little faster, but more importantly does not leak memory. Still needs more work though, per my recent note to pgsql-hackers.
2003-08-15 02:22:26 +02:00
/* arg1 is less than arg2 */
result = -1;
break;
}
else
{
/* arg1 is greater than arg2 */
result = 1;
break;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
}
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
/*
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* If arrays contain same data (up to end of shorter one), apply
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* additional rules to sort by dimensionality. The relative significance
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
* of the different bits of information is historical; mainly we just care
* that we don't say "equal" for arrays of different dimensionality.
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
*/
if (result == 0)
{
if (nitems1 != nitems2)
result = (nitems1 < nitems2) ? -1 : 1;
else if (ndims1 != ndims2)
result = (ndims1 < ndims2) ? -1 : 1;
else
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
for (i = 0; i < ndims1; i++)
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
{
if (dims1[i] != dims2[i])
{
result = (dims1[i] < dims2[i]) ? -1 : 1;
break;
}
}
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (result == 0)
{
int *lbound1 = AARR_LBOUND(array1);
int *lbound2 = AARR_LBOUND(array2);
for (i = 0; i < ndims1; i++)
{
if (lbound1[i] != lbound2[i])
{
result = (lbound1[i] < lbound2[i]) ? -1 : 1;
break;
}
}
}
Change array comparison rules to consider dimensionality information, not only the array contents, before claiming two arrays are equal. Per recent discussion.
2005-11-19 20:44:55 +01:00
}
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array1, 0);
AARR_FREE_IF_COPY(array2, 1);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
return result;
}
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
/*-----------------------------------------------------------------------------
* array hashing
* Hash the elements and combine the results.
*----------------------------------------------------------------------------
*/
Datum
hash_array(PG_FUNCTION_ARGS)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 1);
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
int ndims = AARR_NDIM(array);
int *dims = AARR_DIMS(array);
Oid element_type = AARR_ELEMTYPE(array);
Improve hash_array() logic for combining hash values. The new logic is less vulnerable to transpositions. This invalidates the contents of hash indexes built with the old functions; hence, bump catversion. Dean Rasheed
2011-05-23 21:17:18 +02:00
uint32 result = 1;
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
int i;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter iter;
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
/*
* We arrange to look up the hash function only once per series of calls,
* assuming the element type doesn't change underneath us. The typcache
* is used so that we have no memory leakage when being used as an index
* support function.
*/
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
{
typentry = lookup_type_cache(element_type,
TYPECACHE_HASH_PROC_FINFO);
if (!OidIsValid(typentry->hash_proc_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
errmsg("could not identify a hash function for type %s",
format_type_be(element_type))));
fcinfo->flinfo->fn_extra = (void *) typentry;
}
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
/*
* apply the hash function to each array element.
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->hash_proc_finfo, 1,
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
PG_GET_COLLATION(), NULL, NULL);
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
/* Loop over source data */
nitems = ArrayGetNItems(ndims, dims);
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&iter, array);
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
for (i = 0; i < nitems; i++)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Datum elt;
bool isnull;
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
uint32 elthash;
/* Get element, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
elt = array_iter_next(&iter, &isnull, i, typlen, typbyval, typalign);
if (isnull)
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
{
/* Treat nulls as having hashvalue 0 */
elthash = 0;
}
else
{
/* Apply the hash function */
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt;
locfcinfo->args[0].isnull = false;
elthash = DatumGetUInt32(FunctionCallInvoke(locfcinfo));
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
/* We don't expect hash functions to return null */
Assert(!locfcinfo->isnull);
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
}
/*
Improve hash_array() logic for combining hash values. The new logic is less vulnerable to transpositions. This invalidates the contents of hash indexes built with the old functions; hence, bump catversion. Dean Rasheed
2011-05-23 21:17:18 +02:00
* Combine hash values of successive elements by multiplying the
* current value by 31 and adding on the new element's hash value.
*
* The result is a sum in which each element's hash value is
* multiplied by a different power of 31. This is modulo 2^32
* arithmetic, and the powers of 31 modulo 2^32 form a cyclic group of
* order 2^27. So for arrays of up to 2^27 elements, each element's
* hash value is multiplied by a different (odd) number, resulting in
* a good mixing of all the elements' hash values.
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
*/
Improve hash_array() logic for combining hash values. The new logic is less vulnerable to transpositions. This invalidates the contents of hash indexes built with the old functions; hence, bump catversion. Dean Rasheed
2011-05-23 21:17:18 +02:00
result = (result << 5) - result + elthash;
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
}
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array, 0);
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
PG_RETURN_UINT32(result);
}
Introduce 64-bit hash functions with a 64-bit seed. This will be useful for hash partitioning, which needs a way to seed the hash functions to avoid problems such as a hash index on a hash partitioned table clumping all values into a small portion of the bucket space; it's also useful for anything that wants a 64-bit hash value rather than a 32-bit hash value. Just in case somebody wants a 64-bit hash value that is compatible with the existing 32-bit hash values, make the low 32-bits of the 64-bit hash value match the 32-bit hash value when the seed is 0. Robert Haas and Amul Sul Discussion: http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com
2017-09-01 04:21:21 +02:00
/*
* Returns 64-bit value by hashing a value to a 64-bit value, with a seed.
* Otherwise, similar to hash_array.
*/
Datum
hash_array_extended(PG_FUNCTION_ARGS)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array = PG_GETARG_ANY_ARRAY_P(0);
Introduce 64-bit hash functions with a 64-bit seed. This will be useful for hash partitioning, which needs a way to seed the hash functions to avoid problems such as a hash index on a hash partitioned table clumping all values into a small portion of the bucket space; it's also useful for anything that wants a 64-bit hash value rather than a 32-bit hash value. Just in case somebody wants a 64-bit hash value that is compatible with the existing 32-bit hash values, make the low 32-bits of the 64-bit hash value match the 32-bit hash value when the seed is 0. Robert Haas and Amul Sul Discussion: http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com
2017-09-01 04:21:21 +02:00
uint64 seed = PG_GETARG_INT64(1);
int ndims = AARR_NDIM(array);
int *dims = AARR_DIMS(array);
Oid element_type = AARR_ELEMTYPE(array);
uint64 result = 1;
int nitems;
TypeCacheEntry *typentry;
int typlen;
bool typbyval;
char typalign;
int i;
array_iter iter;
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
{
typentry = lookup_type_cache(element_type,
TYPECACHE_HASH_EXTENDED_PROC_FINFO);
if (!OidIsValid(typentry->hash_extended_proc_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
errmsg("could not identify an extended hash function for type %s",
format_type_be(element_type))));
fcinfo->flinfo->fn_extra = (void *) typentry;
}
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->hash_extended_proc_finfo, 2,
Introduce 64-bit hash functions with a 64-bit seed. This will be useful for hash partitioning, which needs a way to seed the hash functions to avoid problems such as a hash index on a hash partitioned table clumping all values into a small portion of the bucket space; it's also useful for anything that wants a 64-bit hash value rather than a 32-bit hash value. Just in case somebody wants a 64-bit hash value that is compatible with the existing 32-bit hash values, make the low 32-bits of the 64-bit hash value match the 32-bit hash value when the seed is 0. Robert Haas and Amul Sul Discussion: http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com
2017-09-01 04:21:21 +02:00
InvalidOid, NULL, NULL);
/* Loop over source data */
nitems = ArrayGetNItems(ndims, dims);
array_iter_setup(&iter, array);
for (i = 0; i < nitems; i++)
{
Datum elt;
bool isnull;
uint64 elthash;
/* Get element, checking for NULL */
elt = array_iter_next(&iter, &isnull, i, typlen, typbyval, typalign);
if (isnull)
{
elthash = 0;
}
else
{
/* Apply the hash function */
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = Int64GetDatum(seed);
locfcinfo->args[1].isnull = false;
elthash = DatumGetUInt64(FunctionCallInvoke(locfcinfo));
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
/* We don't expect hash functions to return null */
Assert(!locfcinfo->isnull);
Introduce 64-bit hash functions with a 64-bit seed. This will be useful for hash partitioning, which needs a way to seed the hash functions to avoid problems such as a hash index on a hash partitioned table clumping all values into a small portion of the bucket space; it's also useful for anything that wants a 64-bit hash value rather than a 32-bit hash value. Just in case somebody wants a 64-bit hash value that is compatible with the existing 32-bit hash values, make the low 32-bits of the 64-bit hash value match the 32-bit hash value when the seed is 0. Robert Haas and Amul Sul Discussion: http://postgr.es/m/CA+Tgmoafx2yoJuhCQQOL5CocEi-w_uG4S2xT0EtgiJnPGcHW3g@mail.gmail.com
2017-09-01 04:21:21 +02:00
}
result = (result << 5) - result + elthash;
}
AARR_FREE_IF_COPY(array, 0);
PG_RETURN_UINT64(result);
}
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
/*-----------------------------------------------------------------------------
* array overlap/containment comparisons
* These use the same methods of comparing array elements as array_eq.
* We consider only the elements of the arrays, ignoring dimensionality.
*----------------------------------------------------------------------------
*/
/*
* array_contain_compare :
* compares two arrays for overlap/containment
*
* When matchall is true, return true if all members of array1 are in array2.
* When matchall is false, return true if any members of array1 are in array2.
*/
static bool
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_contain_compare(AnyArrayType *array1, AnyArrayType *array2, Oid collation,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
bool matchall, void **fn_extra)
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
bool result = matchall;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
Oid element_type = AARR_ELEMTYPE(array1);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
TypeCacheEntry *typentry;
int nelems1;
Datum *values2;
bool *nulls2;
int nelems2;
int typlen;
bool typbyval;
char typalign;
int i;
int j;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter it1;
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (element_type != AARR_ELEMTYPE(array2))
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("cannot compare arrays of different element types")));
/*
* We arrange to look up the equality function only once per series of
* calls, assuming the element type doesn't change underneath us. The
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* typcache is used so that we have no memory leakage when being used as
* an index support function.
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
*/
typentry = (TypeCacheEntry *) *fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
{
typentry = lookup_type_cache(element_type,
TYPECACHE_EQ_OPR_FINFO);
if (!OidIsValid(typentry->eq_opr_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("could not identify an equality operator for type %s",
format_type_be(element_type))));
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
*fn_extra = (void *) typentry;
}
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
/*
* Since we probably will need to scan array2 multiple times, it's
* worthwhile to use deconstruct_array on it. We scan array1 the hard way
* however, since we very likely won't need to look at all of it.
*/
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (VARATT_IS_EXPANDED_HEADER(array2))
{
/* This should be safe even if input is read-only */
deconstruct_expanded_array(&(array2->xpn));
values2 = array2->xpn.dvalues;
nulls2 = array2->xpn.dnulls;
nelems2 = array2->xpn.nelems;
}
else
Don't read fields of a misaligned ExpandedObjectHeader or AnyArrayType. UBSan complains about this. Instead, cast to a suitable type requiring only 4-byte alignment. DatumGetAnyArrayP() already assumes one can cast between AnyArrayType and ArrayType, so this doesn't introduce a new assumption. Back-patch to 9.5, where AnyArrayType was introduced. Reviewed by Tom Lane. Discussion: https://postgr.es/m/20190629210334.GA1244217@rfd.leadboat.com
2019-07-01 02:34:17 +02:00
deconstruct_array((ArrayType *) array2,
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
element_type, typlen, typbyval, typalign,
&values2, &nulls2, &nelems2);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
/*
* Apply the comparison operator to each pair of array elements.
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2,
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
collation, NULL, NULL);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
/* Loop over source data */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
nelems1 = ArrayGetNItems(AARR_NDIM(array1), AARR_DIMS(array1));
array_iter_setup(&it1, array1);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
for (i = 0; i < nelems1; i++)
{
Datum elt1;
bool isnull1;
/* Get element, checking for NULL */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
elt1 = array_iter_next(&it1, &isnull1, i, typlen, typbyval, typalign);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
/*
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* We assume that the comparison operator is strict, so a NULL can't
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* match anything. XXX this diverges from the "NULL=NULL" behavior of
pgindent run for 8.2.
2006-10-04 02:30:14 +02:00
* array_eq, should we act like that?
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
*/
if (isnull1)
{
if (matchall)
{
result = false;
break;
}
continue;
}
for (j = 0; j < nelems2; j++)
{
Datum elt2 = values2[j];
Fix recently-introduced crash in array_contain_compare(). Silly oversight in commit 1dc5ebc9077ab742079ce5dac9a6664248d42916: when array2 is an expanded array, it might have array2->xpn.dnulls equal to NULL, indicating the array is known null-free. The code wasn't expecting that, because it formerly always used deconstruct_array() which always delivers a nulls array. Per bug #13334 from Regina Obe.
2015-05-23 00:36:48 +02:00
bool isnull2 = nulls2 ? nulls2[j] : false;
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
bool oprresult;
if (isnull2)
continue; /* can't match */
/*
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
* Apply the operator to the element pair; treat NULL as false
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt1;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = elt2;
locfcinfo->args[1].isnull = false;
locfcinfo->isnull = false;
oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo));
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
if (!locfcinfo->isnull && oprresult)
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
break;
}
if (j < nelems2)
{
/* found a match for elt1 */
if (!matchall)
{
result = true;
break;
}
}
else
{
/* no match for elt1 */
if (matchall)
{
result = false;
break;
}
}
}
return result;
}
Datum
arrayoverlap(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array1 = PG_GETARG_ANY_ARRAY_P(0);
AnyArrayType *array2 = PG_GETARG_ANY_ARRAY_P(1);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
Oid collation = PG_GET_COLLATION();
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
bool result;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
result = array_contain_compare(array1, array2, collation, false,
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array1, 0);
AARR_FREE_IF_COPY(array2, 1);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
PG_RETURN_BOOL(result);
}
Datum
arraycontains(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array1 = PG_GETARG_ANY_ARRAY_P(0);
AnyArrayType *array2 = PG_GETARG_ANY_ARRAY_P(1);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
Oid collation = PG_GET_COLLATION();
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
bool result;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
result = array_contain_compare(array2, array1, collation, true,
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array1, 0);
AARR_FREE_IF_COPY(array2, 1);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
PG_RETURN_BOOL(result);
}
Datum
arraycontained(PG_FUNCTION_ARGS)
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *array1 = PG_GETARG_ANY_ARRAY_P(0);
AnyArrayType *array2 = PG_GETARG_ANY_ARRAY_P(1);
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
Oid collation = PG_GET_COLLATION();
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
bool result;
Pass collations to functions in FunctionCallInfoData, not FmgrInfo. Since collation is effectively an argument, not a property of the function, FmgrInfo is really the wrong place for it; and this becomes critical in cases where a cached FmgrInfo is used for varying purposes that might need different collation settings. Fix by passing it in FunctionCallInfoData instead. In particular this allows a clean fix for bug #5970 (record_cmp not working). This requires touching a bit more code than the original method, but nobody ever thought that collations would not be an invasive patch...
2011-04-13 01:19:24 +02:00
result = array_contain_compare(array1, array2, collation, true,
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
&fcinfo->flinfo->fn_extra);
/* Avoid leaking memory when handed toasted input. */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AARR_FREE_IF_COPY(array1, 0);
AARR_FREE_IF_COPY(array2, 1);
If we're going to advertise the array overlap/containment operators, we probably should make them work reliably for all arrays. Fix code to handle NULLs and multidimensional arrays, move it into arrayfuncs.c. GIN is still restricted to indexing arrays with no null elements, however.
2006-09-10 22:14:20 +02:00
PG_RETURN_BOOL(result);
}
Add FOREACH IN ARRAY looping to plpgsql. (I'm not entirely sure that we've finished bikeshedding the syntax details, but the functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost and Tom Lane
2011-02-16 07:52:04 +01:00
/*-----------------------------------------------------------------------------
* Array iteration functions
* These functions are used to iterate efficiently through arrays
*-----------------------------------------------------------------------------
*/
/*
* array_create_iterator --- set up to iterate through an array
*
* If slice_ndim is zero, we will iterate element-by-element; the returned
* datums are of the array's element type.
*
* If slice_ndim is 1..ARR_NDIM(arr), we will iterate by slices: the
* returned datums are of the same array type as 'arr', but of size
* equal to the rightmost N dimensions of 'arr'.
*
* The passed-in array must remain valid for the lifetime of the iterator.
*/
ArrayIterator
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
array_create_iterator(ArrayType *arr, int slice_ndim, ArrayMetaState *mstate)
Add FOREACH IN ARRAY looping to plpgsql. (I'm not entirely sure that we've finished bikeshedding the syntax details, but the functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost and Tom Lane
2011-02-16 07:52:04 +01:00
{
ArrayIterator iterator = palloc0(sizeof(ArrayIteratorData));
/*
* Sanity-check inputs --- caller should have got this right already
*/
Assert(PointerIsValid(arr));
if (slice_ndim < 0 || slice_ndim > ARR_NDIM(arr))
elog(ERROR, "invalid arguments to array_create_iterator");
/*
* Remember basic info about the array and its element type
*/
iterator->arr = arr;
iterator->nullbitmap = ARR_NULLBITMAP(arr);
iterator->nitems = ArrayGetNItems(ARR_NDIM(arr), ARR_DIMS(arr));
array_offset() and array_offsets() These functions return the offset position or positions of a value in an array. Author: Pavel Stěhule Reviewed by: Jim Nasby
2015-03-18 20:01:34 +01:00
if (mstate != NULL)
{
Assert(mstate->element_type == ARR_ELEMTYPE(arr));
iterator->typlen = mstate->typlen;
iterator->typbyval = mstate->typbyval;
iterator->typalign = mstate->typalign;
}
else
get_typlenbyvalalign(ARR_ELEMTYPE(arr),
&iterator->typlen,
&iterator->typbyval,
&iterator->typalign);
Add FOREACH IN ARRAY looping to plpgsql. (I'm not entirely sure that we've finished bikeshedding the syntax details, but the functionality seems OK.) Pavel Stehule, reviewed by Stephen Frost and Tom Lane
2011-02-16 07:52:04 +01:00
/*
* Remember the slicing parameters.
*/
iterator->slice_ndim = slice_ndim;
if (slice_ndim > 0)
{
/*
* Get pointers into the array's dims and lbound arrays to represent
* the dims/lbound arrays of a slice. These are the same as the
* rightmost N dimensions of the array.
*/
iterator->slice_dims = ARR_DIMS(arr) + ARR_NDIM(arr) - slice_ndim;
iterator->slice_lbound = ARR_LBOUND(arr) + ARR_NDIM(arr) - slice_ndim;
/*
* Compute number of elements in a slice.
*/
iterator->slice_len = ArrayGetNItems(slice_ndim,
iterator->slice_dims);
/*
* Create workspace for building sub-arrays.
*/
iterator->slice_values = (Datum *)
palloc(iterator->slice_len * sizeof(Datum));
iterator->slice_nulls = (bool *)
palloc(iterator->slice_len * sizeof(bool));
}
/*
* Initialize our data pointer and linear element number. These will
* advance through the array during array_iterate().
*/
iterator->data_ptr = ARR_DATA_PTR(arr);
iterator->current_item = 0;
return iterator;
}
/*
* Iterate through the array referenced by 'iterator'.
*
* As long as there is another element (or slice), return it into
* *value / *isnull, and return true. Return false when no more data.
*/
bool
array_iterate(ArrayIterator iterator, Datum *value, bool *isnull)
{
/* Done if we have reached the end of the array */
if (iterator->current_item >= iterator->nitems)
return false;
if (iterator->slice_ndim == 0)
{
/*
* Scalar case: return one element.
*/
if (array_get_isnull(iterator->nullbitmap, iterator->current_item++))
{
*isnull = true;
*value = (Datum) 0;
}
else
{
/* non-NULL, so fetch the individual Datum to return */
char *p = iterator->data_ptr;
*isnull = false;
*value = fetch_att(p, iterator->typbyval, iterator->typlen);
/* Move our data pointer forward to the next element */
p = att_addlength_pointer(p, iterator->typlen, p);
p = (char *) att_align_nominal(p, iterator->typalign);
iterator->data_ptr = p;
}
}
else
{
/*
* Slice case: build and return an array of the requested size.
*/
ArrayType *result;
Datum *values = iterator->slice_values;
bool *nulls = iterator->slice_nulls;
char *p = iterator->data_ptr;
int i;
for (i = 0; i < iterator->slice_len; i++)
{
if (array_get_isnull(iterator->nullbitmap,
iterator->current_item++))
{
nulls[i] = true;
values[i] = (Datum) 0;
}
else
{
nulls[i] = false;
values[i] = fetch_att(p, iterator->typbyval, iterator->typlen);
/* Move our data pointer forward to the next element */
p = att_addlength_pointer(p, iterator->typlen, p);
p = (char *) att_align_nominal(p, iterator->typalign);
}
}
iterator->data_ptr = p;
result = construct_md_array(values,
nulls,
iterator->slice_ndim,
iterator->slice_dims,
iterator->slice_lbound,
ARR_ELEMTYPE(iterator->arr),
iterator->typlen,
iterator->typbyval,
iterator->typalign);
*isnull = false;
*value = PointerGetDatum(result);
}
return true;
}
/*
* Release an ArrayIterator data structure
*/
void
array_free_iterator(ArrayIterator iterator)
{
if (iterator->slice_ndim > 0)
{
pfree(iterator->slice_values);
pfree(iterator->slice_nulls);
}
pfree(iterator);
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/***************************************************************************/
/******************| Support Routines |*****************/
/***************************************************************************/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Check whether a specific array element is NULL
*
* nullbitmap: pointer to array's null bitmap (NULL if none)
* offset: 0-based linear element number of array element
*/
static bool
array_get_isnull(const bits8 *nullbitmap, int offset)
{
if (nullbitmap == NULL)
return false; /* assume not null */
if (nullbitmap[offset / 8] & (1 << (offset % 8)))
return false; /* not null */
return true;
}
/*
* Set a specific array element's null-bitmap entry
*
* nullbitmap: pointer to array's null bitmap (mustn't be NULL)
* offset: 0-based linear element number of array element
* isNull: null status to set
*/
static void
array_set_isnull(bits8 *nullbitmap, int offset, bool isNull)
{
int bitmask;
nullbitmap += offset / 8;
bitmask = 1 << (offset % 8);
if (isNull)
*nullbitmap &= ~bitmask;
else
*nullbitmap |= bitmask;
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Fetch array element at pointer, converted correctly to a Datum
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
* Caller must have handled case of NULL element
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
static Datum
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
ArrayCast(char *value, bool byval, int len)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Fix portability problems recently exposed by regression tests on Alphas. 1. Distinguish cases where a Datum representing a tuple datatype is an OID from cases where it is a pointer to TupleTableSlot, and make sure we use the right typlen in each case. 2. Make fetchatt() and related code support 8-byte by-value datatypes on machines where Datum is 8 bytes. Centralize knowledge of the available by-value datatype sizes in two macros in tupmacs.h, so that this will be easier if we ever have to do it again.
2000-12-28 00:59:14 +01:00
return fetch_att(value, byval, len);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
/*
* Copy datum to *dest and return total space used (including align padding)
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
* Caller must have handled case of NULL element
Revise aggregate functions per earlier discussions in pghackers. There's now only one transition value and transition function. NULL handling in aggregates is a lot cleaner. Also, use Numeric accumulators instead of integer accumulators for sum/avg on integer datatypes --- this avoids overflow at the cost of being a little slower. Implement VARIANCE() and STDDEV() aggregates in the standard backend. Also, enable new LIKE selectivity estimators by default. Unrelated change, but as long as I had to force initdb anyway...
2000-07-17 05:05:41 +02:00
*/
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
static int
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
ArrayCastAndSet(Datum src,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
int typlen,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
bool typbyval,
char typalign,
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
char *dest)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int inc;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
if (typlen > 0)
{
if (typbyval)
Fix portability problems recently exposed by regression tests on Alphas. 1. Distinguish cases where a Datum representing a tuple datatype is an OID from cases where it is a pointer to TupleTableSlot, and make sure we use the right typlen in each case. 2. Make fetchatt() and related code support 8-byte by-value datatypes on machines where Datum is 8 bytes. Centralize knowledge of the available by-value datatype sizes in two macros in tupmacs.h, so that this will be easier if we ever have to do it again.
2000-12-28 00:59:14 +01:00
store_att_byval(dest, src, typlen);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
else
Another batch of fmgr updates. I think I have gotten all old-style functions that take pass-by-value datatypes. Should be ready for port testing ...
2000-06-13 09:35:40 +02:00
memmove(dest, DatumGetPointer(src), typlen);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
inc = att_align_nominal(typlen, typalign);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
else
{
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Assert(!typbyval);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
inc = att_addlength_datum(0, typlen, src);
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
memmove(dest, DatumGetPointer(src), inc);
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
inc = att_align_nominal(inc, typalign);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
return inc;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Advance ptr over nitems array elements
*
* ptr: starting location in array
* offset: 0-based linear element number of first element (the one at *ptr)
* nullbitmap: start of array's null bitmap, or NULL if none
* nitems: number of array elements to advance over (>= 0)
* typlen, typbyval, typalign: storage parameters of array element datatype
*
* It is caller's responsibility to ensure that nitems is within range
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static char *
array_seek(char *ptr, int offset, bits8 *nullbitmap, int nitems,
int typlen, bool typbyval, char typalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int bitmask;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* easy if fixed-size elements and no NULLs */
if (typlen > 0 && !nullbitmap)
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
return ptr + nitems * ((Size) att_align_nominal(typlen, typalign));
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* seems worth having separate loops for NULL and no-NULLs cases */
if (nullbitmap)
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
nullbitmap += offset / 8;
bitmask = 1 << (offset % 8);
for (i = 0; i < nitems; i++)
{
if (*nullbitmap & bitmask)
{
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
ptr = att_addlength_pointer(ptr, typlen, ptr);
ptr = (char *) att_align_nominal(ptr, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
bitmask <<= 1;
if (bitmask == 0x100)
{
nullbitmap++;
bitmask = 1;
}
}
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
else
{
for (i = 0; i < nitems; i++)
{
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
ptr = att_addlength_pointer(ptr, typlen, ptr);
ptr = (char *) att_align_nominal(ptr, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
}
}
return ptr;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Compute total size of the nitems array elements starting at *ptr
*
* Parameters same as for array_seek
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
static int
array_nelems_size(char *ptr, int offset, bits8 *nullbitmap, int nitems,
int typlen, bool typbyval, char typalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
return array_seek(ptr, offset, nullbitmap, nitems,
typlen, typbyval, typalign) - ptr;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Copy nitems array elements from srcptr to destptr
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* destptr: starting destination location (must be enough room!)
* nitems: number of array elements to copy (>= 0)
* srcptr: starting location in source array
* offset: 0-based linear element number of first element (the one at *srcptr)
* nullbitmap: start of source array's null bitmap, or NULL if none
* typlen, typbyval, typalign: storage parameters of array element datatype
*
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
* Returns number of bytes copied
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*
* NB: this does not take care of setting up the destination's null bitmap!
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
static int
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_copy(char *destptr, int nitems,
char *srcptr, int offset, bits8 *nullbitmap,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int typlen, bool typbyval, char typalign)
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int numbytes;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
numbytes = array_nelems_size(srcptr, offset, nullbitmap, nitems,
typlen, typbyval, typalign);
memcpy(destptr, srcptr, numbytes);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
return numbytes;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/*
* Copy nitems null-bitmap bits from source to destination
*
* destbitmap: start of destination array's null bitmap (mustn't be NULL)
* destoffset: 0-based linear element number of first dest element
* srcbitmap: start of source array's null bitmap, or NULL if none
* srcoffset: 0-based linear element number of first source element
* nitems: number of bits to copy (>= 0)
*
* If srcbitmap is NULL then we assume the source is all-non-NULL and
* fill 1's into the destination bitmap. Note that only the specified
* bits in the destination map are changed, not any before or after.
*
* Note: this could certainly be optimized using standard bitblt methods.
* However, it's not clear that the typical Postgres array has enough elements
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* to make it worth worrying too much. For the moment, KISS.
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
*/
void
array_bitmap_copy(bits8 *destbitmap, int destoffset,
const bits8 *srcbitmap, int srcoffset,
int nitems)
{
int destbitmask,
destbitval,
srcbitmask,
srcbitval;
Assert(destbitmap);
if (nitems <= 0)
return; /* don't risk fetch off end of memory */
destbitmap += destoffset / 8;
destbitmask = 1 << (destoffset % 8);
destbitval = *destbitmap;
if (srcbitmap)
{
srcbitmap += srcoffset / 8;
srcbitmask = 1 << (srcoffset % 8);
srcbitval = *srcbitmap;
while (nitems-- > 0)
{
if (srcbitval & srcbitmask)
destbitval |= destbitmask;
else
destbitval &= ~destbitmask;
destbitmask <<= 1;
if (destbitmask == 0x100)
{
*destbitmap++ = destbitval;
destbitmask = 1;
if (nitems > 0)
destbitval = *destbitmap;
}
srcbitmask <<= 1;
if (srcbitmask == 0x100)
{
srcbitmap++;
srcbitmask = 1;
if (nitems > 0)
srcbitval = *srcbitmap;
}
}
if (destbitmask != 1)
*destbitmap = destbitval;
}
else
{
while (nitems-- > 0)
{
destbitval |= destbitmask;
destbitmask <<= 1;
if (destbitmask == 0x100)
{
*destbitmap++ = destbitval;
destbitmask = 1;
if (nitems > 0)
destbitval = *destbitmap;
}
}
if (destbitmask != 1)
*destbitmap = destbitval;
}
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Compute space needed for a slice of an array
*
* We assume the caller has verified that the slice coordinates are valid.
*/
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
static int
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_slice_size(char *arraydataptr, bits8 *arraynullsptr,
int ndim, int *dim, int *lb,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int *st, int *endp,
int typlen, bool typbyval, char typalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
int src_offset,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
span[MAXDIM],
prod[MAXDIM],
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
dist[MAXDIM],
indx[MAXDIM];
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
char *ptr;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
j,
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
inc;
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int count = 0;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
mda_get_range(ndim, span, st, endp);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
/* Pretty easy for fixed element length without nulls ... */
if (typlen > 0 && !arraynullsptr)
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
return ArrayGetNItems(ndim, span) * att_align_nominal(typlen, typalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* Else gotta do it the hard way */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
src_offset = ArrayGetOffset(ndim, dim, lb, st);
ptr = array_seek(arraydataptr, 0, arraynullsptr, src_offset,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
typlen, typbyval, typalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
mda_get_prod(ndim, dim, prod);
mda_get_offset_values(ndim, dist, prod, span);
for (i = 0; i < ndim; i++)
Arrays are toastable. (At least if you initdb, which I didn't force.) Remove a bunch of crufty code for large-object-based arrays, which is superseded by TOAST and likely hasn't worked in a long time anyway. Clean up array code a little, and in particular eliminate its habit of scribbling on the input array (ie, modifying the input tuple :-().
2000-07-22 05:34:43 +02:00
indx[i] = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
j = ndim - 1;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
do
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (dist[j])
{
ptr = array_seek(ptr, src_offset, arraynullsptr, dist[j],
typlen, typbyval, typalign);
src_offset += dist[j];
}
if (!array_get_isnull(arraynullsptr, src_offset))
{
Support varlena fields with single-byte headers and unaligned storage. This commit breaks any code that assumes that the mere act of forming a tuple (without writing it to disk) does not "toast" any fields. While all available regression tests pass, I'm not totally sure that we've fixed every nook and cranny, especially in contrib. Greg Stark with some help from Tom Lane
2007-04-06 06:21:44 +02:00
inc = att_addlength_pointer(0, typlen, ptr);
inc = att_align_nominal(inc, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
ptr += inc;
count += inc;
}
src_offset++;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
} while ((j = mda_next_tuple(ndim, indx, span)) != -1);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
return count;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* Extract a slice of an array into consecutive elements in the destination
* array.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* We assume the caller has verified that the slice coordinates are valid,
* allocated enough storage for the result, and initialized the header
* of the new array.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
static void
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_extract_slice(ArrayType *newarray,
int ndim,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int *dim,
int *lb,
char *arraydataptr,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
bits8 *arraynullsptr,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int *st,
int *endp,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int typlen,
bool typbyval,
char typalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char *destdataptr = ARR_DATA_PTR(newarray);
bits8 *destnullsptr = ARR_NULLBITMAP(newarray);
char *srcdataptr;
int src_offset,
dest_offset,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
prod[MAXDIM],
span[MAXDIM],
dist[MAXDIM],
indx[MAXDIM];
int i,
j,
inc;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
src_offset = ArrayGetOffset(ndim, dim, lb, st);
srcdataptr = array_seek(arraydataptr, 0, arraynullsptr, src_offset,
Re-run pgindent, fixing a problem where comment lines after a blank comment line where output as too long, and update typedefs for /lib directory. Also fix case where identifiers were used as variable names in the backend, but as typedefs in ecpg (favor the backend for indenting). Backpatch to 8.1.X.
2005-11-22 19:17:34 +01:00
typlen, typbyval, typalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
mda_get_prod(ndim, dim, prod);
mda_get_range(ndim, span, st, endp);
mda_get_offset_values(ndim, dist, prod, span);
for (i = 0; i < ndim; i++)
indx[i] = 0;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dest_offset = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
j = ndim - 1;
do
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (dist[j])
{
/* skip unwanted elements */
srcdataptr = array_seek(srcdataptr, src_offset, arraynullsptr,
dist[j],
typlen, typbyval, typalign);
src_offset += dist[j];
}
inc = array_copy(destdataptr, 1,
srcdataptr, src_offset, arraynullsptr,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
typlen, typbyval, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (destnullsptr)
array_bitmap_copy(destnullsptr, dest_offset,
arraynullsptr, src_offset,
1);
destdataptr += inc;
srcdataptr += inc;
src_offset++;
dest_offset++;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
} while ((j = mda_next_tuple(ndim, indx, span)) != -1);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/*
* Insert a slice into an array.
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* ndim/dim[]/lb[] are dimensions of the original array. A new array with
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* those same dimensions is to be constructed. destArray must already
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* have been allocated and its header initialized.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* st[]/endp[] identify the slice to be replaced. Elements within the slice
* volume are taken from consecutive elements of the srcArray; elements
* outside it are copied from origArray.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
* We assume the caller has verified that the slice coordinates are valid.
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
*/
static void
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_insert_slice(ArrayType *destArray,
ArrayType *origArray,
ArrayType *srcArray,
int ndim,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
int *dim,
int *lb,
int *st,
int *endp,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
int typlen,
bool typbyval,
char typalign)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
char *destPtr = ARR_DATA_PTR(destArray);
char *origPtr = ARR_DATA_PTR(origArray);
char *srcPtr = ARR_DATA_PTR(srcArray);
bits8 *destBitmap = ARR_NULLBITMAP(destArray);
bits8 *origBitmap = ARR_NULLBITMAP(origArray);
bits8 *srcBitmap = ARR_NULLBITMAP(srcArray);
int orignitems = ArrayGetNItems(ARR_NDIM(origArray),
ARR_DIMS(origArray));
int dest_offset,
orig_offset,
src_offset,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
prod[MAXDIM],
span[MAXDIM],
dist[MAXDIM],
indx[MAXDIM];
Another PGINDENT run that changes variable indenting and case label indenting. Also static variable indenting.
1997-09-08 04:41:22 +02:00
int i,
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
j,
inc;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dest_offset = ArrayGetOffset(ndim, dim, lb, st);
/* copy items before the slice start */
inc = array_copy(destPtr, dest_offset,
origPtr, 0, origBitmap,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
typlen, typbyval, typalign);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
destPtr += inc;
origPtr += inc;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (destBitmap)
array_bitmap_copy(destBitmap, 0, origBitmap, 0, dest_offset);
orig_offset = dest_offset;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
mda_get_prod(ndim, dim, prod);
mda_get_range(ndim, span, st, endp);
mda_get_offset_values(ndim, dist, prod, span);
for (i = 0; i < ndim; i++)
indx[i] = 0;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
src_offset = 0;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
j = ndim - 1;
do
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
{
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* Copy/advance over elements between here and next part of slice */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (dist[j])
{
inc = array_copy(destPtr, dist[j],
origPtr, orig_offset, origBitmap,
typlen, typbyval, typalign);
destPtr += inc;
origPtr += inc;
if (destBitmap)
array_bitmap_copy(destBitmap, dest_offset,
origBitmap, orig_offset,
dist[j]);
dest_offset += dist[j];
orig_offset += dist[j];
}
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* Copy new element at this slice position */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
inc = array_copy(destPtr, 1,
srcPtr, src_offset, srcBitmap,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
typlen, typbyval, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (destBitmap)
array_bitmap_copy(destBitmap, dest_offset,
srcBitmap, src_offset,
1);
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
destPtr += inc;
srcPtr += inc;
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
dest_offset++;
src_offset++;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
/* Advance over old element at this slice position */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
origPtr = array_seek(origPtr, orig_offset, origBitmap, 1,
Modify array operations to include array's element type OID in the array header, and to compute sizing and alignment of array elements the same way normal tuple access operations do --- viz, using the tupmacs.h macros att_addlength and att_align. This makes the world safe for arrays of cstrings or intervals, and should make it much easier to write array-type-polymorphic functions; as examples see the cleanups of array_out and contrib/array_iterator. By Joe Conway and Tom Lane.
2002-08-26 19:54:02 +02:00
typlen, typbyval, typalign);
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
orig_offset++;
Further cleanup of array behavior. Slice assignments to arrays with varlena elements work now. Allow assignment to previously-nonexistent subscript position to extend array, but only for 1-D arrays and only if adjacent to existing positions (could do more if we had a way to represent nulls in arrays, but I don't want to tackle that now). Arrange for assignment of NULL to an array element in UPDATE to be a no-op, rather than setting the entire array to NULL as it used to. (Throwing an error would be a reasonable alternative, but it's never done that...) Update regress test accordingly.
2000-07-23 03:36:05 +02:00
} while ((j = mda_next_tuple(ndim, indx, span)) != -1);
/* don't miss any data at the end */
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
array_copy(destPtr, orignitems - orig_offset,
origPtr, orig_offset, origBitmap,
typlen, typbyval, typalign);
if (destBitmap)
array_bitmap_copy(destBitmap, dest_offset,
origBitmap, orig_offset,
orignitems - orig_offset);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/*
* initArrayResult - initialize an empty ArrayBuildState
*
* element_type is the array element type (must be a valid array element type)
* rcontext is where to keep working state
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* subcontext is a flag determining whether to use a separate memory context
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
*
* Note: there are two common schemes for using accumArrayResult().
* In the older scheme, you start with a NULL ArrayBuildState pointer, and
* call accumArrayResult once per element. In this scheme you end up with
* a NULL pointer if there were no elements, which you need to special-case.
* In the newer scheme, call initArrayResult and then call accumArrayResult
* once per element. In this scheme you always end with a non-NULL pointer
* that you can pass to makeArrayResult; you get an empty array if there
* were no elements. This is preferred if an empty array is what you want.
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
*
* It's possible to choose whether to create a separate memory context for the
* array build state, or whether to allocate it directly within rcontext.
*
* When there are many concurrent small states (e.g. array_agg() using hash
* aggregation of many small groups), using a separate memory context for each
* one may result in severe memory bloat. In such cases, use the same memory
* context to initialize all such array build states, and pass
* subcontext=false.
*
* In cases when the array build states have different lifetimes, using a
* single memory context is impractical. Instead, pass subcontext=true so that
* the array build states can be freed individually.
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
*/
ArrayBuildState *
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
initArrayResult(Oid element_type, MemoryContext rcontext, bool subcontext)
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
{
ArrayBuildState *astate;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
MemoryContext arr_context = rcontext;
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/* Make a temporary context to hold all the junk */
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
if (subcontext)
arr_context = AllocSetContextCreate(rcontext,
"accumArrayResult",
Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>
2016-08-27 23:50:38 +02:00
ALLOCSET_DEFAULT_SIZES);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate = (ArrayBuildState *)
MemoryContextAlloc(arr_context, sizeof(ArrayBuildState));
astate->mcontext = arr_context;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate->private_cxt = subcontext;
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
astate->alen = (subcontext ? 64 : 8); /* arbitrary starting array size */
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate->dvalues = (Datum *)
MemoryContextAlloc(arr_context, astate->alen * sizeof(Datum));
astate->dnulls = (bool *)
MemoryContextAlloc(arr_context, astate->alen * sizeof(bool));
astate->nelems = 0;
astate->element_type = element_type;
get_typlenbyvalalign(element_type,
&astate->typlen,
&astate->typbyval,
&astate->typalign);
return astate;
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
/*
* accumArrayResult - accumulate one (more) Datum for an array result
*
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* astate is working state (can be NULL on first call)
* dvalue/disnull represent the new Datum to append to the array
* element_type is the Datum's type (must be a valid array element type)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
* rcontext is where to keep working state
*/
ArrayBuildState *
Another pgindent run with updated typedefs.
2003-08-08 23:42:59 +02:00
accumArrayResult(ArrayBuildState *astate,
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
Datum dvalue, bool disnull,
Oid element_type,
MemoryContext rcontext)
{
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
MemoryContext oldcontext;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
if (astate == NULL)
{
/* First time through --- initialize */
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate = initArrayResult(element_type, rcontext, true);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
else
{
Assert(astate->element_type == element_type);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
}
oldcontext = MemoryContextSwitchTo(astate->mcontext);
/* enlarge dvalues[]/dnulls[] if needed */
if (astate->nelems >= astate->alen)
{
astate->alen *= 2;
astate->dvalues = (Datum *)
repalloc(astate->dvalues, astate->alen * sizeof(Datum));
astate->dnulls = (bool *)
repalloc(astate->dnulls, astate->alen * sizeof(bool));
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
/*
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* Ensure pass-by-ref stuff is copied into mcontext; and detoast it too if
* it's varlena. (You might think that detoasting is not needed here
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
* because construct_md_array can detoast the array elements later.
* However, we must not let construct_md_array modify the ArrayBuildState
pgindent run for 9.0
2010-02-26 03:01:40 +01:00
* because that would mean array_agg_finalfn damages its input, which is
* verboten. Also, this way frequently saves one copying step.)
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
*/
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
if (!disnull && !astate->typbyval)
Fix things so that array_agg_finalfn does not modify or free its input ArrayBuildState, per trouble report from Merlin Moncure. By adopting this fix, we are essentially deciding that aggregate final-functions should not modify their inputs ever. Adjust documentation and comments to match that conclusion.
2009-06-20 20:45:28 +02:00
{
if (astate->typlen == -1)
dvalue = PointerGetDatum(PG_DETOAST_DATUM_COPY(dvalue));
else
dvalue = datumCopy(dvalue, astate->typbyval, astate->typlen);
}
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
astate->dvalues[astate->nelems] = dvalue;
astate->dnulls[astate->nelems] = disnull;
astate->nelems++;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
MemoryContextSwitchTo(oldcontext);
return astate;
}
/*
* makeArrayResult - produce 1-D final result of accumArrayResult
*
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* Note: only releases astate if it was initialized within a separate memory
* context (i.e. using subcontext=true when calling initArrayResult).
*
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* astate is working state (must not be NULL)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
* rcontext is where to construct result
*/
Datum
Another pgindent run with updated typedefs.
2003-08-08 23:42:59 +02:00
makeArrayResult(ArrayBuildState *astate,
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
MemoryContext rcontext)
{
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
int ndims;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int dims[1];
int lbs[1];
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/* If no elements were presented, we want to create an empty array */
ndims = (astate->nelems > 0) ? 1 : 0;
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
dims[0] = astate->nelems;
lbs[0] = 1;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
return makeMdArrayResult(astate, ndims, dims, lbs, rcontext,
astate->private_cxt);
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
}
/*
* makeMdArrayResult - produce multi-D final result of accumArrayResult
*
* beware: no check that specified dimensions match the number of values
* accumulated.
*
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* Note: if the astate was not initialized within a separate memory context
* (that is, initArrayResult was called with subcontext=false), then using
* release=true is illegal. Instead, release astate along with the rest of its
* context when appropriate.
*
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* astate is working state (must not be NULL)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
* rcontext is where to construct result
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
* release is true if okay to release working state
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
*/
Datum
Another pgindent run with updated typedefs.
2003-08-08 23:42:59 +02:00
makeMdArrayResult(ArrayBuildState *astate,
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
int ndims,
int *dims,
int *lbs,
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
MemoryContext rcontext,
bool release)
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
{
ArrayType *result;
MemoryContext oldcontext;
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
result = construct_md_array(astate->dvalues,
Make SQL arrays support null elements. This commit fixes the core array functionality, but I still need to make another pass looking at places that incidentally use arrays (such as ACL manipulation) to make sure they are null-safe. Contrib needs work too. I have not changed the behaviors that are still under discussion about array comparison and what to do with lower bounds.
2005-11-17 23:14:56 +01:00
astate->dnulls,
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
ndims,
dims,
lbs,
astate->element_type,
astate->typlen,
astate->typbyval,
astate->typalign);
MemoryContextSwitchTo(oldcontext);
/* Clean up all the junk */
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
if (release)
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
{
Assert(astate->private_cxt);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
MemoryContextDelete(astate->mcontext);
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
}
Create real array comparison functions (that use the element datatype's comparison functions), replacing the highly bogus bitwise array_eq. Create a btree index opclass for ANYARRAY --- it is now possible to create indexes on array columns. Arrange to cache the results of catalog lookups across multiple array operations, instead of repeating the lookups on every call. Add string_to_array and array_to_string functions. Remove singleton_array, array_accum, array_assign, and array_subscript functions, since these were for proof-of-concept and not intended to become supported functions. Minor adjustments to behavior in some corner cases with empty or zero-dimensional arrays. Joe Conway (with some editorializing by Tom Lane).
2003-06-27 02:33:26 +02:00
return PointerGetDatum(result);
}
Implement max() and min() aggregates for array types. Patch from Koju Iijima, reviewed by Neil Conway. Catalog version number bumped, regression tests updated.
2005-02-28 04:45:24 +01:00
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/*
* The following three functions provide essentially the same API as
* initArrayResult/accumArrayResult/makeArrayResult, but instead of accepting
* inputs that are array elements, they accept inputs that are arrays and
* produce an output array having N+1 dimensions. The inputs must all have
* identical dimensionality as well as element type.
*/
/*
* initArrayResultArr - initialize an empty ArrayBuildStateArr
*
* array_type is the array type (must be a valid varlena array type)
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* element_type is the type of the array's elements (lookup if InvalidOid)
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
* rcontext is where to keep working state
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* subcontext is a flag determining whether to use a separate memory context
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
*/
ArrayBuildStateArr *
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
initArrayResultArr(Oid array_type, Oid element_type, MemoryContext rcontext,
bool subcontext)
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
{
ArrayBuildStateArr *astate;
Phase 2 of pgindent updates. Change pg_bsd_indent to follow upstream rules for placement of comments to the right of code, and remove pgindent hack that caused comments following #endif to not obey the general rule. Commit e3860ffa4dd0dad0dd9eea4be9cc1412373a8c89 wasn't actually using the published version of pg_bsd_indent, but a hacked-up version that tried to minimize the amount of movement of comments to the right of code. The situation of interest is where such a comment has to be moved to the right of its default placement at column 33 because there's code there. BSD indent has always moved right in units of tab stops in such cases --- but in the previous incarnation, indent was working in 8-space tab stops, while now it knows we use 4-space tabs. So the net result is that in about half the cases, such comments are placed one tab stop left of before. This is better all around: it leaves more room on the line for comment text, and it means that in such cases the comment uniformly starts at the next 4-space tab stop after the code, rather than sometimes one and sometimes two tabs after. Also, ensure that comments following #endif are indented the same as comments following other preprocessor commands such as #else. That inconsistency turns out to have been self-inflicted damage from a poorly-thought-through post-indent "fixup" in pgindent. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:18:54 +02:00
MemoryContext arr_context = rcontext; /* by default use the parent ctx */
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
/* Lookup element type, unless element_type already provided */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (!OidIsValid(element_type))
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
{
element_type = get_element_type(array_type);
if (!OidIsValid(element_type))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("data type %s is not an array type",
format_type_be(array_type))));
}
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/* Make a temporary context to hold all the junk */
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
if (subcontext)
arr_context = AllocSetContextCreate(rcontext,
"accumArrayResultArr",
Add macros to make AllocSetContextCreate() calls simpler and safer. I found that half a dozen (nearly 5%) of our AllocSetContextCreate calls had typos in the context-sizing parameters. While none of these led to especially significant problems, they did create minor inefficiencies, and it's now clear that expecting people to copy-and-paste those calls accurately is not a great idea. Let's reduce the risk of future errors by introducing single macros that encapsulate the common use-cases. Three such macros are enough to cover all but two special-purpose contexts; those two calls can be left as-is, I think. While this patch doesn't in itself improve matters for third-party extensions, it doesn't break anything for them either, and they can gradually adopt the simplified notation over time. In passing, change TopMemoryContext to use the default allocation parameters. Formerly it could only be extended 8K at a time. That was probably reasonable when this code was written; but nowadays we create many more contexts than we did then, so that it's not unusual to have a couple hundred K in TopMemoryContext, even without considering various dubious code that sticks other things there. There seems no good reason not to let it use growing blocks like most other contexts. Back-patch to 9.6, mostly because that's still close enough to HEAD that it's easy to do so, and keeping the branches in sync can be expected to avoid some future back-patching pain. The bugs fixed by these changes don't seem to be significant enough to justify fixing them further back. Discussion: <21072.1472321324@sss.pgh.pa.us>
2016-08-27 23:50:38 +02:00
ALLOCSET_DEFAULT_SIZES);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/* Note we initialize all fields to zero */
astate = (ArrayBuildStateArr *)
MemoryContextAllocZero(arr_context, sizeof(ArrayBuildStateArr));
astate->mcontext = arr_context;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate->private_cxt = subcontext;
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
/* Save relevant datatype information */
astate->array_type = array_type;
astate->element_type = element_type;
return astate;
}
/*
* accumArrayResultArr - accumulate one (more) sub-array for an array result
*
* astate is working state (can be NULL on first call)
* dvalue/disnull represent the new sub-array to append to the array
* array_type is the array type (must be a valid varlena array type)
* rcontext is where to keep working state
*/
ArrayBuildStateArr *
accumArrayResultArr(ArrayBuildStateArr *astate,
Datum dvalue, bool disnull,
Oid array_type,
MemoryContext rcontext)
{
ArrayType *arg;
MemoryContext oldcontext;
int *dims,
*lbs,
ndims,
nitems,
ndatabytes;
char *data;
int i;
/*
* We disallow accumulating null subarrays. Another plausible definition
* is to ignore them, but callers that want that can just skip calling
* this function.
*/
if (disnull)
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("cannot accumulate null arrays")));
/* Detoast input array in caller's context */
arg = DatumGetArrayTypeP(dvalue);
if (astate == NULL)
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate = initArrayResultArr(array_type, InvalidOid, rcontext, true);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
else
Assert(astate->array_type == array_type);
oldcontext = MemoryContextSwitchTo(astate->mcontext);
/* Collect this input's dimensions */
ndims = ARR_NDIM(arg);
dims = ARR_DIMS(arg);
lbs = ARR_LBOUND(arg);
data = ARR_DATA_PTR(arg);
nitems = ArrayGetNItems(ndims, dims);
ndatabytes = ARR_SIZE(arg) - ARR_DATA_OFFSET(arg);
if (astate->ndims == 0)
{
/* First input; check/save the dimensionality info */
/* Should we allow empty inputs and just produce an empty output? */
if (ndims == 0)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot accumulate empty arrays")));
if (ndims + 1 > MAXDIM)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
ndims + 1, MAXDIM)));
/*
* The output array will have n+1 dimensions, with the ones after the
* first matching the input's dimensions.
*/
astate->ndims = ndims + 1;
astate->dims[0] = 0;
memcpy(&astate->dims[1], dims, ndims * sizeof(int));
astate->lbs[0] = 1;
memcpy(&astate->lbs[1], lbs, ndims * sizeof(int));
/* Allocate at least enough data space for this item */
Modify additional power 2 calculations to use new helper functions 2nd pass of modifying various places which obtain the next power of 2 of a number and make them use the new functions added in f0705bb62. In passing, also modify num_combinations(). This can be implemented using simple bitshifting rather than looping. Reviewed-by: John Naylor Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org
2020-04-08 08:29:51 +02:00
astate->abytes = pg_nextpower2_32(Max(1024, ndatabytes + 1));
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate->data = (char *) palloc(astate->abytes);
}
else
{
/* Second or later input: must match first input's dimensionality */
if (astate->ndims != ndims + 1)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("cannot accumulate arrays of different dimensionality")));
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
for (i = 0; i < ndims; i++)
{
if (astate->dims[i + 1] != dims[i] || astate->lbs[i + 1] != lbs[i])
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("cannot accumulate arrays of different dimensionality")));
}
/* Enlarge data space if needed */
if (astate->nbytes + ndatabytes > astate->abytes)
{
astate->abytes = Max(astate->abytes * 2,
astate->nbytes + ndatabytes);
astate->data = (char *) repalloc(astate->data, astate->abytes);
}
}
/*
* Copy the data portion of the sub-array. Note we assume that the
* advertised data length of the sub-array is properly aligned. We do not
* have to worry about detoasting elements since whatever's in the
* sub-array should be OK already.
*/
memcpy(astate->data + astate->nbytes, data, ndatabytes);
astate->nbytes += ndatabytes;
/* Deal with null bitmap if needed */
if (astate->nullbitmap || ARR_HASNULL(arg))
{
int newnitems = astate->nitems + nitems;
if (astate->nullbitmap == NULL)
{
/*
* First input with nulls; we must retrospectively handle any
* previous inputs by marking all their items non-null.
*/
Modify additional power 2 calculations to use new helper functions 2nd pass of modifying various places which obtain the next power of 2 of a number and make them use the new functions added in f0705bb62. In passing, also modify num_combinations(). This can be implemented using simple bitshifting rather than looping. Reviewed-by: John Naylor Discussion: https://postgr.es/m/20200114173553.GE32763%40fetter.org
2020-04-08 08:29:51 +02:00
astate->aitems = pg_nextpower2_32(Max(256, newnitems + 1));
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate->nullbitmap = (bits8 *) palloc((astate->aitems + 7) / 8);
array_bitmap_copy(astate->nullbitmap, 0,
NULL, 0,
astate->nitems);
}
else if (newnitems > astate->aitems)
{
astate->aitems = Max(astate->aitems * 2, newnitems);
astate->nullbitmap = (bits8 *)
repalloc(astate->nullbitmap, (astate->aitems + 7) / 8);
}
array_bitmap_copy(astate->nullbitmap, astate->nitems,
ARR_NULLBITMAP(arg), 0,
nitems);
}
astate->nitems += nitems;
astate->dims[0] += 1;
MemoryContextSwitchTo(oldcontext);
/* Release detoasted copy if any */
if ((Pointer) arg != DatumGetPointer(dvalue))
pfree(arg);
return astate;
}
/*
* makeArrayResultArr - produce N+1-D final result of accumArrayResultArr
*
* astate is working state (must not be NULL)
* rcontext is where to construct result
* release is true if okay to release working state
*/
Datum
makeArrayResultArr(ArrayBuildStateArr *astate,
MemoryContext rcontext,
bool release)
{
ArrayType *result;
MemoryContext oldcontext;
/* Build the final array result in rcontext */
oldcontext = MemoryContextSwitchTo(rcontext);
if (astate->ndims == 0)
{
/* No inputs, return empty array */
result = construct_empty_array(astate->element_type);
}
else
{
int dataoffset,
nbytes;
/* Compute required space */
nbytes = astate->nbytes;
if (astate->nullbitmap != NULL)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(astate->ndims, astate->nitems);
nbytes += dataoffset;
}
else
{
dataoffset = 0;
nbytes += ARR_OVERHEAD_NONULLS(astate->ndims);
}
result = (ArrayType *) palloc0(nbytes);
SET_VARSIZE(result, nbytes);
result->ndim = astate->ndims;
result->dataoffset = dataoffset;
result->elemtype = astate->element_type;
memcpy(ARR_DIMS(result), astate->dims, astate->ndims * sizeof(int));
memcpy(ARR_LBOUND(result), astate->lbs, astate->ndims * sizeof(int));
memcpy(ARR_DATA_PTR(result), astate->data, astate->nbytes);
if (astate->nullbitmap != NULL)
array_bitmap_copy(ARR_NULLBITMAP(result), 0,
astate->nullbitmap, 0,
astate->nitems);
}
MemoryContextSwitchTo(oldcontext);
/* Clean up all the junk */
if (release)
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
{
Assert(astate->private_cxt);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
MemoryContextDelete(astate->mcontext);
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
}
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
return PointerGetDatum(result);
}
/*
* The following three functions provide essentially the same API as
* initArrayResult/accumArrayResult/makeArrayResult, but can accept either
* scalar or array inputs, invoking the appropriate set of functions above.
*/
/*
* initArrayResultAny - initialize an empty ArrayBuildStateAny
*
* input_type is the input datatype (either element or array type)
* rcontext is where to keep working state
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
* subcontext is a flag determining whether to use a separate memory context
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
*/
ArrayBuildStateAny *
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
initArrayResultAny(Oid input_type, MemoryContext rcontext, bool subcontext)
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
{
ArrayBuildStateAny *astate;
Oid element_type = get_element_type(input_type);
if (OidIsValid(element_type))
{
/* Array case */
ArrayBuildStateArr *arraystate;
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
arraystate = initArrayResultArr(input_type, InvalidOid, rcontext, subcontext);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate = (ArrayBuildStateAny *)
MemoryContextAlloc(arraystate->mcontext,
sizeof(ArrayBuildStateAny));
astate->scalarstate = NULL;
astate->arraystate = arraystate;
}
else
{
/* Scalar case */
ArrayBuildState *scalarstate;
/* Let's just check that we have a type that can be put into arrays */
Assert(OidIsValid(get_array_type(input_type)));
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
scalarstate = initArrayResult(input_type, rcontext, subcontext);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
astate = (ArrayBuildStateAny *)
MemoryContextAlloc(scalarstate->mcontext,
sizeof(ArrayBuildStateAny));
astate->scalarstate = scalarstate;
astate->arraystate = NULL;
}
return astate;
}
/*
* accumArrayResultAny - accumulate one (more) input for an array result
*
* astate is working state (can be NULL on first call)
* dvalue/disnull represent the new input to append to the array
* input_type is the input datatype (either element or array type)
* rcontext is where to keep working state
*/
ArrayBuildStateAny *
accumArrayResultAny(ArrayBuildStateAny *astate,
Datum dvalue, bool disnull,
Oid input_type,
MemoryContext rcontext)
{
if (astate == NULL)
In array_agg(), don't create a new context for every group. Previously, each new array created a new memory context that started out at 8kB. This is incredibly wasteful when there are lots of small groups of just a few elements each. Change initArrayResult() and friends to accept a "subcontext" argument to indicate whether the caller wants the ArrayBuildState allocated in a new subcontext or not. If not, it can no longer be released separately from the rest of the memory context. Fixes bug report by Frank van Vugt on 2013-10-19. Tomas Vondra. Reviewed by Ali Akbar, Tom Lane, and me.
2015-02-22 02:24:48 +01:00
astate = initArrayResultAny(input_type, rcontext, true);
Support arrays as input to array_agg() and ARRAY(SELECT ...). These cases formerly failed with errors about "could not find array type for data type". Now they yield arrays of the same element type and one higher dimension. The implementation involves creating functions with API similar to the existing accumArrayResult() family. I (tgl) also extended the base family by adding an initArrayResult() function, which allows callers to avoid special-casing the zero-inputs case if they just want an empty array as result. (Not all do, so the previous calling convention remains valid.) This allowed simplifying some existing code in xml.c and plperl.c. Ali Akbar, reviewed by Pavel Stehule, significantly modified by me
2014-11-25 18:21:22 +01:00
if (astate->scalarstate)
(void) accumArrayResult(astate->scalarstate,
dvalue, disnull,
input_type, rcontext);
else
(void) accumArrayResultArr(astate->arraystate,
dvalue, disnull,
input_type, rcontext);
return astate;
}
/*
* makeArrayResultAny - produce final result of accumArrayResultAny
*
* astate is working state (must not be NULL)
* rcontext is where to construct result
* release is true if okay to release working state
*/
Datum
makeArrayResultAny(ArrayBuildStateAny *astate,
MemoryContext rcontext, bool release)
{
Datum result;
if (astate->scalarstate)
{
/* Must use makeMdArrayResult to support "release" parameter */
int ndims;
int dims[1];
int lbs[1];
/* If no elements were presented, we want to create an empty array */
ndims = (astate->scalarstate->nelems > 0) ? 1 : 0;
dims[0] = astate->scalarstate->nelems;
lbs[0] = 1;
result = makeMdArrayResult(astate->scalarstate, ndims, dims, lbs,
rcontext, release);
}
else
{
result = makeArrayResultArr(astate->arraystate,
rcontext, release);
}
return result;
}
Implement max() and min() aggregates for array types. Patch from Koju Iijima, reviewed by Neil Conway. Catalog version number bumped, regression tests updated.
2005-02-28 04:45:24 +01:00
Datum
array_larger(PG_FUNCTION_ARGS)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (array_cmp(fcinfo) > 0)
PG_RETURN_DATUM(PG_GETARG_DATUM(0));
else
PG_RETURN_DATUM(PG_GETARG_DATUM(1));
Implement max() and min() aggregates for array types. Patch from Koju Iijima, reviewed by Neil Conway. Catalog version number bumped, regression tests updated.
2005-02-28 04:45:24 +01:00
}
Datum
array_smaller(PG_FUNCTION_ARGS)
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (array_cmp(fcinfo) < 0)
PG_RETURN_DATUM(PG_GETARG_DATUM(0));
else
PG_RETURN_DATUM(PG_GETARG_DATUM(1));
Implement max() and min() aggregates for array types. Patch from Koju Iijima, reviewed by Neil Conway. Catalog version number bumped, regression tests updated.
2005-02-28 04:45:24 +01:00
}
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
typedef struct generate_subscripts_fctx
{
Replace int2/int4 in C code with int16/int32 The latter was already the dominant use, and it's preferable because in C the convention is that intXX means XX bits. Therefore, allowing mixed use of int2, int4, int8, int16, int32 is obviously confusing. Remove the typedefs for int2 and int4 for now. They don't seem to be widely used outside of the PostgreSQL source tree, and the few uses can probably be cleaned up by the time this ships.
2012-06-25 00:51:46 +02:00
int32 lower;
int32 upper;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
bool reverse;
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
} generate_subscripts_fctx;
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
/*
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
* generate_subscripts(array anyarray, dim int [, reverse bool])
* Returns all subscripts of the array for any dimension
*/
Datum
generate_subscripts(PG_FUNCTION_ARGS)
{
FuncCallContext *funcctx;
MemoryContext oldcontext;
generate_subscripts_fctx *fctx;
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
AnyArrayType *v = PG_GETARG_ANY_ARRAY_P(0);
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int reqdim = PG_GETARG_INT32(1);
int *lb,
*dimv;
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
/* create a function context for cross-call persistence */
funcctx = SRF_FIRSTCALL_INIT();
/* Sanity check: does it look like an array at all? */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (AARR_NDIM(v) <= 0 || AARR_NDIM(v) > MAXDIM)
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
SRF_RETURN_DONE(funcctx);
/* Sanity check: was the requested dim valid */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (reqdim <= 0 || reqdim > AARR_NDIM(v))
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
SRF_RETURN_DONE(funcctx);
/*
* switch to memory context appropriate for multiple function calls
*/
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
fctx = (generate_subscripts_fctx *) palloc(sizeof(generate_subscripts_fctx));
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
lb = AARR_LBOUND(v);
dimv = AARR_DIMS(v);
Add generate_subscripts, a series-generation function which generates an array's subscripts. Pavel Stehule, some editorialization by me.
2008-04-28 16:48:58 +02:00
fctx->lower = lb[reqdim - 1];
fctx->upper = dimv[reqdim - 1] + lb[reqdim - 1] - 1;
fctx->reverse = (PG_NARGS() < 3) ? false : PG_GETARG_BOOL(2);
funcctx->user_fctx = fctx;
MemoryContextSwitchTo(oldcontext);
}
funcctx = SRF_PERCALL_SETUP();
fctx = funcctx->user_fctx;
if (fctx->lower <= fctx->upper)
{
if (!fctx->reverse)
SRF_RETURN_NEXT(funcctx, Int32GetDatum(fctx->lower++));
else
SRF_RETURN_NEXT(funcctx, Int32GetDatum(fctx->upper--));
}
else
/* done when there are no more elements left */
SRF_RETURN_DONE(funcctx);
}
/*
* generate_subscripts_nodir
* Implements the 2-argument version of generate_subscripts
*/
Datum
generate_subscripts_nodir(PG_FUNCTION_ARGS)
{
/* just call the other one -- it can handle both cases */
return generate_subscripts(fcinfo);
}
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
/*
* array_fill_with_lower_bounds
* Create and fill array with defined lower bounds.
*/
Datum
array_fill_with_lower_bounds(PG_FUNCTION_ARGS)
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ArrayType *dims;
ArrayType *lbs;
ArrayType *result;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Oid elmtype;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Datum value;
bool isnull;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
if (PG_ARGISNULL(1) || PG_ARGISNULL(2))
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("dimension array or low bound array cannot be null")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
dims = PG_GETARG_ARRAYTYPE_P(1);
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
lbs = PG_GETARG_ARRAYTYPE_P(2);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
if (!PG_ARGISNULL(0))
{
value = PG_GETARG_DATUM(0);
isnull = false;
}
else
{
value = 0;
isnull = true;
}
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
elmtype = get_fn_expr_argtype(fcinfo->flinfo, 0);
if (!OidIsValid(elmtype))
elog(ERROR, "could not determine data type of input");
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
result = array_fill_internal(dims, lbs, value, isnull, elmtype, fcinfo);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
PG_RETURN_ARRAYTYPE_P(result);
}
/*
* array_fill
* Create and fill array with default lower bounds.
*/
Datum
array_fill(PG_FUNCTION_ARGS)
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ArrayType *dims;
ArrayType *result;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Oid elmtype;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Datum value;
bool isnull;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
if (PG_ARGISNULL(1))
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("dimension array or low bound array cannot be null")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
dims = PG_GETARG_ARRAYTYPE_P(1);
if (!PG_ARGISNULL(0))
{
value = PG_GETARG_DATUM(0);
isnull = false;
}
else
{
value = 0;
isnull = true;
}
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
elmtype = get_fn_expr_argtype(fcinfo->flinfo, 0);
if (!OidIsValid(elmtype))
elog(ERROR, "could not determine data type of input");
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
result = array_fill_internal(dims, NULL, value, isnull, elmtype, fcinfo);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
PG_RETURN_ARRAYTYPE_P(result);
}
static ArrayType *
create_array_envelope(int ndims, int *dimv, int *lbsv, int nbytes,
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Oid elmtype, int dataoffset)
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ArrayType *result;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
result = (ArrayType *) palloc0(nbytes);
SET_VARSIZE(result, nbytes);
result->ndim = ndims;
result->dataoffset = dataoffset;
result->elemtype = elmtype;
memcpy(ARR_DIMS(result), dimv, ndims * sizeof(int));
memcpy(ARR_LBOUND(result), lbsv, ndims * sizeof(int));
return result;
}
static ArrayType *
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
array_fill_internal(ArrayType *dims, ArrayType *lbs,
Datum value, bool isnull, Oid elmtype,
FunctionCallInfo fcinfo)
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
ArrayType *result;
int *dimv;
int *lbsv;
int ndims;
int nitems;
int deflbs[MAXDIM];
int16 elmlen;
bool elmbyval;
char elmalign;
ArrayMetaState *my_extra;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
/*
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
* Params checks
*/
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
if (ARR_NDIM(dims) > 1)
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("wrong number of array subscripts"),
errdetail("Dimension array must be one dimensional.")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Use array_contains_nulls instead of ARR_HASNULL on user-supplied arrays. This applies the fix for bug #5784 to remaining places where we wish to reject nulls in user-supplied arrays. In all these places, there's no reason not to allow a null bitmap to be present, so long as none of the current elements are actually null. I did not change some other places where we are looking at system catalog entries or aggregate transition values, as the presence of a null bitmap in such an array would be suspicious.
2011-01-09 19:09:07 +01:00
if (array_contains_nulls(dims))
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("dimension values cannot be null")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
dimv = (int *) ARR_DATA_PTR(dims);
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
ndims = (ARR_NDIM(dims) > 0) ? ARR_DIMS(dims)[0] : 0;
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
if (ndims < 0) /* we do allow zero-dimension arrays */
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("invalid number of dimensions: %d", ndims)));
if (ndims > MAXDIM)
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("number of array dimensions (%d) exceeds the maximum allowed (%d)",
ndims, MAXDIM)));
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
if (lbs != NULL)
{
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
if (ARR_NDIM(lbs) > 1)
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("wrong number of array subscripts"),
errdetail("Dimension array must be one dimensional.")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Use array_contains_nulls instead of ARR_HASNULL on user-supplied arrays. This applies the fix for bug #5784 to remaining places where we wish to reject nulls in user-supplied arrays. In all these places, there's no reason not to allow a null bitmap to be present, so long as none of the current elements are actually null. I did not change some other places where we are looking at system catalog entries or aggregate transition values, as the presence of a null bitmap in such an array would be suspicious.
2011-01-09 19:09:07 +01:00
if (array_contains_nulls(lbs))
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("dimension values cannot be null")));
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
if (ndims != ((ARR_NDIM(lbs) > 0) ? ARR_DIMS(lbs)[0] : 0))
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
ereport(ERROR,
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("wrong number of array subscripts"),
errdetail("Low bound array has different size than dimensions array.")));
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
lbsv = (int *) ARR_DATA_PTR(lbs);
}
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
else
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int i;
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
for (i = 0; i < MAXDIM; i++)
deflbs[i] = 1;
lbsv = deflbs;
}
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
nitems = ArrayGetNItems(ndims, dimv);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
/* fast track for empty array */
Fix handling of empty arrays in array_fill(). array_fill(..., array[0]) produced an empty array, which is probably what users expect, but it was a one-dimensional zero-length array which is not our standard representation of empty arrays. Also, for no very good reason, it rejected empty input arrays; that case should be allowed and produce an empty output array. In passing, remove the restriction that the input array(s) have lower bound 1. That seems rather pointless, and it would have needed extra complexity to make the check deal with empty input arrays. Per bug #14487 from Andrew Gierth. It's been broken all along, so back-patch to all supported branches. Discussion: https://postgr.es/m/20170105152156.10135.64195@wrigleys.postgresql.org
2017-01-05 17:33:51 +01:00
if (nitems <= 0)
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
return construct_empty_array(elmtype);
/*
* We arrange to look up info about element type only once per series of
* calls, assuming the element type doesn't change underneath us.
*/
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
if (my_extra == NULL)
{
fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
sizeof(ArrayMetaState));
my_extra = (ArrayMetaState *) fcinfo->flinfo->fn_extra;
my_extra->element_type = InvalidOid;
}
if (my_extra->element_type != elmtype)
{
/* Get info about element type */
get_typlenbyvalalign(elmtype,
&my_extra->typlen,
&my_extra->typbyval,
&my_extra->typalign);
my_extra->element_type = elmtype;
}
elmlen = my_extra->typlen;
elmbyval = my_extra->typbyval;
elmalign = my_extra->typalign;
/* compute required space */
if (!isnull)
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int i;
char *p;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
int nbytes;
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
int totbytes;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
/* make sure data is not toasted */
if (elmlen == -1)
value = PointerGetDatum(PG_DETOAST_DATUM(value));
nbytes = att_addlength_datum(0, elmlen, value);
nbytes = att_align_nominal(nbytes, elmalign);
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Assert(nbytes > 0);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
totbytes = nbytes * nitems;
/* check for overflow of multiplication or total request */
if (totbytes / nbytes != nitems ||
!AllocSizeIsValid(totbytes))
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
/*
* This addition can't overflow, but it might cause us to go past
* MaxAllocSize. We leave it to palloc to complain in that case.
*/
totbytes += ARR_OVERHEAD_NONULLS(ndims);
result = create_array_envelope(ndims, dimv, lbsv, totbytes,
elmtype, 0);
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
p = ARR_DATA_PTR(result);
for (i = 0; i < nitems; i++)
p += ArrayCastAndSet(value, elmlen, elmbyval, elmalign, p);
}
else
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int nbytes;
int dataoffset;
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
dataoffset = ARR_OVERHEAD_WITHNULLS(ndims, nitems);
nbytes = dataoffset;
result = create_array_envelope(ndims, dimv, lbsv, nbytes,
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
elmtype, dataoffset);
/* create_array_envelope already zeroed the bitmap, so we're done */
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
}
Code review for array_fill patch: fix inadequate check for array size overflow and bogus documentation (dimension arrays are int[] not anyarray). Also the errhint() messages seem to be really errdetail(), since there is nothing heuristic about them. Some other trivial cosmetic improvements.
2008-07-21 06:47:00 +02:00
Add array_fill() to create arrays initialized with a value. Pavel Stehule
2008-07-16 02:48:54 +02:00
return result;
}
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
/*
* UNNEST
*/
Datum
array_unnest(PG_FUNCTION_ARGS)
{
typedef struct
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter iter;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int nextelem;
int numelems;
int16 elmlen;
bool elmbyval;
char elmalign;
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
} array_unnest_fctx;
FuncCallContext *funcctx;
array_unnest_fctx *fctx;
MemoryContext oldcontext;
/* stuff done only on the first call of the function */
if (SRF_IS_FIRSTCALL())
{
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
AnyArrayType *arr;
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
/* create a function context for cross-call persistence */
funcctx = SRF_FIRSTCALL_INIT();
/*
* switch to memory context appropriate for multiple function calls
*/
oldcontext = MemoryContextSwitchTo(funcctx->multi_call_memory_ctx);
Fix unnest() to handle a toasted input array safely. Per report from Alvaro.
2009-01-30 22:21:18 +01:00
/*
* Get the array value and detoast if needed. We can't do this
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* earlier because if we have to detoast, we want the detoasted copy
* to be in multi_call_memory_ctx, so it will go away when we're done
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* and not before. (If no detoast happens, we assume the originally
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
* passed array will stick around till then.)
Fix unnest() to handle a toasted input array safely. Per report from Alvaro.
2009-01-30 22:21:18 +01:00
*/
Make DatumGetFoo/PG_GETARG_FOO/PG_RETURN_FOO macro names more consistent. By project convention, these names should include "P" when dealing with a pointer type; that is, if the result of a GETARG macro is of type FOO *, it should be called PG_GETARG_FOO_P not just PG_GETARG_FOO. Some newer types such as JSONB and ranges had not followed the convention, and a number of contrib modules hadn't gotten that memo either. Rename the offending macros to improve consistency. In passing, fix a few places that thought PG_DETOAST_DATUM() returns a Datum; it does not, it returns "struct varlena *". Applying DatumGetPointer to that happens not to cause any bad effects today, but it's formally wrong. Also, adjust an ltree macro that was designed without any thought for what pgindent would do with it. This is all cosmetic and shouldn't have any impact on generated code. Mark Dilger, some further tweaks by me Discussion: https://postgr.es/m/EA5676F4-766F-4F38-8348-ECC7DB427C6A@gmail.com
2017-09-18 21:21:23 +02:00
arr = PG_GETARG_ANY_ARRAY_P(0);
Fix unnest() to handle a toasted input array safely. Per report from Alvaro.
2009-01-30 22:21:18 +01:00
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
/* allocate memory for user context */
fctx = (array_unnest_fctx *) palloc(sizeof(array_unnest_fctx));
Fix unnest() to handle a toasted input array safely. Per report from Alvaro.
2009-01-30 22:21:18 +01:00
/* initialize state */
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
array_iter_setup(&fctx->iter, arr);
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
fctx->nextelem = 0;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
fctx->numelems = ArrayGetNItems(AARR_NDIM(arr), AARR_DIMS(arr));
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
if (VARATT_IS_EXPANDED_HEADER(arr))
{
/* we can just grab the type data from expanded array */
fctx->elmlen = arr->xpn.typlen;
fctx->elmbyval = arr->xpn.typbyval;
fctx->elmalign = arr->xpn.typalign;
}
else
get_typlenbyvalalign(AARR_ELEMTYPE(arr),
&fctx->elmlen,
&fctx->elmbyval,
&fctx->elmalign);
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
funcctx->user_fctx = fctx;
MemoryContextSwitchTo(oldcontext);
}
/* stuff done on every call of the function */
funcctx = SRF_PERCALL_SETUP();
fctx = funcctx->user_fctx;
if (fctx->nextelem < fctx->numelems)
{
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
int offset = fctx->nextelem++;
Datum elem;
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
elem = array_iter_next(&fctx->iter, &fcinfo->isnull, offset,
fctx->elmlen, fctx->elmbyval, fctx->elmalign);
Implement the basic form of UNNEST, ie unnest(anyarray) returns setof anyelement. This lacks the WITH ORDINALITY option, as well as the multiple input arrays option added in the most recent SQL specs. But it's still a pretty useful subset of the spec's functionality, and it is enough to allow obsoleting contrib/intagg.
2008-11-14 01:51:47 +01:00
SRF_RETURN_NEXT(funcctx, elem);
}
else
{
/* do when there is no more left */
SRF_RETURN_DONE(funcctx);
}
}
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
Build out the planner support function infrastructure. Add support function requests for estimating the selectivity, cost, and number of result rows (if a SRF) of the target function. The lack of a way to estimate selectivity of a boolean-returning function in WHERE has been a recognized deficiency of the planner since Berkeley days. This commit finally fixes it. In addition, non-constant estimates of cost and number of output rows are now possible. We still fall back to looking at procost and prorows if the support function doesn't service the request, of course. To make concrete use of the possibility of estimating output rowcount for SRFs, this commit adds support functions for array_unnest(anyarray) and the integer variants of generate_series; the lack of plausible rowcount estimates for those, even when it's obvious to a human, has been a repeated subject of complaints. Obviously, much more could now be done in this line, but I'm mostly just trying to get the infrastructure in place. Discussion: https://postgr.es/m/15193.1548028093@sss.pgh.pa.us
2019-02-10 00:32:23 +01:00
/*
* Planner support function for array_unnest(anyarray)
*/
Datum
array_unnest_support(PG_FUNCTION_ARGS)
{
Node *rawreq = (Node *) PG_GETARG_POINTER(0);
Node *ret = NULL;
if (IsA(rawreq, SupportRequestRows))
{
/* Try to estimate the number of rows returned */
SupportRequestRows *req = (SupportRequestRows *) rawreq;
if (is_funcclause(req->node)) /* be paranoid */
{
List *args = ((FuncExpr *) req->node)->args;
Node *arg1;
/* We can use estimated argument values here */
arg1 = estimate_expression_value(req->root, linitial(args));
req->rows = estimate_array_length(arg1);
ret = (Node *) req;
}
}
PG_RETURN_POINTER(ret);
}
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
/*
* array_replace/array_remove support
*
* Find all array entries matching (not distinct from) search/search_isnull,
* and delete them if remove is true, else replace them with
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* replace/replace_isnull. Comparisons are done using the specified
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
* collation. fcinfo is passed only for caching purposes.
*/
static ArrayType *
array_replace_internal(ArrayType *array,
Datum search, bool search_isnull,
Datum replace, bool replace_isnull,
bool remove, Oid collation,
FunctionCallInfo fcinfo)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
ArrayType *result;
Oid element_type;
Datum *values;
bool *nulls;
int *dim;
int ndim;
int nitems,
nresult;
int i;
int32 nbytes = 0;
int32 dataoffset;
bool hasnulls;
int typlen;
bool typbyval;
char typalign;
char *arraydataptr;
bits8 *bitmap;
int bitmask;
bool changed = false;
TypeCacheEntry *typentry;
element_type = ARR_ELEMTYPE(array);
ndim = ARR_NDIM(array);
dim = ARR_DIMS(array);
nitems = ArrayGetNItems(ndim, dim);
/* Return input array unmodified if it is empty */
if (nitems <= 0)
return array;
/*
* We can't remove elements from multi-dimensional arrays, since the
* result might not be rectangular.
*/
if (remove && ndim > 1)
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("removing elements from multidimensional arrays is not supported")));
/*
* We arrange to look up the equality function only once per series of
* calls, assuming the element type doesn't change underneath us.
*/
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
{
typentry = lookup_type_cache(element_type,
TYPECACHE_EQ_OPR_FINFO);
if (!OidIsValid(typentry->eq_opr_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("could not identify an equality operator for type %s",
format_type_be(element_type))));
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
fcinfo->flinfo->fn_extra = (void *) typentry;
}
typlen = typentry->typlen;
typbyval = typentry->typbyval;
typalign = typentry->typalign;
/*
pgindent run for 9.4 This includes removing tabs after periods in C comments, which was applied to back branches, so this change should not effect backpatching.
2014-05-06 18:12:18 +02:00
* Detoast values if they are toasted. The replacement value must be
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
* detoasted for insertion into the result array, while detoasting the
* search value only once saves cycles.
*/
if (typlen == -1)
{
if (!search_isnull)
search = PointerGetDatum(PG_DETOAST_DATUM(search));
if (!replace_isnull)
replace = PointerGetDatum(PG_DETOAST_DATUM(replace));
}
/* Prepare to apply the comparison operator */
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->eq_opr_finfo, 2,
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
collation, NULL, NULL);
/* Allocate temporary arrays for new values */
values = (Datum *) palloc(nitems * sizeof(Datum));
nulls = (bool *) palloc(nitems * sizeof(bool));
/* Loop over source data */
arraydataptr = ARR_DATA_PTR(array);
bitmap = ARR_NULLBITMAP(array);
bitmask = 1;
hasnulls = false;
nresult = 0;
for (i = 0; i < nitems; i++)
{
Datum elt;
bool isNull;
bool oprresult;
bool skip = false;
/* Get source element, checking for NULL */
if (bitmap && (*bitmap & bitmask) == 0)
{
isNull = true;
/* If searching for NULL, we have a match */
if (search_isnull)
{
if (remove)
{
skip = true;
changed = true;
}
else if (!replace_isnull)
{
values[nresult] = replace;
isNull = false;
changed = true;
}
}
}
else
{
isNull = false;
elt = fetch_att(arraydataptr, typbyval, typlen);
arraydataptr = att_addlength_datum(arraydataptr, typlen, elt);
arraydataptr = (char *) att_align_nominal(arraydataptr, typalign);
if (search_isnull)
{
/* no match possible, keep element */
values[nresult] = elt;
}
else
{
/*
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
* Apply the operator to the element pair; treat NULL as false
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
*/
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = elt;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = search;
locfcinfo->args[1].isnull = false;
locfcinfo->isnull = false;
oprresult = DatumGetBool(FunctionCallInvoke(locfcinfo));
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
if (locfcinfo->isnull || !oprresult)
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
{
/* no match, keep element */
values[nresult] = elt;
}
else
{
/* match, so replace or delete */
changed = true;
if (remove)
skip = true;
else
{
values[nresult] = replace;
isNull = replace_isnull;
}
}
}
}
if (!skip)
{
nulls[nresult] = isNull;
if (isNull)
hasnulls = true;
else
{
/* Update total result size */
nbytes = att_addlength_datum(nbytes, typlen, values[nresult]);
nbytes = att_align_nominal(nbytes, typalign);
/* check for overflow of total request */
if (!AllocSizeIsValid(nbytes))
ereport(ERROR,
(errcode(ERRCODE_PROGRAM_LIMIT_EXCEEDED),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("array size exceeds the maximum allowed (%d)",
(int) MaxAllocSize)));
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
}
nresult++;
}
/* advance bitmap pointer if any */
if (bitmap)
{
bitmask <<= 1;
if (bitmask == 0x100)
{
bitmap++;
bitmask = 1;
}
}
}
/*
* If not changed just return the original array
*/
if (!changed)
{
pfree(values);
pfree(nulls);
return array;
}
Don't emit non-canonical empty arrays in array_remove(). Dean Rasheed
2013-06-01 03:50:59 +02:00
/* If all elements were removed return an empty array */
if (nresult == 0)
{
pfree(values);
pfree(nulls);
return construct_empty_array(element_type);
}
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
/* Allocate and initialize the result array */
if (hasnulls)
{
dataoffset = ARR_OVERHEAD_WITHNULLS(ndim, nresult);
nbytes += dataoffset;
}
else
{
dataoffset = 0; /* marker for no null bitmap */
nbytes += ARR_OVERHEAD_NONULLS(ndim);
}
result = (ArrayType *) palloc0(nbytes);
SET_VARSIZE(result, nbytes);
result->ndim = ndim;
result->dataoffset = dataoffset;
result->elemtype = element_type;
Support "expanded" objects, particularly arrays, for better performance. This patch introduces the ability for complex datatypes to have an in-memory representation that is different from their on-disk format. On-disk formats are typically optimized for minimal size, and in any case they can't contain pointers, so they are often not well-suited for computation. Now a datatype can invent an "expanded" in-memory format that is better suited for its operations, and then pass that around among the C functions that operate on the datatype. There are also provisions (rudimentary as yet) to allow an expanded object to be modified in-place under suitable conditions, so that operations like assignment to an element of an array need not involve copying the entire array. The initial application for this feature is arrays, but it is not hard to foresee using it for other container types like JSON, XML and hstore. I have hopes that it will be useful to PostGIS as well. In this initial implementation, a few heuristics have been hard-wired into plpgsql to improve performance for arrays that are stored in plpgsql variables. We would like to generalize those hacks so that other datatypes can obtain similar improvements, but figuring out some appropriate APIs is left as a task for future work. (The heuristics themselves are probably not optimal yet, either, as they sometimes force expansion of arrays that would be better left alone.) Preliminary performance testing shows impressive speed gains for plpgsql functions that do element-by-element access or update of large arrays. There are other cases that get a little slower, as a result of added array format conversions; but we can hope to improve anything that's annoyingly bad. In any case most applications should see a net win. Tom Lane, reviewed by Andres Freund
2015-05-14 18:08:40 +02:00
memcpy(ARR_DIMS(result), ARR_DIMS(array), ndim * sizeof(int));
memcpy(ARR_LBOUND(result), ARR_LBOUND(array), ndim * sizeof(int));
Add array_remove() and array_replace() functions. These functions support removing or replacing array element value(s) matching a given search value. Although intended mainly to support a future array-foreign-key feature, they seem useful in their own right. Marco Nenciarini and Gabriele Bartolini, reviewed by Alex Hunsaker
2012-07-11 19:59:35 +02:00
if (remove)
{
/* Adjust the result length */
ARR_DIMS(result)[0] = nresult;
}
/* Insert data into result array */
CopyArrayEls(result,
values, nulls, nresult,
typlen, typbyval, typalign,
false);
pfree(values);
pfree(nulls);
return result;
}
/*
* Remove any occurrences of an element from an array
*
* If used on a multi-dimensional array this will raise an error.
*/
Datum
array_remove(PG_FUNCTION_ARGS)
{
ArrayType *array;
Datum search = PG_GETARG_DATUM(1);
bool search_isnull = PG_ARGISNULL(1);
if (PG_ARGISNULL(0))
PG_RETURN_NULL();
array = PG_GETARG_ARRAYTYPE_P(0);
array = array_replace_internal(array,
search, search_isnull,
(Datum) 0, true,
true, PG_GET_COLLATION(),
fcinfo);
PG_RETURN_ARRAYTYPE_P(array);
}
/*
* Replace any occurrences of an element in an array
*/
Datum
array_replace(PG_FUNCTION_ARGS)
{
ArrayType *array;
Datum search = PG_GETARG_DATUM(1);
bool search_isnull = PG_ARGISNULL(1);
Datum replace = PG_GETARG_DATUM(2);
bool replace_isnull = PG_ARGISNULL(2);
if (PG_ARGISNULL(0))
PG_RETURN_NULL();
array = PG_GETARG_ARRAYTYPE_P(0);
array = array_replace_internal(array,
search, search_isnull,
replace, replace_isnull,
false, PG_GET_COLLATION(),
fcinfo);
PG_RETURN_ARRAYTYPE_P(array);
}
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
/*
* Implements width_bucket(anyelement, anyarray).
*
* 'thresholds' is an array containing lower bound values for each bucket;
* these must be sorted from smallest to largest, or bogus results will be
* produced. If N thresholds are supplied, the output is from 0 to N:
* 0 is for inputs < first threshold, N is for inputs >= last threshold.
*/
Datum
width_bucket_array(PG_FUNCTION_ARGS)
{
Datum operand = PG_GETARG_DATUM(0);
ArrayType *thresholds = PG_GETARG_ARRAYTYPE_P(1);
Oid collation = PG_GET_COLLATION();
Oid element_type = ARR_ELEMTYPE(thresholds);
int result;
/* Check input */
if (ARR_NDIM(thresholds) > 1)
ereport(ERROR,
(errcode(ERRCODE_ARRAY_SUBSCRIPT_ERROR),
errmsg("thresholds must be one-dimensional array")));
if (array_contains_nulls(thresholds))
ereport(ERROR,
(errcode(ERRCODE_NULL_VALUE_NOT_ALLOWED),
errmsg("thresholds array must not contain NULLs")));
/* We have a dedicated implementation for float8 data */
if (element_type == FLOAT8OID)
result = width_bucket_array_float8(operand, thresholds);
else
{
TypeCacheEntry *typentry;
/* Cache information about the input type */
typentry = (TypeCacheEntry *) fcinfo->flinfo->fn_extra;
if (typentry == NULL ||
typentry->type_id != element_type)
{
typentry = lookup_type_cache(element_type,
TYPECACHE_CMP_PROC_FINFO);
if (!OidIsValid(typentry->cmp_proc_finfo.fn_oid))
ereport(ERROR,
(errcode(ERRCODE_UNDEFINED_FUNCTION),
Phase 3 of pgindent updates. Don't move parenthesized lines to the left, even if that means they flow past the right margin. By default, BSD indent lines up statement continuation lines that are within parentheses so that they start just to the right of the preceding left parenthesis. However, traditionally, if that resulted in the continuation line extending to the right of the desired right margin, then indent would push it left just far enough to not overrun the margin, if it could do so without making the continuation line start to the left of the current statement indent. That makes for a weird mix of indentations unless one has been completely rigid about never violating the 80-column limit. This behavior has been pretty universally panned by Postgres developers. Hence, disable it with indent's new -lpl switch, so that parenthesized lines are always lined up with the preceding left paren. This patch is much less interesting than the first round of indent changes, but also bulkier, so I thought it best to separate the effects. Discussion: https://postgr.es/m/E1dAmxK-0006EE-1r@gemulon.postgresql.org Discussion: https://postgr.es/m/30527.1495162840@sss.pgh.pa.us
2017-06-21 21:35:54 +02:00
errmsg("could not identify a comparison function for type %s",
format_type_be(element_type))));
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
fcinfo->flinfo->fn_extra = (void *) typentry;
}
/*
* We have separate implementation paths for fixed- and variable-width
* types, since indexing the array is a lot cheaper in the first case.
*/
if (typentry->typlen > 0)
result = width_bucket_array_fixed(operand, thresholds,
collation, typentry);
else
result = width_bucket_array_variable(operand, thresholds,
collation, typentry);
}
/* Avoid leaking memory when handed toasted input. */
PG_FREE_IF_COPY(thresholds, 1);
PG_RETURN_INT32(result);
}
/*
* width_bucket_array for float8 data.
*/
static int
width_bucket_array_float8(Datum operand, ArrayType *thresholds)
{
float8 op = DatumGetFloat8(operand);
float8 *thresholds_data;
int left;
int right;
/*
* Since we know the array contains no NULLs, we can just index it
* directly.
*/
thresholds_data = (float8 *) ARR_DATA_PTR(thresholds);
left = 0;
right = ArrayGetNItems(ARR_NDIM(thresholds), ARR_DIMS(thresholds));
/*
* If the probe value is a NaN, it's greater than or equal to all possible
* threshold values (including other NaNs), so we need not search. Note
* that this would give the same result as searching even if the array
* contains multiple NaNs (as long as they're correctly sorted), since the
* loop logic will find the rightmost of multiple equal threshold values.
*/
if (isnan(op))
return right;
/* Find the bucket */
while (left < right)
{
int mid = (left + right) / 2;
if (isnan(thresholds_data[mid]) || op < thresholds_data[mid])
right = mid;
else
left = mid + 1;
}
return left;
}
/*
* width_bucket_array for generic fixed-width data types.
*/
static int
width_bucket_array_fixed(Datum operand,
ArrayType *thresholds,
Oid collation,
TypeCacheEntry *typentry)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
char *thresholds_data;
int typlen = typentry->typlen;
bool typbyval = typentry->typbyval;
int left;
int right;
/*
* Since we know the array contains no NULLs, we can just index it
* directly.
*/
thresholds_data = (char *) ARR_DATA_PTR(thresholds);
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->cmp_proc_finfo, 2,
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
collation, NULL, NULL);
/* Find the bucket */
left = 0;
right = ArrayGetNItems(ARR_NDIM(thresholds), ARR_DIMS(thresholds));
while (left < right)
{
int mid = (left + right) / 2;
char *ptr;
int32 cmpresult;
ptr = thresholds_data + mid * typlen;
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = operand;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = fetch_att(ptr, typbyval, typlen);
locfcinfo->args[1].isnull = false;
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
cmpresult = DatumGetInt32(FunctionCallInvoke(locfcinfo));
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
/* We don't expect comparison support functions to return null */
Assert(!locfcinfo->isnull);
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
if (cmpresult < 0)
right = mid;
else
left = mid + 1;
}
return left;
}
/*
* width_bucket_array for generic variable-width data types.
*/
static int
width_bucket_array_variable(Datum operand,
ArrayType *thresholds,
Oid collation,
TypeCacheEntry *typentry)
{
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
LOCAL_FCINFO(locfcinfo, 2);
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
char *thresholds_data;
int typlen = typentry->typlen;
bool typbyval = typentry->typbyval;
char typalign = typentry->typalign;
int left;
int right;
thresholds_data = (char *) ARR_DATA_PTR(thresholds);
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
InitFunctionCallInfoData(*locfcinfo, &typentry->cmp_proc_finfo, 2,
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
collation, NULL, NULL);
/* Find the bucket */
left = 0;
right = ArrayGetNItems(ARR_NDIM(thresholds), ARR_DIMS(thresholds));
while (left < right)
{
int mid = (left + right) / 2;
char *ptr;
int i;
int32 cmpresult;
/* Locate mid'th array element by advancing from left element */
ptr = thresholds_data;
for (i = left; i < mid; i++)
{
ptr = att_addlength_pointer(ptr, typlen, ptr);
ptr = (char *) att_align_nominal(ptr, typalign);
}
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
locfcinfo->args[0].value = operand;
locfcinfo->args[0].isnull = false;
locfcinfo->args[1].value = fetch_att(ptr, typbyval, typlen);
locfcinfo->args[1].isnull = false;
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
Change function call information to be variable length. Before this change FunctionCallInfoData, the struct arguments etc for V1 function calls are stored in, always had space for FUNC_MAX_ARGS/100 arguments, storing datums and their nullness in two arrays. For nearly every function call 100 arguments is far more than needed, therefore wasting memory. Arg and argnull being two separate arrays also guarantees that to access a single argument, two cachelines have to be touched. Change the layout so there's a single variable-length array with pairs of value / isnull. That drastically reduces memory consumption for most function calls (on x86-64 a two argument function now uses 64bytes, previously 936 bytes), and makes it very likely that argument value and its nullness are on the same cacheline. Arguments are stored in a new NullableDatum struct, which, due to padding, needs more memory per argument than before. But as usually far fewer arguments are stored, and individual arguments are cheaper to access, that's still a clear win. It's likely that there's other places where conversion to NullableDatum arrays would make sense, e.g. TupleTableSlots, but that's for another commit. Because the function call information is now variable-length allocations have to take the number of arguments into account. For heap allocations that can be done with SizeForFunctionCallInfoData(), for on-stack allocations there's a new LOCAL_FCINFO(name, nargs) macro that helps to allocate an appropriately sized and aligned variable. Some places with stack allocation function call information don't know the number of arguments at compile time, and currently variably sized stack allocations aren't allowed in postgres. Therefore allow for FUNC_MAX_ARGS space in these cases. They're not that common, so for now that seems acceptable. Because of the need to allocate FunctionCallInfo of the appropriate size, older extensions may need to update their code. To avoid subtle breakages, the FunctionCallInfoData struct has been renamed to FunctionCallInfoBaseData. Most code only references FunctionCallInfo, so that shouldn't cause much collateral damage. This change is also a prerequisite for more efficient expression JIT compilation (by allocating the function call information on the stack, allowing LLVM to optimize it away); previously the size of the call information caused problems inside LLVM's optimizer. Author: Andres Freund Reviewed-By: Tom Lane Discussion: https://postgr.es/m/20180605172952.x34m5uz6ju6enaem@alap3.anarazel.de
2019-01-26 23:17:52 +01:00
cmpresult = DatumGetInt32(FunctionCallInvoke(locfcinfo));
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
Fix minor violations of FunctionCallInvoke usage protocol. Working on commit 1c455078b led me to check through FunctionCallInvoke call sites to see if every one was being honest about (a) making sure that fcinfo.isnull is initially false, and (b) checking its state after the call. Sure enough, I found some violations. The main one is that finalize_partialaggregate re-used serialfn_fcinfo without resetting isnull, even though it clearly intends to cater for serialfns that return NULL. There would only be an issue with a non-strict serialfn, since it's unlikely that a serialfn would return NULL for non-null input. We have no non-strict serialfns in core, and there may be none in the wild either, which would account for the lack of complaints. Still, it's clearly wrong, so back-patch that fix to 9.6 where finalize_partialaggregate was introduced. Also, arrayfuncs.c and rowtypes.c contained various callers that were not bothering to check for result nulls. While what's being called is a comparison or hash function that probably *shouldn't* return null, that's a lousy excuse for not having any check at all. There are existing places that just Assert(!fcinfo->isnull) in comparable situations, so I added that to the places that were calling btree comparison or hash support functions. In the places calling boolean-returning equality functions, it's quite cheap to have them treat isnull as FALSE, so make those places do that. Also remove some "locfcinfo->isnull = false" assignments that are unnecessary given the assumption that no previous call returned null. These changes seem like mostly neatnik-ism or debugging support, so I didn't back-patch.
2020-04-21 20:23:42 +02:00
/* We don't expect comparison support functions to return null */
Assert(!locfcinfo->isnull);
Add width_bucket(anyelement, anyarray). This provides a convenient method of classifying input values into buckets that are not necessarily equal-width. It works on any sortable data type. The choice of function name is a bit debatable, perhaps, but showing that there's a relationship to the SQL standard's width_bucket() function seems more attractive than the other proposals. Petr Jelinek, reviewed by Pavel Stehule
2014-09-09 21:34:10 +02:00
if (cmpresult < 0)
right = mid;
else
{
left = mid + 1;
/*
* Move the thresholds pointer to match new "left" index, so we
* don't have to seek over those elements again. This trick
* ensures we do only O(N) array indexing work, not O(N^2).
*/
ptr = att_addlength_pointer(ptr, typlen, ptr);
thresholds_data = (char *) att_align_nominal(ptr, typalign);
}
}
return left;
}