rdelaage
/
postgresql
mirror of https://git.postgresql.org/git/postgresql.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
postgresql/src/backend/nodes/readfuncs.c

Ignoring revisions in .git-blame-ignore-revs. Click here to bypass and see the normal blame view.

3374 lines
68 KiB
C
Raw Normal View History

Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*-------------------------------------------------------------------------
*
Change my-function-name-- to my_function_name, and optimizer renames.
1999-02-14 00:22:53 +01:00
* readfuncs.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
* Reader functions for Postgres tree nodes.
*
Update copyright for 2022 Backpatch-through: 10
2022-01-08 01:04:57 +01:00
* Portions Copyright (c) 1996-2022, PostgreSQL Global Development Group
Add: * Portions Copyright (c) 1996-2000, PostgreSQL, Inc to all files copyright Regents of Berkeley. Man, that's a lot of files.
2000-01-26 06:58:53 +01:00
* Portions Copyright (c) 1994, Regents of the University of California
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
*
* IDENTIFICATION
Remove cvs keywords from all files.
2010-09-20 22:08:53 +02:00
* src/backend/nodes/readfuncs.c
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
* NOTES
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
* Path nodes do not have any readfuncs support, because we never
* have occasion to read them in. (There was once code here that
Phase 3 of read-only-plans project: ExecInitExpr now builds expression execution state trees, and ExecEvalExpr takes an expression state tree not an expression plan tree. The plan tree is now read-only as far as the executor is concerned. Next step is to begin actually exploiting this property.
2002-12-13 20:46:01 +01:00
* claimed to read them, but it was broken as well as unused.) We
* never read executor state trees, either.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
* Parse location fields are written out by outfuncs.c, but only for
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
* debugging use. When reading a location field, we normally discard
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
* the stored value and set the location field to -1 (ie, "unknown").
* This is because nodes coming from a stored rule should not be thought
* to have a known location in the current query's text.
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
* However, if restore_location_fields is true, we do restore location
* fields from the string. This is currently intended only for use by the
* WRITE_READ_PARSE_PLAN_TREES test code, which doesn't want to cause
* any change in the node contents.
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
*
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*-------------------------------------------------------------------------
*/
#include "postgres.h"
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
#include <math.h>
Add stack depth checks to key recursive functions in backend/nodes/*.c. Although copyfuncs.c has a check_stack_depth call in its recursion, equalfuncs.c, outfuncs.c, and readfuncs.c lacked one. This seems unwise. Likewise fix planstate_tree_walker(), in branches where that exists. Discussion: https://postgr.es/m/30253.1544286631@sss.pgh.pa.us
2018-12-10 17:12:43 +01:00
#include "miscadmin.h"
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
#include "nodes/extensible.h"
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
#include "nodes/parsenodes.h"
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
#include "nodes/plannodes.h"
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
#include "nodes/readfuncs.h"
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* Macros to simplify reading of different kinds of fields. Use these
* wherever possible to reduce the chance for silly typos. Note that these
* hard-wire conventions about the names of the local variables in a Read
* routine.
*/
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
/* Macros for declaring appropriate local variables */
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* A few guys need only local_node */
#define READ_LOCALS_NO_FIELDS(nodeTypeName) \
nodeTypeName *local_node = makeNode(nodeTypeName)
/* And a few guys need only the pg_strtok support fields */
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
#define READ_TEMP_LOCALS() \
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token; \
Suppress compiler warnings in readfuncs.c. Commit 7357558fc8866e3a449aa9473c419b593d67b5b6 introduced "(void) token;" into the READ_TEMP_LOCALS() macro, to suppress complaints from gcc 4.6 when the value of token was not used anywhere in a particular node-read function. However, this just moved the warning around: inspection of buildfarm results shows that some compilers are now complaining that token is being read before it's set. Revert the READ_TEMP_LOCALS() macro change and instead put "(void) token;" into READ_NODE_FIELD(), which is the principal culprit for cases where the warning might occur. In principle we might need the same in READ_BITMAPSET_FIELD() and/or READ_LOCATION_FIELD(), but it seems unlikely that a node would consist only of such fields, so I'll leave them alone for now.
2012-07-01 04:27:49 +02:00
int length
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
/* ... but most need both */
#define READ_LOCALS(nodeTypeName) \
READ_LOCALS_NO_FIELDS(nodeTypeName); \
READ_TEMP_LOCALS()
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* Read an integer field (anything written as ":fldname %d") */
#define READ_INT_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = atoi(token)
/* Read an unsigned integer field (anything written as ":fldname %u") */
#define READ_UINT_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = atoui(token)
pg_stat_statements: Widen query IDs from 32 bits to 64 bits. This takes advantage of the infrastructure introduced by commit 81c5e46c490e2426db243eada186995da5bb0ba7 to greatly reduce the likelihood that two different queries will end up with the same query ID. It's still possible, of course, but whereas before it the chances of a collision reached 25% around 50,000 queries, it will now take more than 3 billion queries. Backward incompatibility: Because the type exposed at the SQL level is int8, users may now see negative query IDs in the pg_stat_statements view (and also, query IDs more than 4 billion, which was the old limit). Patch by me, reviewed by Michael Paquier and Peter Geoghegan. Discussion: http://postgr.es/m/CA+TgmobG_Kp4cBKFmsznUAaM1GWW6hhRNiZC0KjRMOOeYnz5Yw@mail.gmail.com
2017-10-12 01:52:46 +02:00
/* Read an unsigned integer field (anything written using UINT64_FORMAT) */
#define READ_UINT64_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
Simplify the general-purpose 64-bit integer parsing APIs pg_strtouint64() is a wrapper around strtoull/strtoul/_strtoui64, but it seems no longer necessary to have this indirection. msvc/Solution.pm claims HAVE_STRTOULL, so the "MSVC only" part seems unnecessary. Also, we have code in c.h to substitute alternatives for strtoull() if not found, and that would appear to cover all currently supported platforms, so having a further fallback in pg_strtouint64() seems unnecessary. Therefore, we could remove pg_strtouint64(), and use strtoull() directly in all call sites. However, it seems useful to keep a separate notation for parsing exactly 64-bit integers, matching the type definition int64/uint64. For that, add new macros strtoi64() and strtou64() in c.h as thin wrappers around strtol()/strtoul() or strtoll()/stroull(). This makes these functions available everywhere instead of just in the server code, and it makes the function naming notably different from the pg_strtointNN() functions in numutils.c, which have a different API. Discussion: https://www.postgresql.org/message-id/flat/a3df47c9-b1b4-29f2-7e91-427baf8b75a3%40enterprisedb.com
2021-12-17 06:05:54 +01:00
local_node->fldname = strtou64(token, NULL, 10)
pg_stat_statements: Widen query IDs from 32 bits to 64 bits. This takes advantage of the infrastructure introduced by commit 81c5e46c490e2426db243eada186995da5bb0ba7 to greatly reduce the likelihood that two different queries will end up with the same query ID. It's still possible, of course, but whereas before it the chances of a collision reached 25% around 50,000 queries, it will now take more than 3 billion queries. Backward incompatibility: Because the type exposed at the SQL level is int8, users may now see negative query IDs in the pg_stat_statements view (and also, query IDs more than 4 billion, which was the old limit). Patch by me, reviewed by Michael Paquier and Peter Geoghegan. Discussion: http://postgr.es/m/CA+TgmobG_Kp4cBKFmsznUAaM1GWW6hhRNiZC0KjRMOOeYnz5Yw@mail.gmail.com
2017-10-12 01:52:46 +02:00
C comments: "a" <--> "an" corrections Reported-by: Michael Paquier, Abhijit Menon-Sen Discussion: https://postgr.es/m/20180305045854.GB2266@paquier.xyz Author: Michael Paquier, Abhijit Menon-Sen, me
2018-03-29 21:18:53 +02:00
/* Read a long integer field (anything written as ":fldname %ld") */
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
#define READ_LONG_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = atol(token)
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* Read an OID field (don't hard-wire assumption that OID is same as uint) */
#define READ_OID_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = atooid(token)
/* Read a char field (ie, one ascii character) */
#define READ_CHAR_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
Fix output of char node fields WRITE_CHAR_FIELD() didn't do any escaping, so that for example a zero byte would cause the whole output string to be truncated. To fix, pass the char through outToken(), so it is escaped like a string. Adjust the reading side to handle this.
2017-06-22 04:57:23 +02:00
/* avoid overhead of calling debackslash() for one char */ \
local_node->fldname = (length == 0) ? '\0' : (token[0] == '\\' ? token[1] : token[0])
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* Read an enumerated-type field that was written as an integer code */
#define READ_ENUM_FIELD(fldname, enumtype) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = (enumtype) atoi(token)
/* Read a float field */
#define READ_FLOAT_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = atof(token)
/* Read a boolean field */
#define READ_BOOL_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = strtobool(token)
/* Read a character-string field */
#define READ_STRING_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = nullable_string(token, length)
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
/* Read a parse location field (and possibly throw away the value) */
#ifdef WRITE_READ_PARSE_PLAN_TREES
#define READ_LOCATION_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
local_node->fldname = restore_location_fields ? atoi(token) : -1
#else
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
#define READ_LOCATION_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
token = pg_strtok(&length); /* get field value */ \
Suppress Coverity complaints in readfuncs.c. Coverity is complaining that the value returned by pg_strtok in READ_LOCATION_FIELD and READ_BITMAPSET_FIELD macros is not used. In commit 39bfc94c86f1990e9db8ea3da0e82995cc1b76db, we did this to the other macros to placate compilers that complained when the variable was completely unused, this extends that to the last remaining macros.
2014-01-16 10:55:08 +01:00
(void) token; /* in case not used elsewhere */ \
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
local_node->fldname = -1 /* set field to "unknown" */
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
#endif
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* Read a Node field */
#define READ_NODE_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
Suppress compiler warnings in readfuncs.c. Commit 7357558fc8866e3a449aa9473c419b593d67b5b6 introduced "(void) token;" into the READ_TEMP_LOCALS() macro, to suppress complaints from gcc 4.6 when the value of token was not used anywhere in a particular node-read function. However, this just moved the warning around: inspection of buildfarm results shows that some compilers are now complaining that token is being read before it's set. Revert the READ_TEMP_LOCALS() macro change and instead put "(void) token;" into READ_NODE_FIELD(), which is the principal culprit for cases where the warning might occur. In principle we might need the same in READ_BITMAPSET_FIELD() and/or READ_LOCATION_FIELD(), but it seems unlikely that a node would consist only of such fields, so I'll leave them alone for now.
2012-07-01 04:27:49 +02:00
(void) token; /* in case not used elsewhere */ \
Rewrite nodeRead() in a less obfuscated fashion, per discussion with Neil Conway.
2004-05-06 16:01:33 +02:00
local_node->fldname = nodeRead(NULL, 0)
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
Support column-level privileges, as required by SQL standard. Stephen Frost, with help from KaiGai Kohei and others
2009-01-22 21:16:10 +01:00
/* Read a bitmapset field */
#define READ_BITMAPSET_FIELD(fldname) \
token = pg_strtok(&length); /* skip :fldname */ \
Suppress Coverity complaints in readfuncs.c. Coverity is complaining that the value returned by pg_strtok in READ_LOCATION_FIELD and READ_BITMAPSET_FIELD macros is not used. In commit 39bfc94c86f1990e9db8ea3da0e82995cc1b76db, we did this to the other macros to placate compilers that complained when the variable was completely unused, this extends that to the last remaining macros.
2014-01-16 10:55:08 +01:00
(void) token; /* in case not used elsewhere */ \
Support column-level privileges, as required by SQL standard. Stephen Frost, with help from KaiGai Kohei and others
2009-01-22 21:16:10 +01:00
local_node->fldname = _readBitmapset()
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/* Read an attribute number array */
#define READ_ATTRNUMBER_ARRAY(fldname, len) \
token = pg_strtok(&length); /* skip :fldname */ \
Get rid of trailing semicolons in C macro definitions. Writing a trailing semicolon in a macro is almost never the right thing, because you almost always want to write a semicolon after each macro call instead. (Even if there was some reason to prefer not to, pgindent would probably make a hash of code formatted that way; so within PG the rule should basically be "don't do it".) Thus, if we have a semi inside the macro, the compiler sees "something;;". Much of the time the extra empty statement is harmless, but it could lead to mysterious syntax errors at call sites. In perhaps an overabundance of neatnik-ism, let's run around and get rid of the excess semicolons whereever possible. The only thing worse than a mysterious syntax error is a mysterious syntax error that only happens in the back branches; therefore, backpatch these changes where relevant, which is most of them because most of these mistakes are old. (The lack of reported problems shows that this is largely a hypothetical issue, but still, it could bite us in some future patch.) John Naylor and Tom Lane Discussion: https://postgr.es/m/CACPNZCs0qWTqJ2QUSGJ07B7uvAvzMb-KbG2q+oo+J3tsWN5cqw@mail.gmail.com
2020-05-01 23:28:00 +02:00
local_node->fldname = readAttrNumberCols(len)
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/* Read an oid array */
#define READ_OID_ARRAY(fldname, len) \
token = pg_strtok(&length); /* skip :fldname */ \
Get rid of trailing semicolons in C macro definitions. Writing a trailing semicolon in a macro is almost never the right thing, because you almost always want to write a semicolon after each macro call instead. (Even if there was some reason to prefer not to, pgindent would probably make a hash of code formatted that way; so within PG the rule should basically be "don't do it".) Thus, if we have a semi inside the macro, the compiler sees "something;;". Much of the time the extra empty statement is harmless, but it could lead to mysterious syntax errors at call sites. In perhaps an overabundance of neatnik-ism, let's run around and get rid of the excess semicolons whereever possible. The only thing worse than a mysterious syntax error is a mysterious syntax error that only happens in the back branches; therefore, backpatch these changes where relevant, which is most of them because most of these mistakes are old. (The lack of reported problems shows that this is largely a hypothetical issue, but still, it could bite us in some future patch.) John Naylor and Tom Lane Discussion: https://postgr.es/m/CACPNZCs0qWTqJ2QUSGJ07B7uvAvzMb-KbG2q+oo+J3tsWN5cqw@mail.gmail.com
2020-05-01 23:28:00 +02:00
local_node->fldname = readOidCols(len)
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/* Read an int array */
#define READ_INT_ARRAY(fldname, len) \
token = pg_strtok(&length); /* skip :fldname */ \
Get rid of trailing semicolons in C macro definitions. Writing a trailing semicolon in a macro is almost never the right thing, because you almost always want to write a semicolon after each macro call instead. (Even if there was some reason to prefer not to, pgindent would probably make a hash of code formatted that way; so within PG the rule should basically be "don't do it".) Thus, if we have a semi inside the macro, the compiler sees "something;;". Much of the time the extra empty statement is harmless, but it could lead to mysterious syntax errors at call sites. In perhaps an overabundance of neatnik-ism, let's run around and get rid of the excess semicolons whereever possible. The only thing worse than a mysterious syntax error is a mysterious syntax error that only happens in the back branches; therefore, backpatch these changes where relevant, which is most of them because most of these mistakes are old. (The lack of reported problems shows that this is largely a hypothetical issue, but still, it could bite us in some future patch.) John Naylor and Tom Lane Discussion: https://postgr.es/m/CACPNZCs0qWTqJ2QUSGJ07B7uvAvzMb-KbG2q+oo+J3tsWN5cqw@mail.gmail.com
2020-05-01 23:28:00 +02:00
local_node->fldname = readIntCols(len)
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/* Read a bool array */
#define READ_BOOL_ARRAY(fldname, len) \
token = pg_strtok(&length); /* skip :fldname */ \
Get rid of trailing semicolons in C macro definitions. Writing a trailing semicolon in a macro is almost never the right thing, because you almost always want to write a semicolon after each macro call instead. (Even if there was some reason to prefer not to, pgindent would probably make a hash of code formatted that way; so within PG the rule should basically be "don't do it".) Thus, if we have a semi inside the macro, the compiler sees "something;;". Much of the time the extra empty statement is harmless, but it could lead to mysterious syntax errors at call sites. In perhaps an overabundance of neatnik-ism, let's run around and get rid of the excess semicolons whereever possible. The only thing worse than a mysterious syntax error is a mysterious syntax error that only happens in the back branches; therefore, backpatch these changes where relevant, which is most of them because most of these mistakes are old. (The lack of reported problems shows that this is largely a hypothetical issue, but still, it could bite us in some future patch.) John Naylor and Tom Lane Discussion: https://postgr.es/m/CACPNZCs0qWTqJ2QUSGJ07B7uvAvzMb-KbG2q+oo+J3tsWN5cqw@mail.gmail.com
2020-05-01 23:28:00 +02:00
local_node->fldname = readBoolCols(len)
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* Routine exit */
#define READ_DONE() \
return local_node
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
/*
* NOTE: use atoi() to read values written with %d, or atoui() to read
* values written with %u in outfuncs.c. An exception is OID values,
* for which use atooid(). (As of 7.1, outfuncs.c writes OIDs as %u,
* but this will probably change in the future.)
*/
#define atoui(x) ((unsigned int) strtoul((x), NULL, 10))
#define strtobool(x) ((*(x) == 't') ? true : false)
#define nullable_string(token,length) \
More janitorial work: remove the explicit casting of NULL literals to a pointer type when it is not necessary to do so. For future reference, casting NULL to a pointer type is only necessary when (a) invoking a function AND either (b) the function has no prototype OR (c) the function is a varargs function.
2004-01-07 19:56:30 +01:00
((length) == 0 ? NULL : debackslash(token, length))
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
Support column-level privileges, as required by SQL standard. Stephen Frost, with help from KaiGai Kohei and others
2009-01-22 21:16:10 +01:00
/*
* _readBitmapset
*/
static Bitmapset *
_readBitmapset(void)
{
Bitmapset *result = NULL;
8.4 pgindent run, with new combined Linux/FreeBSD/MinGW typedef list provided by Andrew.
2009-06-11 16:49:15 +02:00
Support column-level privileges, as required by SQL standard. Stephen Frost, with help from KaiGai Kohei and others
2009-01-22 21:16:10 +01:00
READ_TEMP_LOCALS();
token = pg_strtok(&length);
if (token == NULL)
elog(ERROR, "incomplete Bitmapset structure");
if (length != 1 || token[0] != '(')
elog(ERROR, "unrecognized token: \"%.*s\"", length, token);
token = pg_strtok(&length);
if (token == NULL)
elog(ERROR, "incomplete Bitmapset structure");
if (length != 1 || token[0] != 'b')
elog(ERROR, "unrecognized token: \"%.*s\"", length, token);
for (;;)
{
int val;
char *endptr;
token = pg_strtok(&length);
if (token == NULL)
elog(ERROR, "unterminated Bitmapset structure");
if (length == 1 && token[0] == ')')
break;
val = (int) strtol(token, &endptr, 10);
if (endptr != token + length)
elog(ERROR, "unrecognized integer: \"%.*s\"", length, token);
result = bms_add_member(result, val);
}
return result;
}
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
/*
* for use by extensions which define extensible nodes
*/
Bitmapset *
readBitmapset(void)
{
return _readBitmapset();
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readQuery
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static Query *
Makeover for Unixware 7.1.1 * Makefile: Add more standard targets. Improve shell redirection in GNU make detection. * src/backend/access/transam/rmgr.c: Fix incorrect(?) C. * src/backend/libpq/pqcomm.c (StreamConnection): Work around accept() bug. * src/include/port/unixware.h: ...with help from here. * src/backend/nodes/print.c (plannode_type): Remove some "break"s after "return"s. * src/backend/tcop/dest.c (DestToFunction): ditto. * src/backend/nodes/readfuncs.c: Add proper prototypes. * src/backend/utils/adt/numutils.c (pg_atoi): Cope specially with strtol() setting EINVAL. This saves us from creating an extra set of regression test output for the affected systems. * src/include/storage/s_lock.h (tas): Correct prototype. * src/interfaces/libpq/fe-connect.c (parseServiceInfo): Don't use variable as dimension in array definition. * src/makefiles/Makefile.unixware: Add support for GCC. * src/template/unixware: same here * src/test/regress/expected/abstime-solaris-1947.out: Adjust whitespace. * src/test/regress/expected/horology-solaris-1947.out: Part of this file was evidently missing. * src/test/regress/pg_regress.sh: Fix shell. mkdir -p returns non-zero if the directory exists. * src/test/regress/resultmap: Add entries for Unixware.
2000-10-23 00:15:13 +02:00
_readQuery(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(Query);
READ_ENUM_FIELD(commandType, CmdType);
READ_ENUM_FIELD(querySource, QuerySource);
pg_stat_statements: Widen query IDs from 32 bits to 64 bits. This takes advantage of the infrastructure introduced by commit 81c5e46c490e2426db243eada186995da5bb0ba7 to greatly reduce the likelihood that two different queries will end up with the same query ID. It's still possible, of course, but whereas before it the chances of a collision reached 25% around 50,000 queries, it will now take more than 3 billion queries. Backward incompatibility: Because the type exposed at the SQL level is int8, users may now see negative query IDs in the pg_stat_statements view (and also, query IDs more than 4 billion, which was the old limit). Patch by me, reviewed by Michael Paquier and Peter Geoghegan. Discussion: http://postgr.es/m/CA+TgmobG_Kp4cBKFmsznUAaM1GWW6hhRNiZC0KjRMOOeYnz5Yw@mail.gmail.com
2017-10-12 01:52:46 +02:00
local_node->queryId = UINT64CONST(0); /* not saved in output format */
Portal and memory management infrastructure for extended query protocol. Both plannable queries and utility commands are now always executed within Portals, which have been revamped so that they can handle the load (they used to be good only for single SELECT queries). Restructure code to push command-completion-tag selection logic out of postgres.c, so that it won't have to be duplicated between simple and extended queries. initdb forced due to addition of a field to Query nodes.
2003-05-02 22:54:36 +02:00
READ_BOOL_FIELD(canSetTag);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(utilityStmt);
READ_INT_FIELD(resultRelation);
READ_BOOL_FIELD(hasAggs);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
READ_BOOL_FIELD(hasWindowFuncs);
Improve parser's and planner's handling of set-returning functions. Teach the parser to reject misplaced set-returning functions during parse analysis using p_expr_kind, in much the same way as we do for aggregates and window functions (cf commit eaccfded9). While this isn't complete (it misses nesting-based restrictions), it's much better than the previous error reporting for such cases, and it allows elimination of assorted ad-hoc expression_returns_set() error checks. We could add nesting checks later if it seems important to catch all cases at parse time. There is one case the parser will now throw error for although previous versions allowed it, which is SRFs in the tlist of an UPDATE. That never behaved sensibly (since it's ill-defined which generated row should be used to perform the update) and it's hard to see why it should not be treated as an error. It's a release-note-worthy change though. Also, add a new Query field hasTargetSRFs reporting whether there are any SRFs in the targetlist (including GROUP BY/ORDER BY expressions). The parser can now set that basically for free during parse analysis, and we can use it in a number of places to avoid expression_returns_set searches. (There will be more such checks soon.) In some places, this allows decontorting the logic since it's no longer expensive to check for SRFs in the tlist --- so I made the checks parallel to the handling of hasAggs/hasWindowFuncs wherever it seemed appropriate. catversion bump because adding a Query field changes stored rules. Andres Freund and Tom Lane Discussion: <24639.1473782855@sss.pgh.pa.us>
2016-09-13 19:54:24 +02:00
READ_BOOL_FIELD(hasTargetSRFs);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_BOOL_FIELD(hasSubLinks);
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
READ_BOOL_FIELD(hasDistinctOn);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_BOOL_FIELD(hasRecursive);
Support data-modifying commands (INSERT/UPDATE/DELETE) in WITH. This patch implements data-modifying WITH queries according to the semantics that the updates all happen with the same command counter value, and in an unspecified order. Therefore one WITH clause can't see the effects of another, nor can the outer query see the effects other than through the RETURNING values. And attempts to do conflicting updates will have unpredictable results. We'll need to document all that. This commit just fixes the code; documentation updates are waiting on author. Marko Tiikkaja and Hitoshi Harada
2011-02-26 00:56:23 +01:00
READ_BOOL_FIELD(hasModifyingCTE);
When FOR UPDATE/SHARE is used with LIMIT, put the LockRows plan node underneath the Limit node, not atop it. This fixes the old problem that such a query might unexpectedly return fewer rows than the LIMIT says, due to LockRows discarding updated rows. There is a related problem that LockRows might destroy the sort ordering produced by earlier steps; but fixing that by pushing LockRows below Sort would create serious performance problems that are unjustified in many real-world applications, as well as potential deadlock problems from locking many more rows than expected. Instead, keep the present semantics of applying FOR UPDATE after ORDER BY within a single query level; but allow the user to specify the other way by writing FOR UPDATE in a sub-select. To make that work, track whether FOR UPDATE appeared explicitly in sub-selects or got pushed down from the parent, and don't flatten a sub-select that contained an explicit FOR UPDATE.
2009-10-28 15:55:47 +01:00
READ_BOOL_FIELD(hasForUpdate);
Row-Level Security Policies (RLS) Building on the updatable security-barrier views work, add the ability to define policies on tables to limit the set of rows which are returned from a query and which are allowed to be added to a table. Expressions defined by the policy for filtering are added to the security barrier quals of the query, while expressions defined to check records being added to a table are added to the with-check options of the query. New top-level commands are CREATE/ALTER/DROP POLICY and are controlled by the table owner. Row Security is able to be enabled and disabled by the owner on a per-table basis using ALTER TABLE .. ENABLE/DISABLE ROW SECURITY. Per discussion, ROW SECURITY is disabled on tables by default and must be enabled for policies on the table to be used. If no policies exist on a table with ROW SECURITY enabled, a default-deny policy is used and no records will be visible. By default, row security is applied at all times except for the table owner and the superuser. A new GUC, row_security, is added which can be set to ON, OFF, or FORCE. When set to FORCE, row security will be applied even for the table owner and superusers. When set to OFF, row security will be disabled when allowed and an error will be thrown if the user does not have rights to bypass row security. Per discussion, pg_dump sets row_security = OFF by default to ensure that exports and backups will have all data in the table or will error if there are insufficient privileges to bypass row security. A new option has been added to pg_dump, --enable-row-security, to ask pg_dump to export with row security enabled. A new role capability, BYPASSRLS, which can only be set by the superuser, is added to allow other users to be able to bypass row security using row_security = OFF. Many thanks to the various individuals who have helped with the design, particularly Robert Haas for his feedback. Authors include Craig Ringer, KaiGai Kohei, Adam Brightwell, Dean Rasheed, with additional changes and rework by me. Reviewers have included all of the above, Greg Smith, Jeff McCormick, and Robert Haas.
2014-09-19 17:18:35 +02:00
READ_BOOL_FIELD(hasRowSecurity);
SQL-standard function body This adds support for writing CREATE FUNCTION and CREATE PROCEDURE statements for language SQL with a function body that conforms to the SQL standard and is portable to other implementations. Instead of the PostgreSQL-specific AS $$ string literal $$ syntax, this allows writing out the SQL statements making up the body unquoted, either as a single statement: CREATE FUNCTION add(a integer, b integer) RETURNS integer LANGUAGE SQL RETURN a + b; or as a block CREATE PROCEDURE insert_data(a integer, b integer) LANGUAGE SQL BEGIN ATOMIC INSERT INTO tbl VALUES (a); INSERT INTO tbl VALUES (b); END; The function body is parsed at function definition time and stored as expression nodes in a new pg_proc column prosqlbody. So at run time, no further parsing is required. However, this form does not support polymorphic arguments, because there is no more parse analysis done at call time. Dependencies between the function and the objects it uses are fully tracked. A new RETURN statement is introduced. This can only be used inside function bodies. Internally, it is treated much like a SELECT statement. psql needs some new intelligence to keep track of function body boundaries so that it doesn't send off statements when it sees semicolons that are inside a function body. Tested-by: Jaime Casanova <jcasanov@systemguards.com.ec> Reviewed-by: Julien Rouhaud <rjuju123@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/1c11f1eb-f00c-43b7-799d-2d44132c02d7@2ndquadrant.com
2021-04-07 21:30:08 +02:00
READ_BOOL_FIELD(isReturn);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_NODE_FIELD(cteList);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(rtable);
READ_NODE_FIELD(jointree);
READ_NODE_FIELD(targetList);
Identity columns This is the SQL standard-conforming variant of PostgreSQL's serial columns. It fixes a few usability issues that serial columns have: - CREATE TABLE / LIKE copies default but refers to same sequence - cannot add/drop serialness with ALTER TABLE - dropping default does not drop sequence - need to grant separate privileges to sequence - other slight weirdnesses because serial is some kind of special macro Reviewed-by: Vitaly Burovoy <vitaly.burovoy@gmail.com>
2017-04-06 14:33:16 +02:00
READ_ENUM_FIELD(override, OverridingKind);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
READ_NODE_FIELD(onConflict);
Add INSERT/UPDATE/DELETE RETURNING, with basic docs and regression tests. plpgsql support to come later. Along the way, convert execMain's SELECT INTO support into a DestReceiver, in order to eliminate some ugly special cases. Jonah Harris and Tom Lane
2006-08-12 04:52:06 +02:00
READ_NODE_FIELD(returningList);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(groupClause);
Implement GROUP BY DISTINCT With grouping sets, it's possible that some of the grouping sets are duplicate. This is especially common with CUBE and ROLLUP clauses. For example GROUP BY CUBE (a,b), CUBE (b,c) is equivalent to GROUP BY GROUPING SETS ( (a, b, c), (a, b, c), (a, b, c), (a, b), (a, b), (a, b), (a), (a), (a), (c, a), (c, a), (c, a), (c), (b, c), (b), () ) Some of the grouping sets are calculated multiple times, which is mostly unnecessary. This commit implements a new GROUP BY DISTINCT feature, as defined in the SQL standard, which eliminates the duplicate sets. Author: Vik Fearing Reviewed-by: Erik Rijkers, Georgios Kokolatos, Tomas Vondra Discussion: https://postgr.es/m/bf3805a8-d7d1-ae61-fece-761b7ff41ecc@postgresfriends.org
2021-03-18 17:45:38 +01:00
READ_BOOL_FIELD(groupDistinct);
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
READ_NODE_FIELD(groupingSets);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(havingQual);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
READ_NODE_FIELD(windowClause);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(distinctClause);
READ_NODE_FIELD(sortClause);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
Support FETCH FIRST WITH TIES WITH TIES is an option to the FETCH FIRST N ROWS clause (the SQL standard's spelling of LIMIT), where you additionally get rows that compare equal to the last of those N rows by the columns in the mandatory ORDER BY clause. There was a proposal by Andrew Gierth to implement this functionality in a more powerful way that would yield more features, but the other patch had not been finished at this time, so we decided to use this one for now in the spirit of incremental development. Author: Surafel Temesgen <surafel3000@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> Discussion: https://postgr.es/m/CALAY4q9ky7rD_A4vf=FVQvCGngm3LOes-ky0J6euMrg=_Se+ag@mail.gmail.com Discussion: https://postgr.es/m/87o8wvz253.fsf@news-spur.riddles.org.uk
2020-04-07 22:22:13 +02:00
READ_ENUM_FIELD(limitOption, LimitOption);
Improve the representation of FOR UPDATE/FOR SHARE so that we can support both FOR UPDATE and FOR SHARE in one command, as well as both NOWAIT and normal WAIT behavior. The more general code is actually simpler and cleaner.
2006-04-30 20:30:40 +02:00
READ_NODE_FIELD(rowMarks);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(setOperations);
Recognize functional dependency on primary keys. This allows a table's other columns to be referenced without listing them in GROUP BY, so long as the primary key column(s) are listed in GROUP BY. Eventually we should also allow functional dependency on a UNIQUE constraint when the columns are marked NOT NULL, but that has to wait until NOT NULL constraints are represented in pg_constraint, because we need to have pg_constraint OIDs for all the conditions needed to ensure functional dependency. Peter Eisentraut, reviewed by Alex Hunsaker and Tom Lane
2010-08-07 04:44:09 +02:00
READ_NODE_FIELD(constraintDeps);
Fix some minor issues exposed by outfuncs/readfuncs testing. A test patch to pass parse and plan trees through outfuncs + readfuncs exposed several issues that need to be fixed to get clean matches: Query.withCheckOptions failed to get copied; it's intentionally ignored by outfuncs/readfuncs on the grounds that it'd always be NIL anyway in stored rules. This seems less than future-proof, and it's not even saving very much, so just undo the decision and treat the field like all others. Several places that convert a view RTE into a subquery RTE, or similar manipulations, failed to clear out fields that were specific to the original RTE type and should be zero in a subquery RTE. Since readfuncs.c will leave such fields as zero, equalfuncs.c thinks the nodes are different leading to a reported mismatch. It seems like a good idea to clear out the no-longer-needed fields, even though in principle nothing should look at them; the node ought to be indistinguishable from how it would look if we'd built a new node instead of scribbling on the old one. BuildOnConflictExcludedTargetlist randomly set the resname of some TargetEntries to "" not NULL. outfuncs/readfuncs don't distinguish those cases, and so the string will read back in as NULL ... but equalfuncs.c does distinguish. Perhaps we ought to try to make things more consistent in this area --- but it's just useless extra code space for BuildOnConflictExcludedTargetlist to not use NULL here, so I fixed it for now by making it do that. catversion bumped because the change in handling of Query.withCheckOptions affects stored rules. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 21:08:28 +02:00
READ_NODE_FIELD(withCheckOptions);
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
READ_NODE_FIELD(mergeActionList);
READ_BOOL_FIELD(mergeUseOuterJoin);
Change representation of statement lists, and add statement location info. This patch makes several changes that improve the consistency of representation of lists of statements. It's always been the case that the output of parse analysis is a list of Query nodes, whatever the types of the individual statements in the list. This patch brings similar consistency to the outputs of raw parsing and planning steps: * The output of raw parsing is now always a list of RawStmt nodes; the statement-type-dependent nodes are one level down from that. * The output of pg_plan_queries() is now always a list of PlannedStmt nodes, even for utility statements. In the case of a utility statement, "planning" just consists of wrapping a CMD_UTILITY PlannedStmt around the utility node. This list representation is now used in Portal and CachedPlan plan lists, replacing the former convention of intermixing PlannedStmts with bare utility-statement nodes. Now, every list of statements has a consistent head-node type depending on how far along it is in processing. This allows changing many places that formerly used generic "Node *" pointers to use a more specific pointer type, thus reducing the number of IsA() tests and casts needed, as well as improving code clarity. Also, the post-parse-analysis representation of DECLARE CURSOR is changed so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained SELECT remains a child of the DeclareCursorStmt rather than getting flipped around to be the other way. It's now true for both Query and PlannedStmt that utilityStmt is non-null if and only if commandType is CMD_UTILITY. That allows simplifying a lot of places that were testing both fields. (I think some of those were just defensive programming, but in many places, it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.) Because PlannedStmt carries a canSetTag field, we're also able to get rid of some ad-hoc rules about how to reconstruct canSetTag for a bare utility statement; specifically, the assumption that a utility is canSetTag if and only if it's the only one in its list. While I see no near-term need for relaxing that restriction, it's nice to get rid of the ad-hocery. The API of ProcessUtility() is changed so that what it's passed is the wrapper PlannedStmt not just the bare utility statement. This will affect all users of ProcessUtility_hook, but the changes are pretty trivial; see the affected contrib modules for examples of the minimum change needed. (Most compilers should give pointer-type-mismatch warnings for uncorrected code.) There's also a change in the API of ExplainOneQuery_hook, to pass through cursorOptions instead of expecting hook functions to know what to pick. This is needed because of the DECLARE CURSOR changes, but really should have been done in 9.6; it's unlikely that any extant hook functions know about using CURSOR_OPT_PARALLEL_OK. Finally, teach gram.y to save statement boundary locations in RawStmt nodes, and pass those through to Query and PlannedStmt nodes. This allows more intelligent handling of cases where a source query string contains multiple statements. This patch doesn't actually do anything with the information, but a follow-on patch will. (Passing this information through cleanly is the true motivation for these changes; while I think this is all good cleanup, it's unlikely we'd have bothered without this end goal.) catversion bump because addition of location fields to struct Query affects stored rules. This patch is by me, but it owes a good deal to Fabien Coelho who did a lot of preliminary work on the problem, and also reviewed the patch. Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
READ_LOCATION_FIELD(stmt_location);
Reconcile nodes/*funcs.c. The stmt_len changes do not affect behavior. LimitPath has no other support functions, so that part changes only debugging output.
2020-05-26 01:23:48 +02:00
READ_INT_FIELD(stmt_len);
Restructure handling of inheritance queries so that they work with outer joins, and clean things up a good deal at the same time. Append plan node no longer hacks on rangetable at runtime --- instead, all child tables are given their own RT entries during planning. Concept of multiple target tables pushed up into execMain, replacing bug-prone implementation within nodeAppend. Planner now supports generating Append plans for inheritance sets either at the top of the plan (the old way) or at the bottom. Expanding at the bottom is appropriate for tables used as sources, since they may appear inside an outer join; but we must still expand at the top when the target of an UPDATE or DELETE is an inheritance set, because we actually need a different targetlist and junkfilter for each target table in that case. Fortunately a target table can't be inside an outer join... Bizarre mutual recursion between union_planner and prepunion.c is gone --- in fact, union_planner doesn't really have much to do with union queries anymore, so I renamed it grouping_planner.
2000-11-12 01:37:02 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Change some labels in bootparse to make ctags happy. Clean up outfunc/readfunc code and add missing fields for Query structure and new Union fields. Fix optimizer bug shown in new \do command. Change WARN to ERROR in contrib and regression stuff.
1998-01-06 19:53:02 +01:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readNotifyStmt
First phase of SCHEMA changes, concentrating on fixing the grammar and the parsetree representation. As yet we don't *do* anything with schema names, just drop 'em on the floor; but you can enter schema-compatible command syntax, and there's even a primitive CREATE SCHEMA command. No doc updates yet, except to note that you can now extract a field from a function-returning-row's result with (foo(...)).fieldname.
2002-03-21 17:02:16 +01:00
*/
static NotifyStmt *
_readNotifyStmt(void)
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(NotifyStmt);
First phase of SCHEMA changes, concentrating on fixing the grammar and the parsetree representation. As yet we don't *do* anything with schema names, just drop 'em on the floor; but you can enter schema-compatible command syntax, and there's even a primitive CREATE SCHEMA command. No doc updates yet, except to note that you can now extract a field from a function-returning-row's result with (foo(...)).fieldname.
2002-03-21 17:02:16 +01:00
Add a bunch of new error location reports to parse-analysis error messages. There are still some weak spots around JOIN USING and relation alias lists, but most errors reported within backend/parser/ now have locations.
2008-09-01 22:42:46 +02:00
READ_STRING_FIELD(conditionname);
Replace the pg_listener-based LISTEN/NOTIFY mechanism with an in-memory queue. In addition, add support for a "payload" string to be passed along with each notify event. This implementation should be significantly more efficient than the old one, and is also more compatible with Hot Standby usage. There is not yet any facility for HS slaves to receive notifications generated on the master, although such a thing is possible in future. Joachim Wieland, reviewed by Jeff Davis; also hacked on by me.
2010-02-16 23:34:57 +01:00
READ_STRING_FIELD(payload);
First phase of SCHEMA changes, concentrating on fixing the grammar and the parsetree representation. As yet we don't *do* anything with schema names, just drop 'em on the floor; but you can enter schema-compatible command syntax, and there's even a primitive CREATE SCHEMA command. No doc updates yet, except to note that you can now extract a field from a function-returning-row's result with (foo(...)).fieldname.
2002-03-21 17:02:16 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
First phase of SCHEMA changes, concentrating on fixing the grammar and the parsetree representation. As yet we don't *do* anything with schema names, just drop 'em on the floor; but you can enter schema-compatible command syntax, and there's even a primitive CREATE SCHEMA command. No doc updates yet, except to note that you can now extract a field from a function-returning-row's result with (foo(...)).fieldname.
2002-03-21 17:02:16 +01:00
}
Restructure parsetree representation of DECLARE CURSOR: now it's a utility statement (DeclareCursorStmt) with a SELECT query dangling from it, rather than a SELECT query with a few unusual fields in it. Add code to determine whether a planned query can safely be run backwards. If DECLARE CURSOR specifies SCROLL, ensure that the plan can be run backwards by adding a Materialize plan node if it can't. Without SCROLL, you get an error if you try to fetch backwards from a cursor that can't handle it. (There is still some discussion about what the exact behavior should be, but this is necessary infrastructure in any case.) Along the way, make EXPLAIN DECLARE CURSOR work.
2003-03-10 04:53:52 +01:00
/*
* _readDeclareCursorStmt
*/
static DeclareCursorStmt *
_readDeclareCursorStmt(void)
{
READ_LOCALS(DeclareCursorStmt);
READ_STRING_FIELD(portalname);
READ_INT_FIELD(options);
READ_NODE_FIELD(query);
READ_DONE();
}
WITH CHECK OPTION support for auto-updatable VIEWs For simple views which are automatically updatable, this patch allows the user to specify what level of checking should be done on records being inserted or updated. For 'LOCAL CHECK', new tuples are validated against the conditionals of the view they are being inserted into, while for 'CASCADED CHECK' the new tuples are validated against the conditionals for all views involved (from the top down). This option is part of the SQL specification. Dean Rasheed, reviewed by Pavel Stehule
2013-07-18 23:10:16 +02:00
/*
* _readWithCheckOption
*/
static WithCheckOption *
_readWithCheckOption(void)
{
READ_LOCALS(WithCheckOption);
Perform RLS WITH CHECK before constraints, etc The RLS capability is built on top of the WITH CHECK OPTION system which was added for auto-updatable views, however, unlike WCOs on views (which are mandated by the SQL spec to not fire until after all other constraints and checks are done), it makes much more sense for RLS checks to happen earlier than constraint and uniqueness checks. This patch reworks the structure which holds the WCOs a bit to be explicitly either VIEW or RLS checks and the RLS-related checks are done prior to the constraint and uniqueness checks. This also allows better error reporting as we are now reporting when a violation is due to a WITH CHECK OPTION and when it's due to an RLS policy violation, which was independently noted by Craig Ringer as being confusing. The documentation is also updated to include a paragraph about when RLS WITH CHECK handling is performed, as there have been a number of questions regarding that and the documentation was previously silent on the matter. Author: Dean Rasheed, with some kabitzing and comment changes by me.
2015-04-25 02:34:26 +02:00
READ_ENUM_FIELD(kind, WCOKind);
READ_STRING_FIELD(relname);
RLS refactoring This refactors rewrite/rowsecurity.c to simplify the handling of the default deny case (reducing the number of places where we check for and add the default deny policy from three to one) by splitting up the retrival of the policies from the application of them. This also allowed us to do away with the policy_id field. A policy_name field was added for WithCheckOption policies and is used in error reporting, when available. Patch by Dean Rasheed, with various mostly cosmetic changes by me. Back-patch to 9.5 where RLS was introduced to avoid unnecessary differences, since we're still in alpha, per discussion with Robert.
2015-09-15 21:49:31 +02:00
READ_STRING_FIELD(polname);
WITH CHECK OPTION support for auto-updatable VIEWs For simple views which are automatically updatable, this patch allows the user to specify what level of checking should be done on records being inserted or updated. For 'LOCAL CHECK', new tuples are validated against the conditionals of the view they are being inserted into, while for 'CASCADED CHECK' the new tuples are validated against the conditionals for all views involved (from the top down). This option is part of the SQL specification. Dean Rasheed, reviewed by Pavel Stehule
2013-07-18 23:10:16 +02:00
READ_NODE_FIELD(qual);
READ_BOOL_FIELD(cascaded);
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
* _readSortGroupClause
Change some labels in bootparse to make ctags happy. Clean up outfunc/readfunc code and add missing fields for Query structure and new Union fields. Fix optimizer bug shown in new \do command. Change WARN to ERROR in contrib and regression stuff.
1998-01-06 19:53:02 +01:00
*/
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
static SortGroupClause *
_readSortGroupClause(void)
Change some labels in bootparse to make ctags happy. Clean up outfunc/readfunc code and add missing fields for Query structure and new Union fields. Fix optimizer bug shown in new \do command. Change WARN to ERROR in contrib and regression stuff.
1998-01-06 19:53:02 +01:00
{
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
READ_LOCALS(SortGroupClause);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_UINT_FIELD(tleSortGroupRef);
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
READ_OID_FIELD(eqop);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(sortop);
Support ORDER BY ... NULLS FIRST/LAST, and add ASC/DESC/NULLS FIRST/NULLS LAST per-column options for btree indexes. The planner's support for this is still pretty rudimentary; it does not yet know how to plan mergejoins with nondefault ordering options. The documentation is pretty rudimentary, too. I'll work on improving that stuff later. Note incompatible change from prior behavior: ORDER BY ... USING will now be rejected if the operator is not a less-than or greater-than member of some btree opclass. This prevents less-than-sane behavior if an operator that doesn't actually define a proper sort ordering is selected.
2007-01-09 03:14:16 +01:00
READ_BOOL_FIELD(nulls_first);
Provide hashing support for arrays. The core of this patch is hash_array() and associated typcache infrastructure, which works just about exactly like the existing support for array comparison. In addition I did some work to ensure that the planner won't think that an array type is hashable unless its element type is hashable, and similarly for sorting. This includes adding a datatype parameter to op_hashjoinable and op_mergejoinable, and adding an explicit "hashable" flag to SortGroupClause. The lack of a cross-check on the element type was a pre-existing bug in mergejoin support --- but it didn't matter so much before, because if you couldn't sort the element type there wasn't any good alternative to failing anyhow. Now that we have the alternative of hashing the array type, there are cases where we can avoid a failure by being picky at the planner stage, so it's time to be picky. The issue of exactly how to combine the per-element hash values to produce an array hash is still open for discussion, but the rest of this is pretty solid, so I'll commit it as-is.
2010-10-31 02:55:20 +01:00
READ_BOOL_FIELD(hashable);
Replaced targetlist entry in GroupClause by reference number in Resdom and GroupClause so changing of resno's doesn't confuse the grouping any more. Jan
1999-05-12 17:02:39 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
/*
* _readGroupingSet
*/
static GroupingSet *
_readGroupingSet(void)
{
READ_LOCALS(GroupingSet);
READ_ENUM_FIELD(kind, GroupingSetKind);
READ_NODE_FIELD(content);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
/*
* _readWindowClause
*/
static WindowClause *
_readWindowClause(void)
{
READ_LOCALS(WindowClause);
READ_STRING_FIELD(name);
READ_STRING_FIELD(refname);
READ_NODE_FIELD(partitionClause);
READ_NODE_FIELD(orderClause);
Add some basic support for window frame clauses to the window-functions patch. This includes the ability to force the frame to cover the whole partition, and the ability to make the frame end exactly on the current row rather than its last ORDER BY peer. Supporting any more of the full SQL frame-clause syntax will require nontrivial hacking on the window aggregate code, so it'll have to wait for 8.5 or beyond.
2008-12-31 01:08:39 +01:00
READ_INT_FIELD(frameOptions);
Extend the set of frame options supported for window functions. This patch allows the frame to start from CURRENT ROW (in either RANGE or ROWS mode), and it also adds support for ROWS n PRECEDING and ROWS n FOLLOWING start and end points. (RANGE value PRECEDING/FOLLOWING isn't there yet --- the grammar works, but that's all.) Hitoshi Harada, reviewed by Pavel Stehule
2010-02-12 18:33:21 +01:00
READ_NODE_FIELD(startOffset);
READ_NODE_FIELD(endOffset);
Teach planner and executor about monotonic window funcs Window functions such as row_number() always return a value higher than the previously returned value for tuples in any given window partition. Traditionally queries such as; SELECT * FROM ( SELECT *, row_number() over (order by c) rn FROM t ) t WHERE rn <= 10; were executed fairly inefficiently. Neither the query planner nor the executor knew that once rn made it to 11 that nothing further would match the outer query's WHERE clause. It would blindly continue until all tuples were exhausted from the subquery. Here we implement means to make the above execute more efficiently. This is done by way of adding a pg_proc.prosupport function to various of the built-in window functions and adding supporting code to allow the support function to inform the planner if the window function is monotonically increasing, monotonically decreasing, both or neither. The planner is then able to make use of that information and possibly allow the executor to short-circuit execution by way of adding a "run condition" to the WindowAgg to allow it to determine if some of its execution work can be skipped. This "run condition" is not like a normal filter. These run conditions are only built using quals comparing values to monotonic window functions. For monotonic increasing functions, quals making use of the btree operators for <, <= and = can be used (assuming the window function column is on the left). You can see here that once such a condition becomes false that a monotonic increasing function could never make it subsequently true again. For monotonically decreasing functions the >, >= and = btree operators for the given type can be used for run conditions. The best-case situation for this is when there is a single WindowAgg node without a PARTITION BY clause. Here when the run condition becomes false the WindowAgg node can simply return NULL. No more tuples will ever match the run condition. It's a little more complex when there is a PARTITION BY clause. In this case, we cannot return NULL as we must still process other partitions. To speed this case up we pull tuples from the outer plan to check if they're from the same partition and simply discard them if they are. When we find a tuple belonging to another partition we start processing as normal again until the run condition becomes false or we run out of tuples to process. When there are multiple WindowAgg nodes to evaluate then this complicates the situation. For intermediate WindowAggs we must ensure we always return all tuples to the calling node. Any filtering done could lead to incorrect results in WindowAgg nodes above. For all intermediate nodes, we can still save some work when the run condition becomes false. We've no need to evaluate the WindowFuncs anymore. Other WindowAgg nodes cannot reference the value of these and these tuples will not appear in the final result anyway. The savings here are small in comparison to what can be saved in the top-level WingowAgg, but still worthwhile. Intermediate WindowAgg nodes never filter out tuples, but here we change WindowAgg so that the top-level WindowAgg filters out tuples that don't match the intermediate WindowAgg node's run condition. Such filters appear in the "Filter" clause in EXPLAIN for the top-level WindowAgg node. Here we add prosupport functions to allow the above to work for; row_number(), rank(), dense_rank(), count(*) and count(expr). It appears technically possible to do the same for min() and max(), however, it seems unlikely to be useful enough, so that's not done here. Bump catversion Author: David Rowley Reviewed-by: Andy Fan, Zhihong Yu Discussion: https://postgr.es/m/CAApHDvqvp3At8++yF8ij06sdcoo1S_b2YoaT9D4Nf+MObzsrLQ@mail.gmail.com
2022-04-08 00:34:36 +02:00
READ_NODE_FIELD(runCondition);
Support all SQL:2011 options for window frame clauses. This patch adds the ability to use "RANGE offset PRECEDING/FOLLOWING" frame boundaries in window functions. We'd punted on that back in the original patch to add window functions, because it was not clear how to do it in a reasonably data-type-extensible fashion. That problem is resolved here by adding the ability for btree operator classes to provide an "in_range" support function that defines how to add or subtract the RANGE offset value. Factoring it this way also allows the operator class to avoid overflow problems near the ends of the datatype's range, if it wishes to expend effort on that. (In the committed patch, the integer opclasses handle that issue, but it did not seem worth the trouble to avoid overflow failures for datetime types.) The patch includes in_range support for the integer_ops opfamily (int2/int4/int8) as well as the standard datetime types. Support for other numeric types has been requested, but that seems like suitable material for a follow-on patch. In addition, the patch adds GROUPS mode which counts the offset in ORDER-BY peer groups rather than rows, and it adds the frame_exclusion options specified by SQL:2011. As far as I can see, we are now fully up to spec on window framing options. Existing behaviors remain unchanged, except that I changed the errcode for a couple of existing error reports to meet the SQL spec's expectation that negative "offset" values should be reported as SQLSTATE 22013. Internally and in relevant parts of the documentation, we now consistently use the terminology "offset PRECEDING/FOLLOWING" rather than "value PRECEDING/FOLLOWING", since the term "value" is confusingly vague. Oliver Ford, reviewed and whacked around some by me Discussion: https://postgr.es/m/CAGMVOdu9sivPAxbNN0X+q19Sfv9edEPv=HibOJhB14TJv_RCQg@mail.gmail.com
2018-02-07 06:06:50 +01:00
READ_OID_FIELD(startInRangeFunc);
READ_OID_FIELD(endInRangeFunc);
READ_OID_FIELD(inRangeColl);
READ_BOOL_FIELD(inRangeAsc);
READ_BOOL_FIELD(inRangeNullsFirst);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
READ_UINT_FIELD(winref);
READ_BOOL_FIELD(copiedOrder);
READ_DONE();
}
Improve the representation of FOR UPDATE/FOR SHARE so that we can support both FOR UPDATE and FOR SHARE in one command, as well as both NOWAIT and normal WAIT behavior. The more general code is actually simpler and cleaner.
2006-04-30 20:30:40 +02:00
/*
* _readRowMarkClause
*/
static RowMarkClause *
_readRowMarkClause(void)
{
READ_LOCALS(RowMarkClause);
READ_UINT_FIELD(rti);
Improve concurrency of foreign key locking This patch introduces two additional lock modes for tuples: "SELECT FOR KEY SHARE" and "SELECT FOR NO KEY UPDATE". These don't block each other, in contrast with already existing "SELECT FOR SHARE" and "SELECT FOR UPDATE". UPDATE commands that do not modify the values stored in the columns that are part of the key of the tuple now grab a SELECT FOR NO KEY UPDATE lock on the tuple, allowing them to proceed concurrently with tuple locks of the FOR KEY SHARE variety. Foreign key triggers now use FOR KEY SHARE instead of FOR SHARE; this means the concurrency improvement applies to them, which is the whole point of this patch. The added tuple lock semantics require some rejiggering of the multixact module, so that the locking level that each transaction is holding can be stored alongside its Xid. Also, multixacts now need to persist across server restarts and crashes, because they can now represent not only tuple locks, but also tuple updates. This means we need more careful tracking of lifetime of pg_multixact SLRU files; since they now persist longer, we require more infrastructure to figure out when they can be removed. pg_upgrade also needs to be careful to copy pg_multixact files over from the old server to the new, or at least part of multixact.c state, depending on the versions of the old and new servers. Tuple time qualification rules (HeapTupleSatisfies routines) need to be careful not to consider tuples with the "is multi" infomask bit set as being only locked; they might need to look up MultiXact values (i.e. possibly do pg_multixact I/O) to find out the Xid that updated a tuple, whereas they previously were assured to only use information readily available from the tuple header. This is considered acceptable, because the extra I/O would involve cases that would previously cause some commands to block waiting for concurrent transactions to finish. Another important change is the fact that locking tuples that have previously been updated causes the future versions to be marked as locked, too; this is essential for correctness of foreign key checks. This causes additional WAL-logging, also (there was previously a single WAL record for a locked tuple; now there are as many as updated copies of the tuple there exist.) With all this in place, contention related to tuples being checked by foreign key rules should be much reduced. As a bonus, the old behavior that a subtransaction grabbing a stronger tuple lock than the parent (sub)transaction held on a given tuple and later aborting caused the weaker lock to be lost, has been fixed. Many new spec files were added for isolation tester framework, to ensure overall behavior is sane. There's probably room for several more tests. There were several reviewers of this patch; in particular, Noah Misch and Andres Freund spent considerable time in it. Original idea for the patch came from Simon Riggs, after a problem report by Joel Jacobson. Most code is from me, with contributions from Marti Raudsepp, Alexander Shulgin, Noah Misch and Andres Freund. This patch was discussed in several pgsql-hackers threads; the most important start at the following message-ids: AANLkTimo9XVcEzfiBR-ut3KVNDkjm2Vxh+t8kAmWjPuv@mail.gmail.com 1290721684-sup-3951@alvh.no-ip.org 1294953201-sup-2099@alvh.no-ip.org 1320343602-sup-2290@alvh.no-ip.org 1339690386-sup-8927@alvh.no-ip.org 4FE5FF020200002500048A3D@gw.wicourts.gov 4FEAB90A0200002500048B7D@gw.wicourts.gov
2013-01-23 16:04:59 +01:00
READ_ENUM_FIELD(strength, LockClauseStrength);
Implement SKIP LOCKED for row-level locks This clause changes the behavior of SELECT locking clauses in the presence of locked rows: instead of causing a process to block waiting for the locks held by other processes (or raise an error, with NOWAIT), SKIP LOCKED makes the new reader skip over such rows. While this is not appropriate behavior for general purposes, there are some cases in which it is useful, such as queue-like tables. Catalog version bumped because this patch changes the representation of stored rules. Reviewed by Craig Ringer (based on a previous attempt at an implementation by Simon Riggs, who also provided input on the syntax used in the current patch), David Rowley, and Álvaro Herrera. Author: Thomas Munro
2014-10-07 22:23:34 +02:00
READ_ENUM_FIELD(waitPolicy, LockWaitPolicy);
When FOR UPDATE/SHARE is used with LIMIT, put the LockRows plan node underneath the Limit node, not atop it. This fixes the old problem that such a query might unexpectedly return fewer rows than the LIMIT says, due to LockRows discarding updated rows. There is a related problem that LockRows might destroy the sort ordering produced by earlier steps; but fixing that by pushing LockRows below Sort would create serious performance problems that are unjustified in many real-world applications, as well as potential deadlock problems from locking many more rows than expected. Instead, keep the present semantics of applying FOR UPDATE after ORDER BY within a single query level; but allow the user to specify the other way by writing FOR UPDATE in a sub-select. To make that work, track whether FOR UPDATE appeared explicitly in sub-selects or got pushed down from the parent, and don't flatten a sub-select that contained an explicit FOR UPDATE.
2009-10-28 15:55:47 +01:00
READ_BOOL_FIELD(pushedDown);
Improve the representation of FOR UPDATE/FOR SHARE so that we can support both FOR UPDATE and FOR SHARE in one command, as well as both NOWAIT and normal WAIT behavior. The more general code is actually simpler and cleaner.
2006-04-30 20:30:40 +02:00
READ_DONE();
}
SEARCH and CYCLE clauses This adds the SQL standard feature that adds the SEARCH and CYCLE clauses to recursive queries to be able to do produce breadth- or depth-first search orders and detect cycles. These clauses can be rewritten into queries using existing syntax, and that is what this patch does in the rewriter. Reviewed-by: Vik Fearing <vik@postgresfriends.org> Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/db80ceee-6f97-9b4a-8ee8-3ba0c58e5be2@2ndquadrant.com
2021-02-01 13:54:59 +01:00
/*
* _readCTESearchClause
*/
static CTESearchClause *
_readCTESearchClause(void)
{
READ_LOCALS(CTESearchClause);
READ_NODE_FIELD(search_col_list);
READ_BOOL_FIELD(search_breadth_first);
READ_STRING_FIELD(search_seq_column);
READ_LOCATION_FIELD(location);
READ_DONE();
}
/*
* _readCTECycleClause
*/
static CTECycleClause *
_readCTECycleClause(void)
{
READ_LOCALS(CTECycleClause);
READ_NODE_FIELD(cycle_col_list);
READ_STRING_FIELD(cycle_mark_column);
READ_NODE_FIELD(cycle_mark_value);
READ_NODE_FIELD(cycle_mark_default);
READ_STRING_FIELD(cycle_path_column);
READ_LOCATION_FIELD(location);
READ_OID_FIELD(cycle_mark_type);
READ_INT_FIELD(cycle_mark_typmod);
READ_OID_FIELD(cycle_mark_collation);
READ_OID_FIELD(cycle_mark_neop);
READ_DONE();
}
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
/*
* _readCommonTableExpr
*/
static CommonTableExpr *
_readCommonTableExpr(void)
{
READ_LOCALS(CommonTableExpr);
READ_STRING_FIELD(ctename);
READ_NODE_FIELD(aliascolnames);
Allow user control of CTE materialization, and change the default behavior. Historically we've always materialized the full output of a CTE query, treating WITH as an optimization fence (so that, for example, restrictions from the outer query cannot be pushed into it). This is appropriate when the CTE query is INSERT/UPDATE/DELETE, or is recursive; but when the CTE query is non-recursive and side-effect-free, there's no hazard of changing the query results by pushing restrictions down. Another argument for materialization is that it can avoid duplicate computation of an expensive WITH query --- but that only applies if the WITH query is called more than once in the outer query. Even then it could still be a net loss, if each call has restrictions that would allow just a small part of the WITH query to be computed. Hence, let's change the behavior for WITH queries that are non-recursive and side-effect-free. By default, we will inline them into the outer query (removing the optimization fence) if they are called just once. If they are called more than once, we will keep the old behavior by default, but the user can override this and force inlining by specifying NOT MATERIALIZED. Lastly, the user can force the old behavior by specifying MATERIALIZED; this would mainly be useful when the query had deliberately been employing WITH as an optimization fence to prevent a poor choice of plan. Andreas Karlsson, Andrew Gierth, David Fetter Discussion: https://postgr.es/m/87sh48ffhb.fsf@news-spur.riddles.org.uk
2019-02-16 22:11:12 +01:00
READ_ENUM_FIELD(ctematerialized, CTEMaterialize);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_NODE_FIELD(ctequery);
SEARCH and CYCLE clauses This adds the SQL standard feature that adds the SEARCH and CYCLE clauses to recursive queries to be able to do produce breadth- or depth-first search orders and detect cycles. These clauses can be rewritten into queries using existing syntax, and that is what this patch does in the rewriter. Reviewed-by: Vik Fearing <vik@postgresfriends.org> Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/db80ceee-6f97-9b4a-8ee8-3ba0c58e5be2@2ndquadrant.com
2021-02-01 13:54:59 +01:00
READ_NODE_FIELD(search_clause);
READ_NODE_FIELD(cycle_clause);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_LOCATION_FIELD(location);
READ_BOOL_FIELD(cterecursive);
READ_INT_FIELD(cterefcount);
READ_NODE_FIELD(ctecolnames);
READ_NODE_FIELD(ctecoltypes);
READ_NODE_FIELD(ctecoltypmods);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
READ_NODE_FIELD(ctecolcollations);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_DONE();
}
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
/*
* _readMergeWhenClause
*/
static MergeWhenClause *
_readMergeWhenClause(void)
{
READ_LOCALS(MergeWhenClause);
READ_BOOL_FIELD(matched);
READ_ENUM_FIELD(commandType, CmdType);
Fix wrong field order in _readMergeWhenClause(). We hadn't noticed this because it's dead code: there is no situation where we read raw parse trees from text format. So maybe the right fix is to remove the function altogether, but I'll forbear for now; it's not the only dead code in readfuncs.c, I think. Noted while comparing existing code to the results of Peter's auto-generation script.
2022-07-06 23:26:27 +02:00
READ_ENUM_FIELD(override, OverridingKind);
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
READ_NODE_FIELD(condition);
READ_NODE_FIELD(targetList);
READ_NODE_FIELD(values);
READ_DONE();
}
/*
* _readMergeAction
*/
static MergeAction *
_readMergeAction(void)
{
READ_LOCALS(MergeAction);
READ_BOOL_FIELD(matched);
READ_ENUM_FIELD(commandType, CmdType);
READ_ENUM_FIELD(override, OverridingKind);
READ_NODE_FIELD(qual);
READ_NODE_FIELD(targetList);
READ_NODE_FIELD(updateColnos);
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readSetOperationStmt
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
*/
static SetOperationStmt *
Makeover for Unixware 7.1.1 * Makefile: Add more standard targets. Improve shell redirection in GNU make detection. * src/backend/access/transam/rmgr.c: Fix incorrect(?) C. * src/backend/libpq/pqcomm.c (StreamConnection): Work around accept() bug. * src/include/port/unixware.h: ...with help from here. * src/backend/nodes/print.c (plannode_type): Remove some "break"s after "return"s. * src/backend/tcop/dest.c (DestToFunction): ditto. * src/backend/nodes/readfuncs.c: Add proper prototypes. * src/backend/utils/adt/numutils.c (pg_atoi): Cope specially with strtol() setting EINVAL. This saves us from creating an extra set of regression test output for the affected systems. * src/include/storage/s_lock.h (tas): Correct prototype. * src/interfaces/libpq/fe-connect.c (parseServiceInfo): Don't use variable as dimension in array definition. * src/makefiles/Makefile.unixware: Add support for GCC. * src/template/unixware: same here * src/test/regress/expected/abstime-solaris-1947.out: Adjust whitespace. * src/test/regress/expected/horology-solaris-1947.out: Part of this file was evidently missing. * src/test/regress/pg_regress.sh: Fix shell. mkdir -p returns non-zero if the directory exists. * src/test/regress/resultmap: Add entries for Unixware.
2000-10-23 00:15:13 +02:00
_readSetOperationStmt(void)
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(SetOperationStmt);
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_ENUM_FIELD(op, SetOperation);
READ_BOOL_FIELD(all);
READ_NODE_FIELD(larg);
READ_NODE_FIELD(rarg);
Reimplement the linked list data structure used throughout the backend. In the past, we used a 'Lispy' linked list implementation: a "list" was merely a pointer to the head node of the list. The problem with that design is that it makes lappend() and length() linear time. This patch fixes that problem (and others) by maintaining a count of the list length and a pointer to the tail node along with each head node pointer. A "list" is now a pointer to a structure containing some meta-data about the list; the head and tail pointers in that structure refer to ListCell structures that maintain the actual linked list of nodes. The function names of the list API have also been changed to, I hope, be more logically consistent. By default, the old function names are still available; they will be disabled-by-default once the rest of the tree has been updated to use the new API names.
2004-05-26 06:41:50 +02:00
READ_NODE_FIELD(colTypes);
Fix UNION/INTERSECT/EXCEPT so that when two inputs being merged have same data type and same typmod, we show that typmod as the output typmod, rather than generic -1. This responds to several complaints over the past few years about UNIONs unexpectedly dropping length or precision info.
2006-08-10 04:36:29 +02:00
READ_NODE_FIELD(colTypmods);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
READ_NODE_FIELD(colCollations);
Teach the system how to use hashing for UNION. (INTERSECT/EXCEPT will follow, but seem like a separate patch since most of the remaining work is on the executor side.) I took the opportunity to push selection of the grouping operators for set operations into the parser where it belongs. Otherwise this is just a small exercise in making prepunion.c consider both alternatives. As with the recent DISTINCT patch, this means we can UNION on datatypes that can hash but not sort, and it means that UNION without ORDER BY is no longer certain to produce sorted output.
2008-08-07 03:11:52 +02:00
READ_NODE_FIELD(groupClauses);
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
}
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* Stuff from primnodes.h.
*/
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
static Alias *
_readAlias(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_LOCALS(Alias);
New cost model for planning, incorporating a penalty for random page accesses versus sequential accesses, a (very crude) estimate of the effects of caching on random page accesses, and cost to evaluate WHERE- clause expressions. Export critical parameters for this model as SET variables. Also, create SET variables for the planner's enable flags (enable_seqscan, enable_indexscan, etc) so that these can be controlled more conveniently than via PGOPTIONS. Planner now estimates both startup cost (cost before retrieving first tuple) and total cost of each path, so it can optimize queries with LIMIT on a reasonable basis by interpolating between these costs. Same facility is a win for EXISTS(...) subqueries and some other cases. Redesign pathkey representation to achieve a major speedup in planning (I saw as much as 5X on a 10-way join); also minor changes in planner to reduce memory consumption by recycling discarded Path nodes and not constructing unnecessary lists. Minor cleanups to display more-plausible costs in some cases in EXPLAIN output. Initdb forced by change in interface to index cost estimation functions.
2000-02-15 21:49:31 +01:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_STRING_FIELD(aliasname);
READ_NODE_FIELD(colnames);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_DONE();
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
static RangeVar *
_readRangeVar(void)
{
READ_LOCALS(RangeVar);
local_node->catalogname = NULL; /* not currently saved in output format */
READ_STRING_FIELD(schemaname);
READ_STRING_FIELD(relname);
Replace enum InhOption with simple boolean. Now that it has only INH_NO and INH_YES values, it's just weird that it's not a plain bool, so make it that way. Also rename RangeVar.inhOpt to "inh", to be like RangeTblEntry.inh. My recollection is that we gave it a different name specifically because it had a different representation than the derived bool value, but it no longer does. And this is a good forcing function to be sure we catch any places that are affected by the change. Bump catversion because of possible effect on stored RangeVar nodes. I'm not exactly convinced that we ever store RangeVar on disk, but we have a readfuncs function for it, so be cautious. (If we do do so, then commit e13486eba was in error not to bump catversion.) Follow-on to commit e13486eba. Discussion: http://postgr.es/m/CA+TgmoYe+EG7LdYX6pkcNxr4ygkP4+A=jm9o-CPXyOvRiCNwaQ@mail.gmail.com
2016-12-23 19:35:11 +01:00
READ_BOOL_FIELD(inh);
Generalize concept of temporary relations to "relation persistence". This commit replaces pg_class.relistemp with pg_class.relpersistence; and also modifies the RangeVar node type to carry relpersistence rather than istemp. It also removes removes rd_istemp from RelationData and instead performs the correct computation based on relpersistence. For clarity, we add three new macros: RelationNeedsWAL(), RelationUsesLocalBuffers(), and RelationUsesTempNamespace(), so that we can clarify the purpose of each check that previous depended on rd_istemp. This is intended as infrastructure for the upcoming unlogged tables patch, as well as for future possible work on global temporary tables.
2010-12-13 18:34:26 +01:00
READ_CHAR_FIELD(relpersistence);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(alias);
Add a bunch of new error location reports to parse-analysis error messages. There are still some weak spots around JOIN USING and relation alias lists, but most errors reported within backend/parser/ now have locations.
2008-09-01 22:42:46 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/*
* _readTableFunc
*/
static TableFunc *
_readTableFunc(void)
{
READ_LOCALS(TableFunc);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_ENUM_FIELD(functype, TableFuncType);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
READ_NODE_FIELD(ns_uris);
Reconcile nodes/*funcs.c with PostgreSQL 10 work. The _equalTableFunc() omission of coltypmods has semantic significance, but I did not track down resulting user-visible bugs, if any. The other changes are cosmetic only, affecting order. catversion bump due to readfuncs.c field order change.
2017-06-16 09:16:11 +02:00
READ_NODE_FIELD(ns_names);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
READ_NODE_FIELD(docexpr);
READ_NODE_FIELD(rowexpr);
READ_NODE_FIELD(colnames);
READ_NODE_FIELD(coltypes);
READ_NODE_FIELD(coltypmods);
READ_NODE_FIELD(colcollations);
READ_NODE_FIELD(colexprs);
READ_NODE_FIELD(coldefexprs);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_NODE_FIELD(colvalexprs);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
READ_BITMAPSET_FIELD(notnulls);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_NODE_FIELD(plan);
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
READ_INT_FIELD(ordinalitycol);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Remove the Query structure from the executor's API. This allows us to stop storing mostly-redundant Query trees in prepared statements, portals, etc. To replace Query, a new node type called PlannedStmt is inserted by the planner at the top of a completed plan tree; this carries just the fields of Query that are still needed at runtime. The statement lists kept in portals etc. now consist of intermixed PlannedStmt and bare utility-statement nodes --- no Query. This incidentally allows us to remove some fields from Query and Plan nodes that shouldn't have been there in the first place. Still to do: simplify the execution-time range table; at the moment the range table passed to the executor still contains Query trees for subqueries. initdb forced due to change of stored rules.
2007-02-20 18:32:18 +01:00
static IntoClause *
_readIntoClause(void)
{
READ_LOCALS(IntoClause);
READ_NODE_FIELD(rel);
READ_NODE_FIELD(colNames);
Fix copy/out/readfuncs for accessMethod addition in 8586bf7ed8. This includes a catversion bump, as IntoClause is theoretically speaking part of storable rules. In practice I don't think that can happen, but there's no reason to be stingy here. Per buildfarm member calliphoridae.
2019-03-06 20:55:28 +01:00
READ_STRING_FIELD(accessMethod);
Remove the Query structure from the executor's API. This allows us to stop storing mostly-redundant Query trees in prepared statements, portals, etc. To replace Query, a new node type called PlannedStmt is inserted by the planner at the top of a completed plan tree; this carries just the fields of Query that are still needed at runtime. The statement lists kept in portals etc. now consist of intermixed PlannedStmt and bare utility-statement nodes --- no Query. This incidentally allows us to remove some fields from Query and Plan nodes that shouldn't have been there in the first place. Still to do: simplify the execution-time range table; at the moment the range table passed to the executor still contains Query trees for subqueries. initdb forced due to change of stored rules.
2007-02-20 18:32:18 +01:00
READ_NODE_FIELD(options);
READ_ENUM_FIELD(onCommit, OnCommitAction);
READ_STRING_FIELD(tableSpaceName);
Clean up the mess around EXPLAIN and materialized views. Revert the matview-related changes in explain.c's API, as per recent complaint from Robert Haas. The reason for these appears to have been principally some ill-considered choices around having intorel_startup do what ought to be parse-time checking, plus a poor arrangement for passing it the view parsetree it needs to store into pg_rewrite when creating a materialized view. Do the latter by having parse analysis stick a copy into the IntoClause, instead of doing it at runtime. (On the whole, I seriously question the choice to represent CREATE MATERIALIZED VIEW as a variant of SELECT INTO/CREATE TABLE AS, because that means injecting even more complexity into what was already a horrid legacy kluge. However, I didn't go so far as to rethink that choice ... yet.) I also moved several error checks into matview parse analysis, and made the check for external Params in a matview more accurate. In passing, clean things up a bit more around interpretOidsOption(), and fix things so that we can use that to force no-oids for views, sequences, etc, thereby eliminating the need to cons up "oids = false" options when creating them. catversion bump due to change in IntoClause. (I wonder though if we really need readfuncs/outfuncs support for IntoClause anymore.)
2013-04-13 01:25:20 +02:00
READ_NODE_FIELD(viewQuery);
Fix unsupported options in CREATE TABLE ... AS EXECUTE. The WITH [NO] DATA option was not supported, nor the ability to specify replacement column names; the former limitation wasn't even documented, as per recent complaint from Naoya Anzai. Fix by moving the responsibility for supporting these options into the executor. It actually takes less code this way ... catversion bump due to change in representation of IntoClause, which might affect stored rules.
2011-11-25 05:21:06 +01:00
READ_BOOL_FIELD(skipData);
Remove the Query structure from the executor's API. This allows us to stop storing mostly-redundant Query trees in prepared statements, portals, etc. To replace Query, a new node type called PlannedStmt is inserted by the planner at the top of a completed plan tree; this carries just the fields of Query that are still needed at runtime. The statement lists kept in portals etc. now consist of intermixed PlannedStmt and bare utility-statement nodes --- no Query. This incidentally allows us to remove some fields from Query and Plan nodes that shouldn't have been there in the first place. Still to do: simplify the execution-time range table; at the moment the range table passed to the executor still contains Query trees for subqueries. initdb forced due to change of stored rules.
2007-02-20 18:32:18 +01:00
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readVar
*/
static Var *
_readVar(void)
{
READ_LOCALS(Var);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Remove arbitrary 64K-or-so limit on rangetable size. Up to now the size of a query's rangetable has been limited by the constants INNER_VAR et al, which mustn't be equal to any real rangetable index. 65000 doubtless seemed like enough for anybody, and it still is orders of magnitude larger than the number of joins we can realistically handle. However, we need a rangetable entry for each child partition that is (or might be) processed by a query. Queries with a few thousand partitions are getting more realistic, so that the day when that limit becomes a problem is in sight, even if it's not here yet. Hence, let's raise the limit. Rather than just increase the values of INNER_VAR et al, this patch adopts the approach of making them small negative values, so that rangetables could theoretically become as long as INT_MAX. The bulk of the patch is concerned with changing Var.varno and some related variables from "Index" (unsigned int) to plain "int". This is basically cosmetic, with little actual effect other than to help debuggers print their values nicely. As such, I've only bothered with changing places that could actually see INNER_VAR et al, which the parser and most of the planner don't. We do have to be careful in places that are performing less/greater comparisons on varnos, but there are very few such places, other than the IS_SPECIAL_VARNO macro itself. A notable side effect of this patch is that while it used to be possible to add INNER_VAR et al to a Bitmapset, that will now draw an error. I don't see any likelihood that it wouldn't be a bug to include these fake varnos in a bitmapset of real varnos, so I think this is all to the good. Although this touches outfuncs/readfuncs, I don't think a catversion bump is required, since stored rules would never contain Vars with these fake varnos. Andrey Lepikhov and Tom Lane, after a suggestion by Peter Eisentraut Discussion: https://postgr.es/m/43c7f2f5-1e27-27aa-8c65-c91859d15190@postgrespro.ru
2021-09-15 20:11:21 +02:00
READ_INT_FIELD(varno);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_INT_FIELD(varattno);
READ_OID_FIELD(vartype);
READ_INT_FIELD(vartypmod);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
READ_OID_FIELD(varcollid);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_UINT_FIELD(varlevelsup);
Reconsider the representation of join alias Vars. The core idea of this patch is to make the parser generate join alias Vars (that is, ones with varno pointing to a JOIN RTE) only when the alias Var is actually different from any raw join input, that is a type coercion and/or COALESCE is necessary to generate the join output value. Otherwise just generate varno/varattno pointing to the relevant join input column. In effect, this means that the planner's flatten_join_alias_vars() transformation is already done in the parser, for all cases except (a) columns that are merged by JOIN USING and are transformed in the process, and (b) whole-row join Vars. In principle that would allow us to skip doing flatten_join_alias_vars() in many more queries than we do now, but we don't have quite enough infrastructure to know that we can do so --- in particular there's no cheap way to know whether there are any whole-row join Vars. I'm not sure if it's worth the trouble to add a Query-level flag for that, and in any case it seems like fit material for a separate patch. But even without skipping the work entirely, this should make flatten_join_alias_vars() faster, particularly where there are nested joins that it previously had to flatten recursively. An essential part of this change is to replace Var nodes' varnoold/varoattno fields with varnosyn/varattnosyn, which have considerably more tightly-defined meanings than the old fields: when they differ from varno/varattno, they identify the Var's position in an aliased JOIN RTE, and the join alias is what ruleutils.c should print for the Var. This is necessary because the varno change destroyed ruleutils.c's ability to find the JOIN RTE from the Var's varno. Another way in which this change broke ruleutils.c is that it's no longer feasible to determine, from a JOIN RTE's joinaliasvars list, which join columns correspond to which columns of the join's immediate input relations. (If those are sub-joins, the joinaliasvars entries may point to columns of their base relations, not the sub-joins.) But that was a horrid mess requiring a lot of fragile assumptions already, so let's just bite the bullet and add some more JOIN RTE fields to make it more straightforward to figure that out. I added two integer-List fields containing the relevant column numbers from the left and right input rels, plus a count of how many merged columns there are. This patch depends on the ParseNamespaceColumn infrastructure that I added in commit 5815696bc. The biggest bit of code change is restructuring transformFromClauseItem's handling of JOINs so that the ParseNamespaceColumn data is propagated upward correctly. Other than that and the ruleutils fixes, everything pretty much just works, though some processing is now inessential. I grabbed two pieces of low-hanging fruit in that line: 1. In find_expr_references, we don't need to recurse into join alias Vars anymore. There aren't any except for references to merged USING columns, which are more properly handled when we scan the join's RTE. This change actually fixes an edge-case issue: we will now record a dependency on any type-coercion function present in a USING column's joinaliasvar, even if that join column has no references in the query text. The odds of the missing dependency causing a problem seem quite small: you'd have to posit somebody dropping an implicit cast between two data types, without removing the types themselves, and then having a stored rule containing a whole-row Var for a join whose USING merge depends on that cast. So I don't feel a great need to change this in the back branches. But in theory this way is more correct. 2. markRTEForSelectPriv and markTargetListOrigin don't need to recurse into join alias Vars either, because the cases they care about don't apply to alias Vars for USING columns that are semantically distinct from the underlying columns. This removes the only case in which markVarForSelectPriv could be called with NULL for the RTE, so adjust the comments to describe that hack as being strictly internal to markRTEForSelectPriv. catversion bump required due to changes in stored rules. Discussion: https://postgr.es/m/7115.1577986646@sss.pgh.pa.us
2020-01-09 17:56:59 +01:00
READ_UINT_FIELD(varnosyn);
READ_INT_FIELD(varattnosyn);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Restructure handling of inheritance queries so that they work with outer joins, and clean things up a good deal at the same time. Append plan node no longer hacks on rangetable at runtime --- instead, all child tables are given their own RT entries during planning. Concept of multiple target tables pushed up into execMain, replacing bug-prone implementation within nodeAppend. Planner now supports generating Append plans for inheritance sets either at the top of the plan (the old way) or at the bottom. Expanding at the bottom is appropriate for tables used as sources, since they may appear inside an outer join; but we must still expand at the top when the target of an UPDATE or DELETE is an inheritance set, because we actually need a different targetlist and junkfilter for each target table in that case. Fortunately a target table can't be inside an outer join... Bizarre mutual recursion between union_planner and prepunion.c is gone --- in fact, union_planner doesn't really have much to do with union queries anymore, so I renamed it grouping_planner.
2000-11-12 01:37:02 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readConst
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static Const *
_readConst(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(Const);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(consttype);
Fix up the remaining places where the expression node structure would lose available information about the typmod of an expression; namely, Const, ArrayRef, ArrayExpr, and EXPR and ARRAY SubLinks. In the ArrayExpr and SubLink cases it wasn't really the data structure's fault, but exprTypmod() being lazy. This seems like a good idea in view of the expected increase in typmod usage from Teodor's work to allow user-defined types to have typmods. In particular this responds to the concerns we had about eliminating the special-purpose hack that exprTypmod() used to have for BPCHAR Consts. We can now tell whether or not such a Const has been cast to a specific length, and report or display properly if so. initdb forced due to changes in stored rules.
2007-03-17 01:11:05 +01:00
READ_INT_FIELD(consttypmod);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
READ_OID_FIELD(constcollid);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_INT_FIELD(constlen);
READ_BOOL_FIELD(constbyval);
READ_BOOL_FIELD(constisnull);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
token = pg_strtok(&length); /* skip :constvalue */
if (local_node->constisnull)
token = pg_strtok(&length); /* skip "<>" */
else
local_node->constvalue = readDatum(local_node->constbyval);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
* _readParam
*/
static Param *
_readParam(void)
{
READ_LOCALS(Param);
Simplify ParamListInfo data structure to support only numbered parameters, not named ones, and replace linear searches of the list with array indexing. The named-parameter support has been dead code for many years anyway, and recent profiling suggests that the searching was costing a noticeable amount of performance for complex queries.
2006-04-22 03:26:01 +02:00
READ_ENUM_FIELD(paramkind, ParamKind);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_INT_FIELD(paramid);
READ_OID_FIELD(paramtype);
Add a paramtypmod field to Param nodes. This is dead weight for Params representing externally-supplied values, since the APIs that carry such values only specify type not typmod. However, for PARAM_SUBLINK Params it is handy to carry the typmod of the sublink's output column. This is a much cleaner solution for the recently reported 'could not find pathkey item to sort' and 'failed to find unique expression in subplan tlist' bugs than my original 8.2-compatible patch. Besides, someday we might want to support typmods for external parameters ...
2006-12-10 23:13:27 +01:00
READ_INT_FIELD(paramtypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(paramcollid);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_DONE();
}
/*
* _readAggref
*/
static Aggref *
_readAggref(void)
{
READ_LOCALS(Aggref);
READ_OID_FIELD(aggfnoid);
READ_OID_FIELD(aggtype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(aggcollid);
READ_OID_FIELD(inputcollid);
Fix handling of argument and result datatypes for partial aggregation. When doing partial aggregation, the args list of the upper (combining) Aggref node is replaced by a Var representing the output of the partial aggregation steps, which has either the aggregate's transition data type or a serialized representation of that. However, nodeAgg.c blindly continued to use the args list as an indication of the user-level argument types. This broke resolution of polymorphic transition datatypes at executor startup (though it accidentally failed to fail for the ANYARRAY case, which is likely the only one anyone had tested). Moreover, the constructed FuncExpr passed to the finalfunc contained completely wrong information, which would have led to bogus answers or crashes for any case where the finalfunc examined that information (which is only likely to be with polymorphic aggregates using a non-polymorphic transition type). As an independent bug, apply_partialaggref_adjustment neglected to resolve a polymorphic transition datatype before assigning it as the output type of the lower-level Aggref node. This again accidentally failed to fail for ANYARRAY but would be unlikely to work in other cases. To fix the first problem, record the user-level argument types in a separate OID-list field of Aggref, and look to that rather than the args list when asking what the argument types were. (It turns out to be convenient to include any "direct" arguments in this list too, although those are not currently subject to being overwritten.) Rather than adding yet another resolve_aggregate_transtype() call to fix the second problem, add an aggtranstype field to Aggref, and store the resolved transition type OID there when the planner first computes it. (By doing this in the planner and not the parser, we can allow the aggregate's transition type to change from time to time, although no DDL support yet exists for that.) This saves nothing of consequence for simple non-polymorphic aggregates, but for polymorphic transition types we save a catalog lookup during executor startup as well as several planner lookups that are new in 9.6 due to parallel query planning. In passing, fix an error that was introduced into count_agg_clauses_walker some time ago: it was applying exprTypmod() to something that wasn't an expression node at all, but a TargetEntry. exprTypmod silently returned -1 so that there was not an obvious failure, but this broke the intended sensitivity of aggregate space consumption estimates to the typmod of varchar and similar data types. This part needs to be back-patched. Catversion bump due to change of stored Aggref nodes. Discussion: <8229.1466109074@sss.pgh.pa.us>
2016-06-18 03:44:37 +02:00
READ_OID_FIELD(aggtranstype);
READ_NODE_FIELD(aggargtypes);
Support ordered-set (WITHIN GROUP) aggregates. This patch introduces generic support for ordered-set and hypothetical-set aggregate functions, as well as implementations of the instances defined in SQL:2008 (percentile_cont(), percentile_disc(), rank(), dense_rank(), percent_rank(), cume_dist()). We also added mode() though it is not in the spec, as well as versions of percentile_cont() and percentile_disc() that can compute multiple percentile values in one pass over the data. Unlike the original submission, this patch puts full control of the sorting process in the hands of the aggregate's support functions. To allow the support functions to find out how they're supposed to sort, a new API function AggGetAggref() is added to nodeAgg.c. This allows retrieval of the aggregate call's Aggref node, which may have other uses beyond the immediate need. There is also support for ordered-set aggregates to install cleanup callback functions, so that they can be sure that infrastructure such as tuplesort objects gets cleaned up. In passing, make some fixes in the recently-added support for variadic aggregates, and make some editorial adjustments in the recent FILTER additions for aggregates. Also, simplify use of IsBinaryCoercible() by allowing it to succeed whenever the target type is ANY or ANYELEMENT. It was inconsistent that it dealt with other polymorphic target types but not these. Atri Sharma and Andrew Gierth; reviewed by Pavel Stehule and Vik Fearing, and rather heavily editorialized upon by Tom Lane
2013-12-23 22:11:35 +01:00
READ_NODE_FIELD(aggdirectargs);
Aggregate functions now support multiple input arguments. I also took the opportunity to treat COUNT(*) as a zero-argument aggregate instead of the old hack that equated it to COUNT(1); this is materially cleaner (no more weird ANYOID cases) and ought to be at least a tiny bit faster. Original patch by Sergey Koposov; review, documentation, simple regression tests, pg_dump and psql support by moi.
2006-07-27 21:52:07 +02:00
READ_NODE_FIELD(args);
Support ORDER BY within aggregate function calls, at long last providing a non-kluge method for controlling the order in which values are fed to an aggregate function. At the same time eliminate the old implementation restriction that DISTINCT was only supported for single-argument aggregates. Possibly release-notable behavioral change: formerly, agg(DISTINCT x) dropped null values of x unconditionally. Now, it does so only if the agg transition function is strict; otherwise nulls are treated as DISTINCT normally would, ie, you get one copy. Andrew Gierth, reviewed by Hitoshi Harada
2009-12-15 18:57:48 +01:00
READ_NODE_FIELD(aggorder);
READ_NODE_FIELD(aggdistinct);
Implement the FILTER clause for aggregate function calls. This is SQL-standard with a few extensions, namely support for subqueries and outer references in clause expressions. catversion bump due to change in Aggref and WindowFunc. David Fetter, reviewed by Dean Rasheed.
2013-07-17 02:15:36 +02:00
READ_NODE_FIELD(aggfilter);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_BOOL_FIELD(aggstar);
Allow aggregate functions to be VARIADIC. There's no inherent reason why an aggregate function can't be variadic (even VARIADIC ANY) if its transition function can handle the case. Indeed, this patch to add the feature touches none of the planner or executor, and little of the parser; the main missing stuff was DDL and pg_dump support. It is true that variadic aggregates can create the same sort of ambiguity about parameters versus ORDER BY keys that was complained of when we (briefly) had both one- and two-argument forms of string_agg(). However, the policy formed in response to that discussion only said that we'd not create any built-in aggregates with varying numbers of arguments, not that we shouldn't allow users to do it. So the logical extension of that is we can allow users to make variadic aggregates as long as we're wary about shipping any such in core. In passing, this patch allows aggregate function arguments to be named, to the extent of remembering the names in pg_proc and dumping them in pg_dump. You can't yet call an aggregate using named-parameter notation. That seems like a likely future extension, but it'll take some work, and it's not what this patch is really about. Likewise, there's still some work needed to make window functions handle VARIADIC fully, but I left that for another day. initdb forced because of new aggvariadic field in Aggref parse nodes.
2013-09-03 23:08:38 +02:00
READ_BOOL_FIELD(aggvariadic);
Support ordered-set (WITHIN GROUP) aggregates. This patch introduces generic support for ordered-set and hypothetical-set aggregate functions, as well as implementations of the instances defined in SQL:2008 (percentile_cont(), percentile_disc(), rank(), dense_rank(), percent_rank(), cume_dist()). We also added mode() though it is not in the spec, as well as versions of percentile_cont() and percentile_disc() that can compute multiple percentile values in one pass over the data. Unlike the original submission, this patch puts full control of the sorting process in the hands of the aggregate's support functions. To allow the support functions to find out how they're supposed to sort, a new API function AggGetAggref() is added to nodeAgg.c. This allows retrieval of the aggregate call's Aggref node, which may have other uses beyond the immediate need. There is also support for ordered-set aggregates to install cleanup callback functions, so that they can be sure that infrastructure such as tuplesort objects gets cleaned up. In passing, make some fixes in the recently-added support for variadic aggregates, and make some editorial adjustments in the recent FILTER additions for aggregates. Also, simplify use of IsBinaryCoercible() by allowing it to succeed whenever the target type is ANY or ANYELEMENT. It was inconsistent that it dealt with other polymorphic target types but not these. Atri Sharma and Andrew Gierth; reviewed by Pavel Stehule and Vik Fearing, and rather heavily editorialized upon by Tom Lane
2013-12-23 22:11:35 +01:00
READ_CHAR_FIELD(aggkind);
Support ORDER BY within aggregate function calls, at long last providing a non-kluge method for controlling the order in which values are fed to an aggregate function. At the same time eliminate the old implementation restriction that DISTINCT was only supported for single-argument aggregates. Possibly release-notable behavioral change: formerly, agg(DISTINCT x) dropped null values of x unconditionally. Now, it does so only if the agg transition function is strict; otherwise nulls are treated as DISTINCT normally would, ie, you get one copy. Andrew Gierth, reviewed by Hitoshi Harada
2009-12-15 18:57:48 +01:00
READ_UINT_FIELD(agglevelsup);
Rethink node-level representation of partial-aggregation modes. The original coding had three separate booleans representing partial aggregation behavior, which was confusing, unreadable, and error-prone, not least because the booleans weren't always listed in the same order. It was also inadequate for the allegedly-desirable future extension to support intermediate partial aggregation, because we'd need separate markers for serialization and deserialization in such a case. Merge these bools into an enum "AggSplit" to provide symbolic names for the supported operating modes (and document what those are). By assigning the values of the enum constants carefully, we can treat AggSplit values as options bitmasks so that tests of what to do aren't noticeably more expensive than before. While at it, get rid of Aggref.aggoutputtype. That's not needed since commit 59a3795c2 got rid of setrefs.c's special-purpose Aggref comparison code, and it likewise seemed more confusing than helpful. Assorted comment cleanup as well (there's still more that I want to do in that line). catversion bump for change in Aggref node contents. Should be the last one for partial-aggregation changes. Discussion: <29309.1466699160@sss.pgh.pa.us>
2016-06-26 20:33:38 +02:00
READ_ENUM_FIELD(aggsplit, AggSplit);
Move per-agg and per-trans duplicate finding to the planner. This has the advantage that the cost estimates for aggregates can count the number of calls to transition and final functions correctly. Bump catalog version, because views can contain Aggrefs. Reviewed-by: Andres Freund Discussion: https://www.postgresql.org/message-id/b2e3536b-1dbc-8303-c97e-89cb0b4a9a48%40iki.fi
2020-11-24 09:45:00 +01:00
READ_INT_FIELD(aggno);
READ_INT_FIELD(aggtransno);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_DONE();
}
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
/*
* _readGroupingFunc
*/
static GroupingFunc *
_readGroupingFunc(void)
{
READ_LOCALS(GroupingFunc);
READ_NODE_FIELD(args);
READ_NODE_FIELD(refs);
READ_NODE_FIELD(cols);
Reconcile nodes/*funcs.c with recent work. A few of the discrepancies had semantic significance, but I did not track down the resulting user-visible bugs, if any. Back-patch to 9.5, where all but one discrepancy appeared. The _equalCreateEventTrigStmt() situation dates to 9.3 but does not affect semantics. catversion bump due to readfuncs.c field order changes.
2015-08-06 02:44:27 +02:00
READ_UINT_FIELD(agglevelsup);
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
READ_LOCATION_FIELD(location);
READ_DONE();
}
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
/*
* _readWindowFunc
*/
static WindowFunc *
_readWindowFunc(void)
{
READ_LOCALS(WindowFunc);
READ_OID_FIELD(winfnoid);
READ_OID_FIELD(wintype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(wincollid);
READ_OID_FIELD(inputcollid);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
READ_NODE_FIELD(args);
Implement the FILTER clause for aggregate function calls. This is SQL-standard with a few extensions, namely support for subqueries and outer references in clause expressions. catversion bump due to change in Aggref and WindowFunc. David Fetter, reviewed by Dean Rasheed.
2013-07-17 02:15:36 +02:00
READ_NODE_FIELD(aggfilter);
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
READ_UINT_FIELD(winref);
READ_BOOL_FIELD(winstar);
READ_BOOL_FIELD(winagg);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/*
Renaming for new subscripting mechanism Over at patch https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more generic subscription mechanism, which allows subscripting not only arrays but also other object types such as JSONB. That functionality is introduced in a largish invasive patch, out of which this internal renaming patch was extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov Discussion: https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
2019-02-01 16:50:32 +01:00
* _readSubscriptingRef
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Renaming for new subscripting mechanism Over at patch https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more generic subscription mechanism, which allows subscripting not only arrays but also other object types such as JSONB. That functionality is introduced in a largish invasive patch, out of which this internal renaming patch was extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov Discussion: https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
2019-02-01 16:50:32 +01:00
static SubscriptingRef *
_readSubscriptingRef(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Renaming for new subscripting mechanism Over at patch https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more generic subscription mechanism, which allows subscripting not only arrays but also other object types such as JSONB. That functionality is introduced in a largish invasive patch, out of which this internal renaming patch was extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov Discussion: https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
2019-02-01 16:50:32 +01:00
READ_LOCALS(SubscriptingRef);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
Renaming for new subscripting mechanism Over at patch https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more generic subscription mechanism, which allows subscripting not only arrays but also other object types such as JSONB. That functionality is introduced in a largish invasive patch, out of which this internal renaming patch was extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov Discussion: https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
2019-02-01 16:50:32 +01:00
READ_OID_FIELD(refcontainertype);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
READ_OID_FIELD(refelemtype);
Support subscripting of arbitrary types, not only arrays. This patch generalizes the subscripting infrastructure so that any data type can be subscripted, if it provides a handler function to define what that means. Traditional variable-length (varlena) arrays all use array_subscript_handler(), while the existing fixed-length types that support subscripting use raw_array_subscript_handler(). It's expected that other types that want to use subscripting notation will define their own handlers. (This patch provides no such new features, though; it only lays the foundation for them.) To do this, move the parser's semantic processing of subscripts (including coercion to whatever data type is required) into a method callback supplied by the handler. On the execution side, replace the ExecEvalSubscriptingRef* layer of functions with direct calls to callback-supplied execution routines. (Thus, essentially no new run-time overhead should be caused by this patch. Indeed, there is room to remove some overhead by supplying specialized execution routines. This patch does a little bit in that line, but more could be done.) Additional work is required here and there to remove formerly hard-wired assumptions about the result type, collation, etc of a SubscriptingRef expression node; and to remove assumptions that the subscript values must be integers. One useful side-effect of this is that we now have a less squishy mechanism for identifying whether a data type is a "true" array: instead of wiring in weird rules about typlen, we can look to see if pg_type.typsubscript == F_ARRAY_SUBSCRIPT_HANDLER. For this to be bulletproof, we have to forbid user-defined types from using that handler directly; but there seems no good reason for them to do so. This patch also removes assumptions that the number of subscripts is limited to MAXDIM (6), or indeed has any hard-wired limit. That limit still applies to types handled by array_subscript_handler or raw_array_subscript_handler, but to discourage other dependencies on this constant, I've moved it from c.h to utils/array.h. Dmitry Dolgov, reviewed at various times by Tom Lane, Arthur Zakirov, Peter Eisentraut, Pavel Stehule Discussion: https://postgr.es/m/CA+q6zcVDuGBv=M0FqBYX8DPebS3F_0KQ6OVFobGJPM507_SZ_w@mail.gmail.com Discussion: https://postgr.es/m/CA+q6zcVovR+XY4mfk-7oNk-rF91gH0PebnNfuUjuuDsyHjOcVA@mail.gmail.com
2020-12-09 18:40:37 +01:00
READ_OID_FIELD(refrestype);
Fix up the remaining places where the expression node structure would lose available information about the typmod of an expression; namely, Const, ArrayRef, ArrayExpr, and EXPR and ARRAY SubLinks. In the ArrayExpr and SubLink cases it wasn't really the data structure's fault, but exprTypmod() being lazy. This seems like a good idea in view of the expected increase in typmod usage from Teodor's work to allow user-defined types to have typmods. In particular this responds to the concerns we had about eliminating the special-purpose hack that exprTypmod() used to have for BPCHAR Consts. We can now tell whether or not such a Const has been cast to a specific length, and report or display properly if so. initdb forced due to changes in stored rules.
2007-03-17 01:11:05 +01:00
READ_INT_FIELD(reftypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(refcollid);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(refupperindexpr);
READ_NODE_FIELD(reflowerindexpr);
READ_NODE_FIELD(refexpr);
READ_NODE_FIELD(refassgnexpr);
READ_DONE();
}
/*
* _readFuncExpr
*/
static FuncExpr *
_readFuncExpr(void)
{
READ_LOCALS(FuncExpr);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(funcid);
READ_OID_FIELD(funcresulttype);
READ_BOOL_FIELD(funcretset);
Add infrastructure for storing a VARIADIC ANY function's VARIADIC flag. Originally we didn't bother to mark FuncExprs with any indication whether VARIADIC had been given in the source text, because there didn't seem to be any need for it at runtime. However, because we cannot fold a VARIADIC ANY function's arguments into an array (since they're not necessarily all the same type), we do actually need that information at runtime if VARIADIC ANY functions are to respond unsurprisingly to use of the VARIADIC keyword. Add the missing field, and also fix ruleutils.c so that VARIADIC ANY function calls are dumped properly. Extracted from a larger patch that also fixes concat() and format() (the only two extant VARIADIC ANY functions) to behave properly when VARIADIC is specified. This portion seems appropriate to review and commit separately. Pavel Stehule
2013-01-22 02:25:26 +01:00
READ_BOOL_FIELD(funcvariadic);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_ENUM_FIELD(funcformat, CoercionForm);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(funccollid);
READ_OID_FIELD(inputcollid);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support use of function argument names to identify which actual arguments match which function parameters. The syntax uses AS, for example funcname(value AS arg1, anothervalue AS arg2) Pavel Stehule
2009-10-08 04:39:25 +02:00
/*
* _readNamedArgExpr
*/
static NamedArgExpr *
_readNamedArgExpr(void)
{
READ_LOCALS(NamedArgExpr);
READ_NODE_FIELD(arg);
READ_STRING_FIELD(name);
READ_INT_FIELD(argnumber);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
* _readOpExpr
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
static OpExpr *
_readOpExpr(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_LOCALS(OpExpr);
First cut at full support for OUTER JOINs. There are still a few loose ends to clean up (see my message of same date to pghackers), but mostly it works. INITDB REQUIRED!
2000-09-12 23:07:18 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(opno);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_OID_FIELD(opfuncid);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(opresulttype);
READ_BOOL_FIELD(opretset);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(opcollid);
READ_OID_FIELD(inputcollid);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
First cut at full support for OUTER JOINs. There are still a few loose ends to clean up (see my message of same date to pghackers), but mostly it works. INITDB REQUIRED!
2000-09-12 23:07:18 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
* _readDistinctExpr
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
static DistinctExpr *
_readDistinctExpr(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_LOCALS(DistinctExpr);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_OID_FIELD(opno);
READ_OID_FIELD(opfuncid);
READ_OID_FIELD(opresulttype);
READ_BOOL_FIELD(opretset);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(opcollid);
READ_OID_FIELD(inputcollid);
READ_NODE_FIELD(args);
READ_LOCATION_FIELD(location);
READ_DONE();
}
/*
* _readNullIfExpr
*/
static NullIfExpr *
_readNullIfExpr(void)
{
READ_LOCALS(NullIfExpr);
READ_OID_FIELD(opno);
READ_OID_FIELD(opfuncid);
READ_OID_FIELD(opresulttype);
READ_BOOL_FIELD(opretset);
READ_OID_FIELD(opcollid);
READ_OID_FIELD(inputcollid);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
2003-06-29 02:33:44 +02:00
/*
* _readScalarArrayOpExpr
*/
static ScalarArrayOpExpr *
_readScalarArrayOpExpr(void)
{
READ_LOCALS(ScalarArrayOpExpr);
READ_OID_FIELD(opno);
READ_OID_FIELD(opfuncid);
Speedup ScalarArrayOpExpr evaluation ScalarArrayOpExprs with "useOr=true" and a set of Consts on the righthand side have traditionally been evaluated by using a linear search over the array. When these arrays contain large numbers of elements then this linear search could become a significant part of execution time. Here we add a new method of evaluating ScalarArrayOpExpr expressions to allow them to be evaluated by first building a hash table containing each element, then on subsequent evaluations, we just probe that hash table to determine if there is a match. The planner is in charge of determining when this optimization is possible and it enables it by setting hashfuncid in the ScalarArrayOpExpr. The executor will only perform the hash table evaluation when the hashfuncid is set. This means that not all cases are optimized. For example CHECK constraints containing an IN clause won't go through the planner, so won't get the hashfuncid set. We could maybe do something about that at some later date. The reason we're not doing it now is from fear that we may slow down cases where the expression is evaluated only once. Those cases can be common, for example, a single row INSERT to a table with a CHECK constraint containing an IN clause. In the planner, we enable this when there are suitable hash functions for the ScalarArrayOpExpr's operator and only when there is at least MIN_ARRAY_SIZE_FOR_HASHED_SAOP elements in the array. The threshold is currently set to 9. Author: James Coleman, David Rowley Reviewed-by: David Rowley, Tomas Vondra, Heikki Linnakangas Discussion: https://postgr.es/m/CAAaqYe8x62+=wn0zvNKCj55tPpg-JBHzhZFFc6ANovdqFw7-dA@mail.gmail.com
2021-04-08 13:51:22 +02:00
READ_OID_FIELD(hashfuncid);
Use a hash table to speed up NOT IN(values) Similar to 50e17ad28, which allowed hash tables to be used for IN clauses with a set of constants, here we add the same feature for NOT IN clauses. NOT IN evaluates the same as: WHERE a <> v1 AND a <> v2 AND a <> v3. Obviously, if we're using a hash table we must be exactly equivalent to that and return the same result taking into account that either side of the condition could contain a NULL. This requires a little bit of special handling to make work with the hash table version. When processing NOT IN, the ScalarArrayOpExpr's operator will be the <> operator. To be able to build and lookup a hash table we must use the <>'s negator operator. The planner checks if that exists and is hashable and sets the relevant fields in ScalarArrayOpExpr to instruct the executor to use hashing. Author: David Rowley, James Coleman Reviewed-by: James Coleman, Zhihong Yu Discussion: https://postgr.es/m/CAApHDvoF1mum_FRk6D621edcB6KSHBi2+GAgWmioj5AhOu2vwQ@mail.gmail.com
2021-07-07 06:29:17 +02:00
READ_OID_FIELD(negfuncid);
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
2003-06-29 02:33:44 +02:00
READ_BOOL_FIELD(useOr);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(inputcollid);
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
2003-06-29 02:33:44 +02:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
2003-06-29 02:33:44 +02:00
READ_DONE();
}
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/*
* _readBoolExpr
*/
static BoolExpr *
_readBoolExpr(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_LOCALS(BoolExpr);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/* do-it-yourself enum representation */
token = pg_strtok(&length); /* skip :boolop */
token = pg_strtok(&length); /* get field value */
if (strncmp(token, "and", 3) == 0)
local_node->boolop = AND_EXPR;
else if (strncmp(token, "or", 2) == 0)
local_node->boolop = OR_EXPR;
else if (strncmp(token, "not", 3) == 0)
local_node->boolop = NOT_EXPR;
else
Error message editing in backend/bootstrap, /lib, /nodes, /port.
2003-07-23 01:30:39 +02:00
elog(ERROR, "unrecognized boolop \"%.*s\"", length, token);
First cut at full support for OUTER JOINs. There are still a few loose ends to clean up (see my message of same date to pghackers), but mostly it works. INITDB REQUIRED!
2000-09-12 23:07:18 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readSubLink
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static SubLink *
_readSubLink(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(SubLink);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_ENUM_FIELD(subLinkType, SubLinkType);
Implement UPDATE tab SET (col1,col2,...) = (SELECT ...), ... This SQL-standard feature allows a sub-SELECT yielding multiple columns (but only one row) to be used to compute the new values of several columns to be updated. While the same results can be had with an independent sub-SELECT per column, such a workaround can require a great deal of duplicated computation. The standard actually says that the source for a multi-column assignment could be any row-valued expression. The implementation used here is tightly tied to our existing sub-SELECT support and can't handle other cases; the Bison grammar would have some issues with them too. However, I don't feel too bad about this since other cases can be converted into sub-SELECTs. For instance, "SET (a,b,c) = row_valued_function(x)" could be written "SET (a,b,c) = (SELECT * FROM row_valued_function(x))".
2014-06-18 19:22:25 +02:00
READ_INT_FIELD(subLinkId);
Implement SQL-compliant treatment of row comparisons for < <= > >= cases (previously we only did = and <> correctly). Also, allow row comparisons with any operators that are in btree opclasses, not only those with these specific names. This gets rid of a whole lot of indefensible assumptions about the behavior of particular operators based on their names ... though it's still true that IN and NOT IN expand to "= ANY". The patch adds a RowCompareExpr expression node type, and makes some changes in the representation of ANY/ALL/ROWCOMPARE SubLinks so that they can share code with RowCompareExpr. I have not yet done anything about making RowCompareExpr an indexable operator, but will look at that soon. initdb forced due to changes in stored rules.
2005-12-28 02:30:02 +01:00
READ_NODE_FIELD(testexpr);
Further tweaking of parsetree & plantree representation of SubLinks. Simplify SubLink by storing just a List of operator OIDs, instead of a list of incomplete OpExprs --- that was a bizarre and bulky choice, with no redeeming social value since we have to build new OpExprs anyway when forming the plan tree.
2003-01-10 22:08:15 +01:00
READ_NODE_FIELD(operName);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(subselect);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/*
Clean up plantree representation of SubPlan-s --- SubLink does not appear in the planned representation of a subplan at all any more, only SubPlan. This means subselect.c doesn't scribble on its input anymore, which seems like a good thing; and there are no longer three different possible interpretations of a SubLink. Simplify node naming and improve comments in primnodes.h. No change to stored rules, though.
2002-12-14 01:17:59 +01:00
* _readSubPlan is not needed since it doesn't appear in stored rules.
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readFieldSelect
*/
static FieldSelect *
_readFieldSelect(void)
{
READ_LOCALS(FieldSelect);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
READ_INT_FIELD(fieldnum);
READ_OID_FIELD(resulttype);
READ_INT_FIELD(resulttypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(resultcollid);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support assignment to subfields of composite columns in UPDATE and INSERT. As a side effect, cause subscripts in INSERT targetlists to do something more or less sensible; previously we evaluated such subscripts and then effectively ignored them. Another side effect is that UPDATE-ing an element or slice of an array value that is NULL now produces a non-null result, namely an array containing just the assigned-to positions.
2004-06-09 21:08:20 +02:00
/*
* _readFieldStore
*/
static FieldStore *
_readFieldStore(void)
{
READ_LOCALS(FieldStore);
READ_NODE_FIELD(arg);
READ_NODE_FIELD(newvals);
READ_NODE_FIELD(fieldnums);
READ_OID_FIELD(resulttype);
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readRelabelType
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static RelabelType *
_readRelabelType(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(RelabelType);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
READ_OID_FIELD(resulttype);
READ_INT_FIELD(resulttypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(resultcollid);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_ENUM_FIELD(relabelformat, CoercionForm);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Downgrade implicit casts to text to be assignment-only, except for the ones from the other string-category types; this eliminates a lot of surprising interpretations that the parser could formerly make when there was no directly applicable operator. Create a general mechanism that supports casts to and from the standard string types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's I/O functions. These new casts are assignment-only in the to-string direction, explicit-only in the other, and therefore should create no surprising behavior. Remove a bunch of thereby-obsoleted datatype-specific casting functions. The "general mechanism" is a new expression node type CoerceViaIO that can actually convert between *any* two datatypes if their external text representations are compatible. This is more general than needed for the immediate feature, but might be useful in plpgsql or other places in future. This commit does nothing about the issue that applying the concatenation operator || to non-text types will now fail, often with strange error messages due to misinterpreting the operator as array concatenation. Since it often (not always) worked before, we should either make it succeed or at least give a more user-friendly error; but details are still under debate. Peter Eisentraut and Tom Lane
2007-06-05 23:31:09 +02:00
/*
* _readCoerceViaIO
*/
static CoerceViaIO *
_readCoerceViaIO(void)
{
READ_LOCALS(CoerceViaIO);
READ_NODE_FIELD(arg);
READ_OID_FIELD(resulttype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(resultcollid);
Downgrade implicit casts to text to be assignment-only, except for the ones from the other string-category types; this eliminates a lot of surprising interpretations that the parser could formerly make when there was no directly applicable operator. Create a general mechanism that supports casts to and from the standard string types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's I/O functions. These new casts are assignment-only in the to-string direction, explicit-only in the other, and therefore should create no surprising behavior. Remove a bunch of thereby-obsoleted datatype-specific casting functions. The "general mechanism" is a new expression node type CoerceViaIO that can actually convert between *any* two datatypes if their external text representations are compatible. This is more general than needed for the immediate feature, but might be useful in plpgsql or other places in future. This commit does nothing about the issue that applying the concatenation operator || to non-text types will now fail, often with strange error messages due to misinterpreting the operator as array concatenation. Since it often (not always) worked before, we should either make it succeed or at least give a more user-friendly error; but details are still under debate. Peter Eisentraut and Tom Lane
2007-06-05 23:31:09 +02:00
READ_ENUM_FIELD(coerceformat, CoercionForm);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Downgrade implicit casts to text to be assignment-only, except for the ones from the other string-category types; this eliminates a lot of surprising interpretations that the parser could formerly make when there was no directly applicable operator. Create a general mechanism that supports casts to and from the standard string types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's I/O functions. These new casts are assignment-only in the to-string direction, explicit-only in the other, and therefore should create no surprising behavior. Remove a bunch of thereby-obsoleted datatype-specific casting functions. The "general mechanism" is a new expression node type CoerceViaIO that can actually convert between *any* two datatypes if their external text representations are compatible. This is more general than needed for the immediate feature, but might be useful in plpgsql or other places in future. This commit does nothing about the issue that applying the concatenation operator || to non-text types will now fail, often with strange error messages due to misinterpreting the operator as array concatenation. Since it often (not always) worked before, we should either make it succeed or at least give a more user-friendly error; but details are still under debate. Peter Eisentraut and Tom Lane
2007-06-05 23:31:09 +02:00
READ_DONE();
}
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
/*
* _readArrayCoerceExpr
*/
static ArrayCoerceExpr *
_readArrayCoerceExpr(void)
{
READ_LOCALS(ArrayCoerceExpr);
READ_NODE_FIELD(arg);
Support arrays over domains. Allowing arrays with a domain type as their element type was left un-done in the original domain patch, but not for any very good reason. This omission leads to such surprising results as array_agg() not working on a domain column, because the parser can't identify a suitable output type for the polymorphic aggregate. In order to fix this, first clean up the APIs of coerce_to_domain() and some internal functions in parse_coerce.c so that we consistently pass around a CoercionContext along with CoercionForm. Previously, we sometimes passed an "isExplicit" boolean flag instead, which is strictly less information; and coerce_to_domain() didn't even get that, but instead had to reverse-engineer isExplicit from CoercionForm. That's contrary to the documentation in primnodes.h that says that CoercionForm only affects display and not semantics. I don't think this change fixes any live bugs, but it makes things more consistent. The main reason for doing it though is that now build_coercion_expression() receives ccontext, which it needs in order to be able to recursively invoke coerce_to_target_type(). Next, reimplement ArrayCoerceExpr so that the node does not directly know any details of what has to be done to the individual array elements while performing the array coercion. Instead, the per-element processing is represented by a sub-expression whose input is a source array element and whose output is a target array element. This simplifies life in parse_coerce.c, because it can build that sub-expression by a recursive invocation of coerce_to_target_type(). The executor now handles the per-element processing as a compiled expression instead of hard-wired code. The main advantage of this is that we can use a single ArrayCoerceExpr to handle as many as three successive steps per element: base type conversion, typmod coercion, and domain constraint checking. The old code used two stacked ArrayCoerceExprs to handle type + typmod coercion, which was pretty inefficient, and adding yet another array deconstruction to do domain constraint checking seemed very unappetizing. In the case where we just need a single, very simple coercion function, doing this straightforwardly leads to a noticeable increase in the per-array-element runtime cost. Hence, add an additional shortcut evalfunc in execExprInterp.c that skips unnecessary overhead for that specific form of expression. The runtime speed of simple cases is within 1% or so of where it was before, while cases that previously required two levels of array processing are significantly faster. Finally, create an implicit array type for every domain type, as we do for base types, enums, etc. Everything except the array-coercion case seems to just work without further effort. Tom Lane, reviewed by Andrew Dunstan Discussion: https://postgr.es/m/9852.1499791473@sss.pgh.pa.us
2017-09-30 19:40:56 +02:00
READ_NODE_FIELD(elemexpr);
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
READ_OID_FIELD(resulttype);
READ_INT_FIELD(resulttypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(resultcollid);
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
READ_ENUM_FIELD(coerceformat, CoercionForm);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
READ_DONE();
}
Instead of supposing (wrongly, in the general case) that the rowtype of an inheritance child table is binary-compatible with the rowtype of its parent, invent an expression node type that does the conversion correctly. Fixes the new bug exhibited by Kris Shannon as well as a lot of old bugs that would only show up when using multiple inheritance or after altering the parent table.
2004-12-12 00:26:51 +01:00
/*
* _readConvertRowtypeExpr
*/
static ConvertRowtypeExpr *
_readConvertRowtypeExpr(void)
{
READ_LOCALS(ConvertRowtypeExpr);
READ_NODE_FIELD(arg);
READ_OID_FIELD(resulttype);
READ_ENUM_FIELD(convertformat, CoercionForm);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Instead of supposing (wrongly, in the general case) that the rowtype of an inheritance child table is binary-compatible with the rowtype of its parent, invent an expression node type that does the conversion correctly. Fixes the new bug exhibited by Kris Shannon as well as a lot of old bugs that would only show up when using multiple inheritance or after altering the parent table.
2004-12-12 00:26:51 +01:00
READ_DONE();
}
Split CollateClause into separate raw and analyzed node types. CollateClause is now used only in raw grammar output, and CollateExpr after parse analysis. This is for clarity and to avoid carrying collation names in post-analysis parse trees: that's both wasteful and possibly misleading, since the collation's name could be changed while the parsetree still exists. Also, clean up assorted infelicities and omissions in processing of the node type.
2011-03-11 22:27:51 +01:00
/*
* _readCollateExpr
*/
static CollateExpr *
_readCollateExpr(void)
{
READ_LOCALS(CollateExpr);
READ_NODE_FIELD(arg);
READ_OID_FIELD(collOid);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readCaseExpr
*/
static CaseExpr *
_readCaseExpr(void)
{
READ_LOCALS(CaseExpr);
Fix using indices in OR. EXPLAIN all indices used.
1998-11-22 11:48:45 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(casetype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(casecollid);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
READ_NODE_FIELD(args);
READ_NODE_FIELD(defresult);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
> > Prevent sorting if result is already sorted > > > > was implemented by Jan Wieck. > > His work is for ascending order cases. > > > > Here is a patch to prevent sorting also in descending > > order cases. > > Because I had already changed _bt_first() to position > > backward correctly before v6.5,this patch would work. > > Hiroshi Inoue Inoue@tpf.co.jp
1999-08-09 08:20:27 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readCaseWhen
Tid access method feature from Hiroshi Inoue, Inoue@tpf.co.jp
1999-11-23 21:07:06 +01:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static CaseWhen *
_readCaseWhen(void)
Tid access method feature from Hiroshi Inoue, Inoue@tpf.co.jp
1999-11-23 21:07:06 +01:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(CaseWhen);
Tid access method feature from Hiroshi Inoue, Inoue@tpf.co.jp
1999-11-23 21:07:06 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(expr);
READ_NODE_FIELD(result);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Tid access method feature from Hiroshi Inoue, Inoue@tpf.co.jp
1999-11-23 21:07:06 +01:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Tid access method feature from Hiroshi Inoue, Inoue@tpf.co.jp
1999-11-23 21:07:06 +01:00
}
Reimplement CASE val WHEN compval1 THEN ... WHEN compval2 THEN ... END so that the 'val' is computed only once, per recent discussion. The speedup is not much when 'val' is just a simple variable, but could be significant for larger expressions. More importantly this avoids issues with multiple evaluations of a volatile 'val', and it allows the CASE expression to be reverse-listed in its original form by ruleutils.c.
2004-03-17 21:48:43 +01:00
/*
* _readCaseTestExpr
*/
static CaseTestExpr *
_readCaseTestExpr(void)
{
READ_LOCALS(CaseTestExpr);
READ_OID_FIELD(typeId);
READ_INT_FIELD(typeMod);
Per-column collation support This adds collation support for columns and domains, a COLLATE clause to override it per expression, and B-tree index support. Peter Eisentraut reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch
2011-02-08 22:04:18 +01:00
READ_OID_FIELD(collation);
Reimplement CASE val WHEN compval1 THEN ... WHEN compval2 THEN ... END so that the 'val' is computed only once, per recent discussion. The speedup is not much when 'val' is just a simple variable, but could be significant for larger expressions. More importantly this avoids issues with multiple evaluations of a volatile 'val', and it allows the CASE expression to be reverse-listed in its original form by ruleutils.c.
2004-03-17 21:48:43 +01:00
READ_DONE();
}
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
/*
* _readArrayExpr
*/
static ArrayExpr *
_readArrayExpr(void)
{
READ_LOCALS(ArrayExpr);
READ_OID_FIELD(array_typeid);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(array_collid);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
READ_OID_FIELD(element_typeid);
READ_NODE_FIELD(elements);
Fix ARRAY[] construct so that in multidimensional case, elements can be anything yielding an array of the proper kind, not only sub-ARRAY[] constructs; do subscript checking at runtime not parse time. Also, adjust array_cat to make array || array comply with the SQL99 spec. Joe Conway
2003-08-18 01:43:27 +02:00
READ_BOOL_FIELD(multidims);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
READ_DONE();
}
Promote row expressions to full-fledged citizens of the expression syntax, rather than allowing them only in a few special cases as before. In particular you can now pass a ROW() construct to a function that accepts a rowtype parameter. Internal generation of RowExprs fixes a number of corner cases that used to not work very well, such as referencing the whole-row result of a JOIN or subquery. This represents a further step in the work I started a month or so back to make rowtype values into first-class citizens.
2004-05-11 00:44:49 +02:00
/*
* _readRowExpr
*/
static RowExpr *
_readRowExpr(void)
{
READ_LOCALS(RowExpr);
READ_NODE_FIELD(args);
READ_OID_FIELD(row_typeid);
READ_ENUM_FIELD(row_format, CoercionForm);
When expanding a whole-row Var into a RowExpr during ResolveNew(), attach the column alias names of the RTE referenced by the Var to the RowExpr. This is needed to allow ruleutils.c to correctly deparse FieldSelect nodes referencing such a construct. Per my recent bug report. Adding a field to RowExpr forces initdb (because of stored rules changes) so this solution is not back-patchable; which is unfortunate because 8.2 and 8.3 have this issue. But it only affects EXPLAIN for some pretty odd corner cases, so we can probably live without a solution for the back branches.
2008-10-06 19:39:26 +02:00
READ_NODE_FIELD(colnames);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Promote row expressions to full-fledged citizens of the expression syntax, rather than allowing them only in a few special cases as before. In particular you can now pass a ROW() construct to a function that accepts a rowtype parameter. Internal generation of RowExprs fixes a number of corner cases that used to not work very well, such as referencing the whole-row result of a JOIN or subquery. This represents a further step in the work I started a month or so back to make rowtype values into first-class citizens.
2004-05-11 00:44:49 +02:00
READ_DONE();
}
Implement SQL-compliant treatment of row comparisons for < <= > >= cases (previously we only did = and <> correctly). Also, allow row comparisons with any operators that are in btree opclasses, not only those with these specific names. This gets rid of a whole lot of indefensible assumptions about the behavior of particular operators based on their names ... though it's still true that IN and NOT IN expand to "= ANY". The patch adds a RowCompareExpr expression node type, and makes some changes in the representation of ANY/ALL/ROWCOMPARE SubLinks so that they can share code with RowCompareExpr. I have not yet done anything about making RowCompareExpr an indexable operator, but will look at that soon. initdb forced due to changes in stored rules.
2005-12-28 02:30:02 +01:00
/*
* _readRowCompareExpr
*/
static RowCompareExpr *
_readRowCompareExpr(void)
{
READ_LOCALS(RowCompareExpr);
READ_ENUM_FIELD(rctype, RowCompareType);
READ_NODE_FIELD(opnos);
Restructure operator classes to allow improved handling of cross-data-type cases. Operator classes now exist within "operator families". While most families are equivalent to a single class, related classes can be grouped into one family to represent the fact that they are semantically compatible. Cross-type operators are now naturally adjunct parts of a family, without having to wedge them into a particular opclass as we had done originally. This commit restructures the catalogs and cleans up enough of the fallout so that everything still works at least as well as before, but most of the work needed to actually improve the planner's behavior will come later. Also, there are not yet CREATE/DROP/ALTER OPERATOR FAMILY commands; the only way to create a new family right now is to allow CREATE OPERATOR CLASS to make one by default. I owe some more documentation work, too. But that can all be done in smaller pieces once this infrastructure is in place.
2006-12-23 01:43:13 +01:00
READ_NODE_FIELD(opfamilies);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_NODE_FIELD(inputcollids);
Implement SQL-compliant treatment of row comparisons for < <= > >= cases (previously we only did = and <> correctly). Also, allow row comparisons with any operators that are in btree opclasses, not only those with these specific names. This gets rid of a whole lot of indefensible assumptions about the behavior of particular operators based on their names ... though it's still true that IN and NOT IN expand to "= ANY". The patch adds a RowCompareExpr expression node type, and makes some changes in the representation of ANY/ALL/ROWCOMPARE SubLinks so that they can share code with RowCompareExpr. I have not yet done anything about making RowCompareExpr an indexable operator, but will look at that soon. initdb forced due to changes in stored rules.
2005-12-28 02:30:02 +01:00
READ_NODE_FIELD(largs);
READ_NODE_FIELD(rargs);
READ_DONE();
}
COALESCE() and NULLIF() are now first-class expressions, not macros that turn into CASE expressions. They evaluate their arguments at most once. Patch by Kris Jurka, review and (very light) editorializing by me.
2003-02-16 03:30:39 +01:00
/*
* _readCoalesceExpr
*/
static CoalesceExpr *
_readCoalesceExpr(void)
{
READ_LOCALS(CoalesceExpr);
READ_OID_FIELD(coalescetype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(coalescecollid);
COALESCE() and NULLIF() are now first-class expressions, not macros that turn into CASE expressions. They evaluate their arguments at most once. Patch by Kris Jurka, review and (very light) editorializing by me.
2003-02-16 03:30:39 +01:00
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
COALESCE() and NULLIF() are now first-class expressions, not macros that turn into CASE expressions. They evaluate their arguments at most once. Patch by Kris Jurka, review and (very light) editorializing by me.
2003-02-16 03:30:39 +01:00
READ_DONE();
}
Add Oracle-compatible GREATEST and LEAST functions. Pavel Stehule
2005-06-27 00:05:42 +02:00
/*
* _readMinMaxExpr
*/
static MinMaxExpr *
_readMinMaxExpr(void)
{
READ_LOCALS(MinMaxExpr);
READ_OID_FIELD(minmaxtype);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(minmaxcollid);
READ_OID_FIELD(inputcollid);
Add Oracle-compatible GREATEST and LEAST functions. Pavel Stehule
2005-06-27 00:05:42 +02:00
READ_ENUM_FIELD(op, MinMaxOp);
READ_NODE_FIELD(args);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Add Oracle-compatible GREATEST and LEAST functions. Pavel Stehule
2005-06-27 00:05:42 +02:00
READ_DONE();
}
Improve parsetree representation of special functions such as CURRENT_DATE. We implement a dozen or so parameterless functions that the SQL standard defines special syntax for. Up to now, that was done by converting them into more or less ad-hoc constructs such as "'now'::text::date". That's messy for multiple reasons: it exposes what should be implementation details to users, and performance is worse than it needs to be in several cases. To improve matters, invent a new expression node type SQLValueFunction that can represent any of these parameterless functions. Bump catversion because this changes stored parsetrees for rules. Discussion: <30058.1463091294@sss.pgh.pa.us>
2016-08-17 02:33:01 +02:00
/*
* _readSQLValueFunction
*/
static SQLValueFunction *
_readSQLValueFunction(void)
{
READ_LOCALS(SQLValueFunction);
READ_ENUM_FIELD(op, SQLValueFunctionOp);
READ_OID_FIELD(type);
READ_INT_FIELD(typmod);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
/*
* _readXmlExpr
*/
static XmlExpr *
_readXmlExpr(void)
{
READ_LOCALS(XmlExpr);
READ_ENUM_FIELD(op, XmlExprOp);
READ_STRING_FIELD(name);
READ_NODE_FIELD(named_args);
READ_NODE_FIELD(arg_names);
READ_NODE_FIELD(args);
Implement XMLSERIALIZE for real. Analogously, make the xml to text cast observe the xmloption. Reorganize the representation of the XML option in the parse tree and the API to make it easier to manage and understand. Add regression tests for parsing back XML expressions.
2007-02-03 15:06:56 +01:00
READ_ENUM_FIELD(xmloption, XmlOptionType);
READ_OID_FIELD(type);
READ_INT_FIELD(typmod);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readNullTest
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
2000-09-29 20:21:41 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static NullTest *
_readNullTest(void)
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
2000-09-29 20:21:41 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(NullTest);
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
2000-09-29 20:21:41 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
READ_ENUM_FIELD(nulltesttype, NullTestType);
Add an "argisrow" field to NullTest nodes, following a plan made way back in 8.2beta but never carried out. This avoids repetitive tests of whether the argument is of scalar or composite type. Also, be a bit more paranoid about composite arguments in some places where we previously weren't checking.
2010-01-02 00:03:10 +01:00
READ_BOOL_FIELD(argisrow);
Add parse location fields to NullTest and BooleanTest structs. We did not need a location tag on NullTest or BooleanTest before, because no error messages referred directly to their locations. That's planned to change though, so add these fields in a separate housekeeping commit. Catversion bump because stored rules may change.
2015-02-22 20:40:27 +01:00
READ_LOCATION_FIELD(location);
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
2000-09-29 20:21:41 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Subselects in FROM clause, per ISO syntax: FROM (SELECT ...) [AS] alias. (Don't forget that an alias is required.) Views reimplemented as expanding to subselect-in-FROM. Grouping, aggregates, DISTINCT in views actually work now (he says optimistically). No UNION support in subselects/views yet, but I have some ideas about that. Rule-related permissions checking moved out of rewriter and into executor. INITDB REQUIRED!
2000-09-29 20:21:41 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readBooleanTest
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static BooleanTest *
_readBooleanTest(void)
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(BooleanTest);
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
READ_ENUM_FIELD(booltesttype, BoolTestType);
Add parse location fields to NullTest and BooleanTest structs. We did not need a location tag on NullTest or BooleanTest before, because no error messages referred directly to their locations. That's planned to change though, so add these fields in a separate housekeeping commit. Catversion bump because stored rules may change.
2015-02-22 20:40:27 +01:00
READ_LOCATION_FIELD(location);
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
* _readCoerceToDomain
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
static CoerceToDomain *
_readCoerceToDomain(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
READ_LOCALS(CoerceToDomain);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(arg);
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
READ_OID_FIELD(resulttype);
READ_INT_FIELD(resulttypmod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(resultcollid);
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
READ_ENUM_FIELD(coercionformat, CoercionForm);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
* _readCoerceToDomainValue
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
*/
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
static CoerceToDomainValue *
_readCoerceToDomainValue(void)
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
{
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
READ_LOCALS(CoerceToDomainValue);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(typeId);
READ_INT_FIELD(typeMod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(collation);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Code review for UPDATE tab SET col = DEFAULT patch ... whack it around so it has some chance of working in rules ...
2003-07-03 18:34:26 +02:00
/*
* _readSetToDefault
*/
static SetToDefault *
_readSetToDefault(void)
{
READ_LOCALS(SetToDefault);
READ_OID_FIELD(typeId);
READ_INT_FIELD(typeMod);
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
READ_OID_FIELD(collation);
Extend the parser location infrastructure to include a location field in most node types used in expression trees (both before and after parse analysis). This allows us to place an error cursor in many situations where we formerly could not, because the information wasn't available beyond the very first level of parse analysis. There's a fair amount of work still to be done to persuade individual ereport() calls to actually include an error location, but this gets the initdb-forcing part of the work out of the way; and the situation is already markedly better than before for complaints about unimplementable implicit casts, such as CASE and UNION constructs with incompatible alternative data types. Per my proposal of a few days ago.
2008-08-29 01:09:48 +02:00
READ_LOCATION_FIELD(location);
Code review for UPDATE tab SET col = DEFAULT patch ... whack it around so it has some chance of working in rules ...
2003-07-03 18:34:26 +02:00
READ_DONE();
}
Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once have been a reason to disallow that, but it seems to work now, and it's really rather necessary if you want to select a row via a cursor and then update it in a concurrent-safe fashion. Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
2007-06-11 03:16:30 +02:00
/*
* _readCurrentOfExpr
*/
static CurrentOfExpr *
_readCurrentOfExpr(void)
{
READ_LOCALS(CurrentOfExpr);
READ_UINT_FIELD(cvarno);
READ_STRING_FIELD(cursor_name);
Improve UPDATE/DELETE WHERE CURRENT OF so that they can be used from plpgsql with a plpgsql-defined cursor. The underlying mechanism for this is that the main SQL engine will now take "WHERE CURRENT OF $n" where $n is a refcursor parameter. Not sure if we should document that fact or consider it an implementation detail. Per discussion with Pavel Stehule.
2007-06-12 00:22:42 +02:00
READ_INT_FIELD(cursor_param);
Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once have been a reason to disallow that, but it seems to work now, and it's really rather necessary if you want to select a row via a cursor and then update it in a concurrent-safe fashion. Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
2007-06-11 03:16:30 +02:00
READ_DONE();
}
Code review for NextValueExpr expression node type. Add missing infrastructure for this node type, notably in ruleutils.c where its lack could demonstrably cause EXPLAIN to fail. Add outfuncs/readfuncs support. (outfuncs support is useful today for debugging purposes. The readfuncs support may never be needed, since at present it would only matter for parallel query and NextValueExpr should never appear in a parallelizable query; but it seems like a bad idea to have a primnode type that isn't fully supported here.) Teach planner infrastructure that NextValueExpr is a volatile, parallel-unsafe, non-leaky expression node with cost cpu_operator_cost. Given its limited scope of usage, there *might* be no live bug today from the lack of that knowledge, but it's certainly going to bite us on the rear someday. Teach pg_stat_statements about the new node type, too. While at it, also teach cost_qual_eval() that MinMaxExpr, SQLValueFunction, XmlExpr, and CoerceToDomain should be charged as cpu_operator_cost. Failing to do this for SQLValueFunction was an oversight in my commit 0bb51aa96. The others are longer-standing oversights, but no time like the present to fix them. (In principle, CoerceToDomain could have cost much higher than this, but it doesn't presently seem worth trying to examine the domain's constraints here.) Modify execExprInterp.c to execute NextValueExpr as an out-of-line function; it seems quite unlikely to me that it's worth insisting that it be inlined in all expression eval methods. Besides, providing the out-of-line function doesn't stop anyone from inlining if they want to. Adjust some places where NextValueExpr support had been inserted with the aid of a dartboard rather than keeping it in the same order as elsewhere. Discussion: https://postgr.es/m/23862.1499981661@sss.pgh.pa.us
2017-07-14 21:25:43 +02:00
/*
* _readNextValueExpr
*/
static NextValueExpr *
_readNextValueExpr(void)
{
READ_LOCALS(NextValueExpr);
READ_OID_FIELD(seqid);
READ_OID_FIELD(typeId);
READ_DONE();
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
* _readInferenceElem
*/
static InferenceElem *
_readInferenceElem(void)
{
READ_LOCALS(InferenceElem);
READ_NODE_FIELD(expr);
READ_OID_FIELD(infercollid);
Refactor ON CONFLICT index inference parse tree representation. Defer lookup of opfamily and input type of a of a user specified opclass until the optimizer selects among available unique indexes; and store the opclass in the parse analyzed tree instead. The primary reason for doing this is that for rule deparsing it's easier to use the opclass than the previous representation. While at it also rename a variable in the inference code to better fit it's purpose. This is separate from the actual fixes for deparsing to make review easier.
2015-05-19 21:17:52 +02:00
READ_OID_FIELD(inferopclass);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
READ_DONE();
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
* _readTargetEntry
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
static TargetEntry *
_readTargetEntry(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(TargetEntry);
READ_NODE_FIELD(expr);
Merge Resdom nodes into TargetEntry nodes to simplify code and save a few palloc's. I also chose to eliminate the restype and restypmod fields entirely, since they are redundant with information stored in the node's contained expression; re-examining the expression at need seems simpler and more reliable than trying to keep restype/restypmod up to date. initdb forced due to change in contents of stored rules.
2005-04-06 18:34:07 +02:00
READ_INT_FIELD(resno);
READ_STRING_FIELD(resname);
READ_UINT_FIELD(ressortgroupref);
READ_OID_FIELD(resorigtbl);
READ_INT_FIELD(resorigcol);
READ_BOOL_FIELD(resjunk);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/*
* _readRangeTblRef
*/
static RangeTblRef *
_readRangeTblRef(void)
{
READ_LOCALS(RangeTblRef);
READ_INT_FIELD(rtindex);
READ_DONE();
}
/*
* _readJoinExpr
*/
static JoinExpr *
_readJoinExpr(void)
{
READ_LOCALS(JoinExpr);
READ_ENUM_FIELD(jointype, JoinType);
READ_BOOL_FIELD(isNatural);
READ_NODE_FIELD(larg);
READ_NODE_FIELD(rarg);
Make backend header files C++ safe This alters various incidental uses of C++ key words to use other similar identifiers, so that a C++ compiler won't choke outright. You still (probably) need extern "C" { }; around the inclusion of backend headers. based on a patch by Kurt Harriman <harriman@acm.org> Also add a script cpluspluscheck to check for C++ compatibility in the future. As of right now, this passes without error for me.
2009-07-16 08:33:46 +02:00
READ_NODE_FIELD(usingClause);
Allow an alias to be attached to a JOIN ... USING This allows something like SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x where x has the columns a, b, c and unlike a regular alias it does not hide the range variables of the tables being joined t1 and t2. Per SQL:2016 feature F404 "Range variable for common column names". Reviewed-by: Vik Fearing <vik.fearing@2ndquadrant.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/454638cf-d563-ab76-a585-2564428062af@2ndquadrant.com
2021-03-31 17:09:24 +02:00
READ_NODE_FIELD(join_using_alias);
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
READ_NODE_FIELD(quals);
READ_NODE_FIELD(alias);
READ_INT_FIELD(rtindex);
READ_DONE();
}
/*
* _readFromExpr
*/
static FromExpr *
_readFromExpr(void)
{
READ_LOCALS(FromExpr);
READ_NODE_FIELD(fromlist);
READ_NODE_FIELD(quals);
READ_DONE();
}
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
/*
* _readOnConflictExpr
*/
static OnConflictExpr *
_readOnConflictExpr(void)
{
READ_LOCALS(OnConflictExpr);
READ_ENUM_FIELD(action, OnConflictAction);
READ_NODE_FIELD(arbiterElems);
READ_NODE_FIELD(arbiterWhere);
Reconcile nodes/*funcs.c with recent work. A few of the discrepancies had semantic significance, but I did not track down the resulting user-visible bugs, if any. Back-patch to 9.5, where all but one discrepancy appeared. The _equalCreateEventTrigStmt() situation dates to 9.3 but does not affect semantics. catversion bump due to readfuncs.c field order changes.
2015-08-06 02:44:27 +02:00
READ_OID_FIELD(constraint);
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
READ_NODE_FIELD(onConflictSet);
READ_NODE_FIELD(onConflictWhere);
READ_INT_FIELD(exclRelIndex);
READ_NODE_FIELD(exclRelTlist);
READ_DONE();
}
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
Common SQL/JSON clauses This introduces some of the building blocks used by the SQL/JSON constructor and query functions. Specifically, it provides node executor and grammar support for the FORMAT JSON [ENCODING foo] clause, and values decorated with it, and for the RETURNING clause. The following SQL/JSON patches will leverage these. Nikita Glukhov (who probably deserves an award for perseverance). Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:00:49 +01:00
/*
* _readJsonFormat
*/
static JsonFormat *
_readJsonFormat(void)
{
READ_LOCALS(JsonFormat);
READ_ENUM_FIELD(format_type, JsonFormatType);
READ_ENUM_FIELD(encoding, JsonEncoding);
READ_LOCATION_FIELD(location);
READ_DONE();
}
/*
* _readJsonReturning
*/
static JsonReturning *
_readJsonReturning(void)
{
READ_LOCALS(JsonReturning);
READ_NODE_FIELD(format);
READ_OID_FIELD(typid);
READ_INT_FIELD(typmod);
READ_DONE();
}
/*
* _readJsonValueExpr
*/
static JsonValueExpr *
_readJsonValueExpr(void)
{
READ_LOCALS(JsonValueExpr);
READ_NODE_FIELD(raw_expr);
READ_NODE_FIELD(formatted_expr);
READ_NODE_FIELD(format);
READ_DONE();
}
SQL/JSON constructors This patch introduces the SQL/JSON standard constructors for JSON: JSON() JSON_ARRAY() JSON_ARRAYAGG() JSON_OBJECT() JSON_OBJECTAGG() For the most part these functions provide facilities that mimic existing json/jsonb functions. However, they also offer some useful additional functionality. In addition to text input, the JSON() function accepts bytea input, which it will decode and constuct a json value from. The other functions provide useful options for handling duplicate keys and null values. This series of patches will be followed by a consolidated documentation patch. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:10 +01:00
/*
* _readJsonConstructorExpr
*/
static JsonConstructorExpr *
_readJsonConstructorExpr(void)
{
READ_LOCALS(JsonConstructorExpr);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_ENUM_FIELD(type, JsonConstructorType);
SQL/JSON constructors This patch introduces the SQL/JSON standard constructors for JSON: JSON() JSON_ARRAY() JSON_ARRAYAGG() JSON_OBJECT() JSON_OBJECTAGG() For the most part these functions provide facilities that mimic existing json/jsonb functions. However, they also offer some useful additional functionality. In addition to text input, the JSON() function accepts bytea input, which it will decode and constuct a json value from. The other functions provide useful options for handling duplicate keys and null values. This series of patches will be followed by a consolidated documentation patch. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:10 +01:00
READ_NODE_FIELD(args);
READ_NODE_FIELD(func);
READ_NODE_FIELD(coercion);
READ_NODE_FIELD(returning);
READ_BOOL_FIELD(absent_on_null);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_BOOL_FIELD(unique);
SQL/JSON constructors This patch introduces the SQL/JSON standard constructors for JSON: JSON() JSON_ARRAY() JSON_ARRAYAGG() JSON_OBJECT() JSON_OBJECTAGG() For the most part these functions provide facilities that mimic existing json/jsonb functions. However, they also offer some useful additional functionality. In addition to text input, the JSON() function accepts bytea input, which it will decode and constuct a json value from. The other functions provide useful options for handling duplicate keys and null values. This series of patches will be followed by a consolidated documentation patch. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:10 +01:00
READ_LOCATION_FIELD(location);
READ_DONE();
}
SQL/JSON query functions This introduces the SQL/JSON functions for querying JSON data using jsonpath expressions. The functions are: JSON_EXISTS() JSON_QUERY() JSON_VALUE() All of these functions only operate on jsonb. The workaround for now is to cast the argument to jsonb. JSON_EXISTS() tests if the jsonpath expression applied to the jsonb value yields any values. JSON_VALUE() must return a single value, and an error occurs if it tries to return multiple values. JSON_QUERY() must return a json object or array, and there are various WRAPPER options for handling scalar or multi-value results. Both these functions have options for handling EMPTY and ERROR conditions. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:11:14 +01:00
/*
* _readJsonBehavior
*/
static JsonBehavior *
_readJsonBehavior(void)
{
READ_LOCALS(JsonBehavior);
READ_ENUM_FIELD(btype, JsonBehaviorType);
READ_NODE_FIELD(default_expr);
READ_DONE();
}
/*
* _readJsonExpr
*/
static JsonExpr *
_readJsonExpr(void)
{
READ_LOCALS(JsonExpr);
READ_ENUM_FIELD(op, JsonExprOp);
READ_NODE_FIELD(formatted_expr);
READ_NODE_FIELD(result_coercion);
READ_NODE_FIELD(format);
READ_NODE_FIELD(path_spec);
READ_NODE_FIELD(passing_names);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_NODE_FIELD(passing_values);
SQL/JSON query functions This introduces the SQL/JSON functions for querying JSON data using jsonpath expressions. The functions are: JSON_EXISTS() JSON_QUERY() JSON_VALUE() All of these functions only operate on jsonb. The workaround for now is to cast the argument to jsonb. JSON_EXISTS() tests if the jsonpath expression applied to the jsonb value yields any values. JSON_VALUE() must return a single value, and an error occurs if it tries to return multiple values. JSON_QUERY() must return a json object or array, and there are various WRAPPER options for handling scalar or multi-value results. Both these functions have options for handling EMPTY and ERROR conditions. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:11:14 +01:00
READ_NODE_FIELD(returning);
READ_NODE_FIELD(on_empty);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_NODE_FIELD(on_error);
SQL/JSON query functions This introduces the SQL/JSON functions for querying JSON data using jsonpath expressions. The functions are: JSON_EXISTS() JSON_QUERY() JSON_VALUE() All of these functions only operate on jsonb. The workaround for now is to cast the argument to jsonb. JSON_EXISTS() tests if the jsonpath expression applied to the jsonb value yields any values. JSON_VALUE() must return a single value, and an error occurs if it tries to return multiple values. JSON_QUERY() must return a json object or array, and there are various WRAPPER options for handling scalar or multi-value results. Both these functions have options for handling EMPTY and ERROR conditions. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:11:14 +01:00
READ_NODE_FIELD(coercions);
READ_ENUM_FIELD(wrapper, JsonWrapper);
READ_BOOL_FIELD(omit_quotes);
READ_LOCATION_FIELD(location);
READ_DONE();
}
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
static JsonTableParent *
_readJsonTableParent(void)
{
READ_LOCALS(JsonTableParent);
READ_NODE_FIELD(path);
PLAN clauses for JSON_TABLE These clauses allow the user to specify how data from nested paths are joined, allowing considerable freedom in shaping the tabular output of JSON_TABLE. PLAN DEFAULT allows the user to specify the global strategies when dealing with sibling or child nested paths. The is often sufficient to achieve the necessary goal, and is considerably simpler than the full PLAN clause, which allows the user to specify the strategy to be used for each named nested path. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-05 20:09:04 +02:00
READ_STRING_FIELD(name);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_NODE_FIELD(child);
PLAN clauses for JSON_TABLE These clauses allow the user to specify how data from nested paths are joined, allowing considerable freedom in shaping the tabular output of JSON_TABLE. PLAN DEFAULT allows the user to specify the global strategies when dealing with sibling or child nested paths. The is often sufficient to achieve the necessary goal, and is considerably simpler than the full PLAN clause, which allows the user to specify the strategy to be used for each named nested path. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-05 20:09:04 +02:00
READ_BOOL_FIELD(outerJoin);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_INT_FIELD(colMin);
READ_INT_FIELD(colMax);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_BOOL_FIELD(errorOnError);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_DONE();
}
static JsonTableSibling *
_readJsonTableSibling(void)
{
READ_LOCALS(JsonTableSibling);
READ_NODE_FIELD(larg);
READ_NODE_FIELD(rarg);
PLAN clauses for JSON_TABLE These clauses allow the user to specify how data from nested paths are joined, allowing considerable freedom in shaping the tabular output of JSON_TABLE. PLAN DEFAULT allows the user to specify the global strategies when dealing with sibling or child nested paths. The is often sufficient to achieve the necessary goal, and is considerably simpler than the full PLAN clause, which allows the user to specify the strategy to be used for each named nested path. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-05 20:09:04 +02:00
READ_BOOL_FIELD(cross);
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
READ_DONE();
}
SQL/JSON query functions This introduces the SQL/JSON functions for querying JSON data using jsonpath expressions. The functions are: JSON_EXISTS() JSON_QUERY() JSON_VALUE() All of these functions only operate on jsonb. The workaround for now is to cast the argument to jsonb. JSON_EXISTS() tests if the jsonpath expression applied to the jsonb value yields any values. JSON_VALUE() must return a single value, and an error occurs if it tries to return multiple values. JSON_QUERY() must return a json object or array, and there are various WRAPPER options for handling scalar or multi-value results. Both these functions have options for handling EMPTY and ERROR conditions. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:11:14 +01:00
/*
* _readJsonCoercion
*/
static JsonCoercion *
_readJsonCoercion(void)
{
READ_LOCALS(JsonCoercion);
READ_NODE_FIELD(expr);
READ_BOOL_FIELD(via_populate);
READ_BOOL_FIELD(via_io);
READ_OID_FIELD(collation);
READ_DONE();
}
/*
* _readJsonItemCoercions
*/
static JsonItemCoercions *
_readJsonItemCoercions(void)
{
READ_LOCALS(JsonItemCoercions);
READ_NODE_FIELD(null);
READ_NODE_FIELD(string);
READ_NODE_FIELD(numeric);
READ_NODE_FIELD(boolean);
READ_NODE_FIELD(date);
READ_NODE_FIELD(time);
READ_NODE_FIELD(timetz);
READ_NODE_FIELD(timestamp);
READ_NODE_FIELD(timestamptz);
READ_NODE_FIELD(composite);
READ_DONE();
}
IS JSON predicate This patch intrdocuces the SQL standard IS JSON predicate. It operates on text and bytea values representing JSON as well as on the json and jsonb types. Each test has an IS and IS NOT variant. The tests are: IS JSON [VALUE] IS JSON ARRAY IS JSON OBJECT IS JSON SCALAR IS JSON WITH | WITHOUT UNIQUE KEYS These are mostly self-explanatory, but note that IS JSON WITHOUT UNIQUE KEYS is true whenever IS JSON is true, and IS JSON WITH UNIQUE KEYS is true whenever IS JSON is true except it IS JSON OBJECT is true and there are duplicate keys (which is never the case when applied to jsonb values). Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:53 +01:00
/*
* _readJsonIsPredicate
*/
static JsonIsPredicate *
_readJsonIsPredicate()
{
READ_LOCALS(JsonIsPredicate);
READ_NODE_FIELD(expr);
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
READ_NODE_FIELD(format);
READ_ENUM_FIELD(item_type, JsonValueType);
IS JSON predicate This patch intrdocuces the SQL standard IS JSON predicate. It operates on text and bytea values representing JSON as well as on the json and jsonb types. Each test has an IS and IS NOT variant. The tests are: IS JSON [VALUE] IS JSON ARRAY IS JSON OBJECT IS JSON SCALAR IS JSON WITH | WITHOUT UNIQUE KEYS These are mostly self-explanatory, but note that IS JSON WITHOUT UNIQUE KEYS is true whenever IS JSON is true, and IS JSON WITH UNIQUE KEYS is true whenever IS JSON is true except it IS JSON OBJECT is true and there are duplicate keys (which is never the case when applied to jsonb values). Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:53 +01:00
READ_BOOL_FIELD(unique_keys);
READ_LOCATION_FIELD(location);
READ_DONE();
}
Add readfuncs.c support for AppendRelInfo. This is made necessary by the fact that commit 6ef77cf46 added AppendRelInfos to plan trees. I'd concluded that this extra code was not necessary because we don't transmit that data to parallel workers ... but I forgot about -DWRITE_READ_PARSE_PLAN_TREES. Per buildfarm.
2019-12-12 01:08:16 +01:00
/*
* Stuff from pathnodes.h.
*
* Mostly we don't need to read planner nodes back in again, but some
* of these also end up in plan trees.
*/
/*
* _readAppendRelInfo
*/
static AppendRelInfo *
_readAppendRelInfo(void)
{
READ_LOCALS(AppendRelInfo);
READ_UINT_FIELD(parent_relid);
READ_UINT_FIELD(child_relid);
READ_OID_FIELD(parent_reltype);
READ_OID_FIELD(child_reltype);
READ_NODE_FIELD(translated_vars);
READ_INT_FIELD(num_child_cols);
READ_ATTRNUMBER_ARRAY(parent_colnos, local_node->num_child_cols);
READ_OID_FIELD(parent_reloid);
READ_DONE();
}
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
/*
* Stuff from parsenodes.h.
*/
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* _readRangeTblEntry
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
static RangeTblEntry *
Makeover for Unixware 7.1.1 * Makefile: Add more standard targets. Improve shell redirection in GNU make detection. * src/backend/access/transam/rmgr.c: Fix incorrect(?) C. * src/backend/libpq/pqcomm.c (StreamConnection): Work around accept() bug. * src/include/port/unixware.h: ...with help from here. * src/backend/nodes/print.c (plannode_type): Remove some "break"s after "return"s. * src/backend/tcop/dest.c (DestToFunction): ditto. * src/backend/nodes/readfuncs.c: Add proper prototypes. * src/backend/utils/adt/numutils.c (pg_atoi): Cope specially with strtol() setting EINVAL. This saves us from creating an extra set of regression test output for the affected systems. * src/include/storage/s_lock.h (tas): Correct prototype. * src/interfaces/libpq/fe-connect.c (parseServiceInfo): Don't use variable as dimension in array definition. * src/makefiles/Makefile.unixware: Add support for GCC. * src/template/unixware: same here * src/test/regress/expected/abstime-solaris-1947.out: Adjust whitespace. * src/test/regress/expected/horology-solaris-1947.out: Part of this file was evidently missing. * src/test/regress/pg_regress.sh: Fix shell. mkdir -p returns non-zero if the directory exists. * src/test/regress/resultmap: Add entries for Unixware.
2000-10-23 00:15:13 +02:00
_readRangeTblEntry(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_LOCALS(RangeTblEntry);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/* put alias + eref first to make dump more legible */
READ_NODE_FIELD(alias);
READ_NODE_FIELD(eref);
READ_ENUM_FIELD(rtekind, RTEKind);
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
switch (local_node->rtekind)
{
case RTE_RELATION:
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_OID_FIELD(relid);
Add a relkind field to RangeTblEntry to avoid some syscache lookups. The recent additions for FDW support required checking foreign-table-ness in several places in the parse/plan chain. While it's not clear whether that would really result in a noticeable slowdown, it seems best to avoid any performance risk by keeping a copy of the relation's relkind in RangeTblEntry. That might have some other uses later, anyway. Per discussion.
2011-02-23 01:23:23 +01:00
READ_CHAR_FIELD(relkind);
Create an RTE field to record the query's lock mode for each relation. Add RangeTblEntry.rellockmode, which records the appropriate lock mode for each RTE_RELATION rangetable entry (either AccessShareLock, RowShareLock, or RowExclusiveLock depending on the RTE's role in the query). This patch creates the field and makes all creators of RTE nodes fill it in reasonably, but for the moment nothing much is done with it. The plan is to replace assorted post-parser logic that re-determines the right lockmode to use with simple uses of rte->rellockmode. For now, just add Asserts in each of those places that the rellockmode matches what they are computing today. (In some cases the match isn't perfect, so the Asserts are weaker than you might expect; but this seems OK, as per discussion.) This passes check-world for me, but it seems worth pushing in this state to see if the buildfarm finds any problems in cases I failed to test. catversion bump due to change of stored rules. Amit Langote, reviewed by David Rowley and Jesper Pedersen, and whacked around a bit more by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp
2018-09-30 19:55:51 +02:00
READ_INT_FIELD(rellockmode);
TABLESAMPLE, SQL Standard and extensible Add a TABLESAMPLE clause to SELECT statements that allows user to specify random BERNOULLI sampling or block level SYSTEM sampling. Implementation allows for extensible sampling functions to be written, using a standard API. Basic version follows SQLStandard exactly. Usable concrete use cases for the sampling API follow in later commits. Petr Jelinek Reviewed by Michael Paquier and Simon Riggs
2015-05-15 20:37:10 +02:00
READ_NODE_FIELD(tablesample);
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
break;
case RTE_SUBQUERY:
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_NODE_FIELD(subquery);
Add a security_barrier option for views. When a view is marked as a security barrier, it will not be pulled up into the containing query, and no quals will be pushed down into it, so that no function or operator chosen by the user can be applied to rows not exposed by the view. Views not configured with this option cannot provide robust row-level security, but will perform far better. Patch by KaiGai Kohei; original problem report by Heikki Linnakangas (in October 2009!). Review (in earlier versions) by Noah Misch and others. Design advice by Tom Lane and myself. Further review and cleanup by me.
2011-12-22 22:15:57 +01:00
READ_BOOL_FIELD(security_barrier);
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
break;
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
case RTE_JOIN:
READ_ENUM_FIELD(jointype, JoinType);
Reconsider the representation of join alias Vars. The core idea of this patch is to make the parser generate join alias Vars (that is, ones with varno pointing to a JOIN RTE) only when the alias Var is actually different from any raw join input, that is a type coercion and/or COALESCE is necessary to generate the join output value. Otherwise just generate varno/varattno pointing to the relevant join input column. In effect, this means that the planner's flatten_join_alias_vars() transformation is already done in the parser, for all cases except (a) columns that are merged by JOIN USING and are transformed in the process, and (b) whole-row join Vars. In principle that would allow us to skip doing flatten_join_alias_vars() in many more queries than we do now, but we don't have quite enough infrastructure to know that we can do so --- in particular there's no cheap way to know whether there are any whole-row join Vars. I'm not sure if it's worth the trouble to add a Query-level flag for that, and in any case it seems like fit material for a separate patch. But even without skipping the work entirely, this should make flatten_join_alias_vars() faster, particularly where there are nested joins that it previously had to flatten recursively. An essential part of this change is to replace Var nodes' varnoold/varoattno fields with varnosyn/varattnosyn, which have considerably more tightly-defined meanings than the old fields: when they differ from varno/varattno, they identify the Var's position in an aliased JOIN RTE, and the join alias is what ruleutils.c should print for the Var. This is necessary because the varno change destroyed ruleutils.c's ability to find the JOIN RTE from the Var's varno. Another way in which this change broke ruleutils.c is that it's no longer feasible to determine, from a JOIN RTE's joinaliasvars list, which join columns correspond to which columns of the join's immediate input relations. (If those are sub-joins, the joinaliasvars entries may point to columns of their base relations, not the sub-joins.) But that was a horrid mess requiring a lot of fragile assumptions already, so let's just bite the bullet and add some more JOIN RTE fields to make it more straightforward to figure that out. I added two integer-List fields containing the relevant column numbers from the left and right input rels, plus a count of how many merged columns there are. This patch depends on the ParseNamespaceColumn infrastructure that I added in commit 5815696bc. The biggest bit of code change is restructuring transformFromClauseItem's handling of JOINs so that the ParseNamespaceColumn data is propagated upward correctly. Other than that and the ruleutils fixes, everything pretty much just works, though some processing is now inessential. I grabbed two pieces of low-hanging fruit in that line: 1. In find_expr_references, we don't need to recurse into join alias Vars anymore. There aren't any except for references to merged USING columns, which are more properly handled when we scan the join's RTE. This change actually fixes an edge-case issue: we will now record a dependency on any type-coercion function present in a USING column's joinaliasvar, even if that join column has no references in the query text. The odds of the missing dependency causing a problem seem quite small: you'd have to posit somebody dropping an implicit cast between two data types, without removing the types themselves, and then having a stored rule containing a whole-row Var for a join whose USING merge depends on that cast. So I don't feel a great need to change this in the back branches. But in theory this way is more correct. 2. markRTEForSelectPriv and markTargetListOrigin don't need to recurse into join alias Vars either, because the cases they care about don't apply to alias Vars for USING columns that are semantically distinct from the underlying columns. This removes the only case in which markVarForSelectPriv could be called with NULL for the RTE, so adjust the comments to describe that hack as being strictly internal to markRTEForSelectPriv. catversion bump required due to changes in stored rules. Discussion: https://postgr.es/m/7115.1577986646@sss.pgh.pa.us
2020-01-09 17:56:59 +01:00
READ_INT_FIELD(joinmergedcols);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
READ_NODE_FIELD(joinaliasvars);
Reconsider the representation of join alias Vars. The core idea of this patch is to make the parser generate join alias Vars (that is, ones with varno pointing to a JOIN RTE) only when the alias Var is actually different from any raw join input, that is a type coercion and/or COALESCE is necessary to generate the join output value. Otherwise just generate varno/varattno pointing to the relevant join input column. In effect, this means that the planner's flatten_join_alias_vars() transformation is already done in the parser, for all cases except (a) columns that are merged by JOIN USING and are transformed in the process, and (b) whole-row join Vars. In principle that would allow us to skip doing flatten_join_alias_vars() in many more queries than we do now, but we don't have quite enough infrastructure to know that we can do so --- in particular there's no cheap way to know whether there are any whole-row join Vars. I'm not sure if it's worth the trouble to add a Query-level flag for that, and in any case it seems like fit material for a separate patch. But even without skipping the work entirely, this should make flatten_join_alias_vars() faster, particularly where there are nested joins that it previously had to flatten recursively. An essential part of this change is to replace Var nodes' varnoold/varoattno fields with varnosyn/varattnosyn, which have considerably more tightly-defined meanings than the old fields: when they differ from varno/varattno, they identify the Var's position in an aliased JOIN RTE, and the join alias is what ruleutils.c should print for the Var. This is necessary because the varno change destroyed ruleutils.c's ability to find the JOIN RTE from the Var's varno. Another way in which this change broke ruleutils.c is that it's no longer feasible to determine, from a JOIN RTE's joinaliasvars list, which join columns correspond to which columns of the join's immediate input relations. (If those are sub-joins, the joinaliasvars entries may point to columns of their base relations, not the sub-joins.) But that was a horrid mess requiring a lot of fragile assumptions already, so let's just bite the bullet and add some more JOIN RTE fields to make it more straightforward to figure that out. I added two integer-List fields containing the relevant column numbers from the left and right input rels, plus a count of how many merged columns there are. This patch depends on the ParseNamespaceColumn infrastructure that I added in commit 5815696bc. The biggest bit of code change is restructuring transformFromClauseItem's handling of JOINs so that the ParseNamespaceColumn data is propagated upward correctly. Other than that and the ruleutils fixes, everything pretty much just works, though some processing is now inessential. I grabbed two pieces of low-hanging fruit in that line: 1. In find_expr_references, we don't need to recurse into join alias Vars anymore. There aren't any except for references to merged USING columns, which are more properly handled when we scan the join's RTE. This change actually fixes an edge-case issue: we will now record a dependency on any type-coercion function present in a USING column's joinaliasvar, even if that join column has no references in the query text. The odds of the missing dependency causing a problem seem quite small: you'd have to posit somebody dropping an implicit cast between two data types, without removing the types themselves, and then having a stored rule containing a whole-row Var for a join whose USING merge depends on that cast. So I don't feel a great need to change this in the back branches. But in theory this way is more correct. 2. markRTEForSelectPriv and markTargetListOrigin don't need to recurse into join alias Vars either, because the cases they care about don't apply to alias Vars for USING columns that are semantically distinct from the underlying columns. This removes the only case in which markVarForSelectPriv could be called with NULL for the RTE, so adjust the comments to describe that hack as being strictly internal to markRTEForSelectPriv. catversion bump required due to changes in stored rules. Discussion: https://postgr.es/m/7115.1577986646@sss.pgh.pa.us
2020-01-09 17:56:59 +01:00
READ_NODE_FIELD(joinleftcols);
READ_NODE_FIELD(joinrightcols);
Allow an alias to be attached to a JOIN ... USING This allows something like SELECT ... FROM t1 JOIN t2 USING (a, b, c) AS x where x has the columns a, b, c and unlike a regular alias it does not hide the range variables of the tables being joined t1 and t2. Per SQL:2016 feature F404 "Range variable for common column names". Reviewed-by: Vik Fearing <vik.fearing@2ndquadrant.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://www.postgresql.org/message-id/flat/454638cf-d563-ab76-a585-2564428062af@2ndquadrant.com
2021-03-31 17:09:24 +02:00
READ_NODE_FIELD(join_using_alias);
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
break;
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
case RTE_FUNCTION:
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions. This patch adds the ability to write TABLE( function1(), function2(), ...) as a single FROM-clause entry. The result is the concatenation of the first row from each function, followed by the second row from each function, etc; with NULLs inserted if any function produces fewer rows than others. This is believed to be a much more useful behavior than what Postgres currently does with multiple SRFs in a SELECT list. This syntax also provides a reasonable way to combine use of column definition lists with WITH ORDINALITY: put the column definition list inside TABLE(), where it's clear that it doesn't control the ordinality column as well. Also implement SQL-compliant multiple-argument UNNEST(), by turning UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)). The SQL standard specifies TABLE() with only a single function, not multiple functions, and it seems to require an implicit UNNEST() which is not what this patch does. There may be something wrong with that reading of the spec, though, because if it's right then the spec's TABLE() is just a pointless alternative spelling of UNNEST(). After further review of that, we might choose to adopt a different syntax for what this patch does, but in any case this functionality seems clearly worthwhile. Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and significantly revised by me
2013-11-22 01:37:02 +01:00
READ_NODE_FIELD(functions);
Add SQL Standard WITH ORDINALITY support for UNNEST (and any other SRF) Author: Andrew Gierth, David Fetter Reviewers: Dean Rasheed, Jeevan Chalke, Stephen Frost
2013-07-29 17:38:01 +02:00
READ_BOOL_FIELD(funcordinality);
First pass at set-returning-functions in FROM, by Joe Conway with some kibitzing from Tom Lane. Not everything works yet, and there's no documentation or regression test, but let's commit this so Joe doesn't need to cope with tracking changes in so many files ...
2002-05-12 22:10:05 +02:00
break;
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
case RTE_TABLEFUNC:
READ_NODE_FIELD(tablefunc);
Fix some probably-minor oversights in readfuncs.c. The system expects TABLEFUNC RTEs to have coltypes, coltypmods, and colcollations lists, but outfuncs doesn't dump them and readfuncs doesn't restore them. This doesn't cause obvious failures, because the only things that look at those fields are expandRTE() and get_rte_attribute_type(), which are mostly used during parse analysis, before anything would've passed the parsetree through outfuncs/readfuncs. But expandRTE() is used in build_physical_tlist(), which means that that function will return a wrong answer for a TABLEFUNC RTE that came from a view. Very accidentally, this doesn't cause serious problems, because what it will return is NIL which callers will interpret as "couldn't build a physical tlist because of dropped columns". So you still get a plan that works, though it's marginally less efficient than it could be. There are also some other expandRTE() calls associated with transformation of whole-row Vars in the planner. I have been unable to exhibit misbehavior from that, and it may be unreachable in any case that anyone would care about ... but I'm not entirely convinced, so this seems like something we should back- patch a fix for. Fortunately, we can fix it without forcing a change of stored rules and a catversion bump, because we can just copy these lists from the subsidiary TableFunc object. readfuncs.c was also missing support for NamedTuplestoreScan plan nodes. This accidentally fails to break parallel query because a query using a named tuplestore would never be considered parallel-safe anyway. However, project policy since parallel query came in is that all plan node types should have outfuncs/readfuncs support, so this is clearly an oversight that should be repaired. Noted while fooling around with a patch to test outfuncs/readfuncs more thoroughly. That exposed some other issues too, but these are the only ones that seem worth back-patching. Back-patch to v10 where both of these features came in. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 19:02:27 +02:00
/* The RTE must have a copy of the column type info, if any */
if (local_node->tablefunc)
{
TableFunc *tf = local_node->tablefunc;
local_node->coltypes = tf->coltypes;
local_node->coltypmods = tf->coltypmods;
local_node->colcollations = tf->colcollations;
}
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
break;
Add support for multi-row VALUES clauses as part of INSERT statements (e.g. "INSERT ... VALUES (...), (...), ...") and elsewhere as allowed by the spec. (e.g. similar to a FROM clause subselect). initdb required. Joe Conway and Tom Lane.
2006-08-02 03:59:48 +02:00
case RTE_VALUES:
READ_NODE_FIELD(values_lists);
Fix reporting of column typmods for multi-row VALUES constructs. expandRTE() and get_rte_attribute_type() reported the exprType() and exprTypmod() values of the expressions in the first row of the VALUES as being the column type/typmod returned by the VALUES RTE. That's fine for the data type, since we coerce all expressions in a column to have the same common type. But we don't coerce them to have a common typmod, so it was possible for rows after the first one to return values that violate the claimed column typmod. This leads to the incorrect result seen in bug #14448 from Hassan Mahmood, as well as some other corner-case misbehaviors. The desired behavior is the same as we use in other type-unification cases: report the common typmod if there is one, but otherwise return -1 indicating no particular constraint. It's cheap for transformValuesClause to determine the common typmod while transforming a multi-row VALUES, but it'd be less cheap for expandRTE() and get_rte_attribute_type() to re-determine that info every time they're asked --- possibly a lot less cheap, if the VALUES has many rows. Therefore, the best fix is to record the common typmods explicitly in a list in the VALUES RTE, as we were already doing for column collations. This looks quite a bit like what we're doing for CTE RTEs, so we can save a little bit of space and code by unifying the representation for those two RTE types. They both now share coltypes/coltypmods/colcollations fields. (At some point it might seem desirable to populate those fields for all RTE types; but right now it looks like constructing them for other RTE types would add more code and cycles than it would save.) The RTE change requires a catversion bump, so this fix is only usable in HEAD. If we fix this at all in the back branches, the patch will need to look quite different. Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
READ_NODE_FIELD(coltypes);
READ_NODE_FIELD(coltypmods);
READ_NODE_FIELD(colcollations);
Add support for multi-row VALUES clauses as part of INSERT statements (e.g. "INSERT ... VALUES (...), (...), ...") and elsewhere as allowed by the spec. (e.g. similar to a FROM clause subselect). initdb required. Joe Conway and Tom Lane.
2006-08-02 03:59:48 +02:00
break;
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
case RTE_CTE:
READ_STRING_FIELD(ctename);
READ_UINT_FIELD(ctelevelsup);
READ_BOOL_FIELD(self_reference);
Fix reporting of column typmods for multi-row VALUES constructs. expandRTE() and get_rte_attribute_type() reported the exprType() and exprTypmod() values of the expressions in the first row of the VALUES as being the column type/typmod returned by the VALUES RTE. That's fine for the data type, since we coerce all expressions in a column to have the same common type. But we don't coerce them to have a common typmod, so it was possible for rows after the first one to return values that violate the claimed column typmod. This leads to the incorrect result seen in bug #14448 from Hassan Mahmood, as well as some other corner-case misbehaviors. The desired behavior is the same as we use in other type-unification cases: report the common typmod if there is one, but otherwise return -1 indicating no particular constraint. It's cheap for transformValuesClause to determine the common typmod while transforming a multi-row VALUES, but it'd be less cheap for expandRTE() and get_rte_attribute_type() to re-determine that info every time they're asked --- possibly a lot less cheap, if the VALUES has many rows. Therefore, the best fix is to record the common typmods explicitly in a list in the VALUES RTE, as we were already doing for column collations. This looks quite a bit like what we're doing for CTE RTEs, so we can save a little bit of space and code by unifying the representation for those two RTE types. They both now share coltypes/coltypmods/colcollations fields. (At some point it might seem desirable to populate those fields for all RTE types; but right now it looks like constructing them for other RTE types would add more code and cycles than it would save.) The RTE change requires a catversion bump, so this fix is only usable in HEAD. If we fix this at all in the back branches, the patch will need to look quite different. Report: https://postgr.es/m/20161205143037.4377.60754@wrigleys.postgresql.org Discussion: https://postgr.es/m/27429.1480968538@sss.pgh.pa.us
2016-12-08 17:40:02 +01:00
READ_NODE_FIELD(coltypes);
READ_NODE_FIELD(coltypmods);
READ_NODE_FIELD(colcollations);
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
break;
Add infrastructure to support EphemeralNamedRelation references. A QueryEnvironment concept is added, which allows new types of objects to be passed into queries from parsing on through execution. At this point, the only thing implemented is a collection of EphemeralNamedRelation objects -- relations which can be referenced by name in queries, but do not exist in the catalogs. The only type of ENR implemented is NamedTuplestore, but provision is made to add more types fairly easily. An ENR can carry its own TupleDesc or reference a relation in the catalogs by relid. Although these features can be used without SPI, convenience functions are added to SPI so that ENRs can easily be used by code run through SPI. The initial use of all this is going to be transition tables in AFTER triggers, but that will be added to each PL as a separate commit. An incidental effect of this patch is to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
2017-04-01 06:17:18 +02:00
case RTE_NAMEDTUPLESTORE:
READ_STRING_FIELD(enrname);
Fix problems related to RangeTblEntry members enrname and enrtuples. Commit 18ce3a4ab22d2984f8540ab480979c851dae5338 failed to update the comments in parsenodes.h for the new members, and made only incomplete updates to src/backend/nodes Thomas Munro, per a report from Noah Misch. Discussion: http://postgr.es/m/20170611062525.GA1628882@rfd.leadboat.com
2017-06-14 22:19:46 +02:00
READ_FLOAT_FIELD(enrtuples);
Add infrastructure to support EphemeralNamedRelation references. A QueryEnvironment concept is added, which allows new types of objects to be passed into queries from parsing on through execution. At this point, the only thing implemented is a collection of EphemeralNamedRelation objects -- relations which can be referenced by name in queries, but do not exist in the catalogs. The only type of ENR implemented is NamedTuplestore, but provision is made to add more types fairly easily. An ENR can carry its own TupleDesc or reference a relation in the catalogs by relid. Although these features can be used without SPI, convenience functions are added to SPI so that ENRs can easily be used by code run through SPI. The initial use of all this is going to be transition tables in AFTER triggers, but that will be added to each PL as a separate commit. An incidental effect of this patch is to produce a more informative error message if an attempt is made to modify the contents of a CTE from a referencing DML statement. No tests previously covered that possibility, so one is added. Kevin Grittner and Thomas Munro Reviewed by Heikki Linnakangas, David Fetter, and Thomas Munro with valuable comments and suggestions from many others
2017-04-01 06:17:18 +02:00
READ_OID_FIELD(relid);
READ_NODE_FIELD(coltypes);
READ_NODE_FIELD(coltypmods);
READ_NODE_FIELD(colcollations);
break;
In the planner, replace an empty FROM clause with a dummy RTE. The fact that "SELECT expression" has no base relations has long been a thorn in the side of the planner. It makes it hard to flatten a sub-query that looks like that, or is a trivial VALUES() item, because the planner generally uses relid sets to identify sub-relations, and such a sub-query would have an empty relid set if we flattened it. prepjointree.c contains some baroque logic that works around this in certain special cases --- but there is a much better answer. We can replace an empty FROM clause with a dummy RTE that acts like a table of one row and no columns, and then there are no such corner cases to worry about. Instead we need some logic to get rid of useless dummy RTEs, but that's simpler and covers more cases than what was there before. For really trivial cases, where the query is just "SELECT expression" and nothing else, there's a hazard that adding the extra RTE makes for a noticeable slowdown; even though it's not much processing, there's not that much for the planner to do overall. However testing says that the penalty is very small, close to the noise level. In more complex queries, this is able to find optimizations that we could not find before. The new RTE type is called RTE_RESULT, since the "scan" plan type it gives rise to is a Result node (the same plan we produced for a "SELECT expression" query before). To avoid confusion, rename the old ResultPath path type to GroupResultPath, reflecting that it's only used in degenerate grouping cases where we know the query produces just one grouped row. (It wouldn't work to unify the two cases, because there are different rules about where the associated quals live during query_planner.) Note: although this touches readfuncs.c, I don't think a catversion bump is required, because the added case can't occur in stored rules, only plans. Patch by me, reviewed by David Rowley and Mark Dilger Discussion: https://postgr.es/m/15944.1521127664@sss.pgh.pa.us
2019-01-28 23:54:10 +01:00
case RTE_RESULT:
/* no extra fields */
break;
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
default:
Error message editing in backend/bootstrap, /lib, /nodes, /port.
2003-07-23 01:30:39 +02:00
elog(ERROR, "unrecognized RTE kind: %d",
(int) local_node->rtekind);
A little further progress on schemas: push down RangeVars into addRangeTableEntry calls. Remove relname field from RTEs, since it will no longer be a useful unique identifier of relations; we want to encourage people to rely on the relation OID instead. Further work on dumping qual expressions in EXPLAIN, too.
2002-03-22 03:56:37 +01:00
break;
}
Implement SQL-standard LATERAL subqueries. This patch implements the standard syntax of LATERAL attached to a sub-SELECT in FROM, and also allows LATERAL attached to a function in FROM, since set-returning function calls are expected to be one of the principal use-cases. The main change here is a rewrite of the mechanism for keeping track of which relations are visible for column references while the FROM clause is being scanned. The parser "namespace" lists are no longer lists of bare RTEs, but are lists of ParseNamespaceItem structs, which carry an RTE pointer as well as some visibility-controlling flags. Aside from supporting LATERAL correctly, this lets us get rid of the ancient hacks that required rechecking subqueries and JOIN/ON and function-in-FROM expressions for invalid references after they were initially parsed. Invalid column references are now always correctly detected on sight. In passing, remove assorted parser error checks that are now dead code by virtue of our having gotten rid of add_missing_from, as well as some comments that are obsolete for the same reason. (It was mainly add_missing_from that caused so much fudging here in the first place.) The planner support for this feature is very minimal, and will be improved in future patches. It works well enough for testing purposes, though. catversion bump forced due to new field in RangeTblEntry.
2012-08-08 01:02:54 +02:00
READ_BOOL_FIELD(lateral);
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_BOOL_FIELD(inh);
READ_BOOL_FIELD(inFromCl);
Fix permission-checking bug reported by Tim Burgess 10-Feb-03 (this time for sure...). Rather than relying on the query context of a rangetable entry to identify what permissions it wants checked, store a full AclMode mask in each RTE, and check exactly those bits. This allows an RTE specifying, say, INSERT privilege on a view to be copied into a derived UPDATE query without changing meaning. Per recent discussion thread. initdb forced due to change of stored rule representation.
2004-01-15 00:01:55 +01:00
READ_UINT_FIELD(requiredPerms);
Replace pg_shadow and pg_group by new role-capable catalogs pg_authid and pg_auth_members. There are still many loose ends to finish in this patch (no documentation, no regression tests, no pg_dump support for instance). But I'm going to commit it now anyway so that Alvaro can make some progress on shared dependencies. The catalog changes should be pretty much done.
2005-06-28 07:09:14 +02:00
READ_OID_FIELD(checkAsUser);
Support column-level privileges, as required by SQL standard. Stephen Frost, with help from KaiGai Kohei and others
2009-01-22 21:16:10 +01:00
READ_BITMAPSET_FIELD(selectedCols);
Represent columns requiring insert and update privileges indentently. Previously, relation range table entries used a single Bitmapset field representing which columns required either UPDATE or INSERT privileges, despite the fact that INSERT and UPDATE privileges are separately cataloged, and may be independently held. As statements so far required either insert or update privileges but never both, that was sufficient. The required permission could be inferred from the top level statement run. The upcoming INSERT ... ON CONFLICT UPDATE feature needs to independently check for both privileges in one statement though, so that is not sufficient anymore. Bumps catversion as stored rules change. Author: Peter Geoghegan Reviewed-By: Andres Freund
2015-05-08 00:20:46 +02:00
READ_BITMAPSET_FIELD(insertedCols);
READ_BITMAPSET_FIELD(updatedCols);
Generated columns This is an SQL-standard feature that allows creating columns that are computed from expressions rather than assigned, similar to a view or materialized view but on a column basis. This implements one kind of generated column: stored (computed on write). Another kind, virtual (computed on read), is planned for the future, and some room is left for it. Reviewed-by: Michael Paquier <michael@paquier.xyz> Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/b151f851-4019-bdb1-699e-ebab07d2f40a@2ndquadrant.com
2019-03-30 08:13:09 +01:00
READ_BITMAPSET_FIELD(extraUpdatedCols);
Make security barrier views automatically updatable Views which are marked as security_barrier must have their quals applied before any user-defined quals are called, to prevent user-defined functions from being able to see rows which the security barrier view is intended to prevent them from seeing. Remove the restriction on security barrier views being automatically updatable by adding a new securityQuals list to the RTE structure which keeps track of the quals from security barrier views at each level, independently of the user-supplied quals. When RTEs are later discovered which have securityQuals populated, they are turned into subquery RTEs which are marked as security_barrier to prevent any user-supplied quals being pushed down (modulo LEAKPROOF quals). Dean Rasheed, reviewed by Craig Ringer, Simon Riggs, KaiGai Kohei
2014-04-13 03:04:58 +02:00
READ_NODE_FIELD(securityQuals);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_DONE();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions. This patch adds the ability to write TABLE( function1(), function2(), ...) as a single FROM-clause entry. The result is the concatenation of the first row from each function, followed by the second row from each function, etc; with NULLs inserted if any function produces fewer rows than others. This is believed to be a much more useful behavior than what Postgres currently does with multiple SRFs in a SELECT list. This syntax also provides a reasonable way to combine use of column definition lists with WITH ORDINALITY: put the column definition list inside TABLE(), where it's clear that it doesn't control the ordinality column as well. Also implement SQL-compliant multiple-argument UNNEST(), by turning UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)). The SQL standard specifies TABLE() with only a single function, not multiple functions, and it seems to require an implicit UNNEST() which is not what this patch does. There may be something wrong with that reading of the spec, though, because if it's right then the spec's TABLE() is just a pointless alternative spelling of UNNEST(). After further review of that, we might choose to adopt a different syntax for what this patch does, but in any case this functionality seems clearly worthwhile. Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and significantly revised by me
2013-11-22 01:37:02 +01:00
/*
* _readRangeTblFunction
*/
static RangeTblFunction *
_readRangeTblFunction(void)
{
READ_LOCALS(RangeTblFunction);
READ_NODE_FIELD(funcexpr);
READ_INT_FIELD(funccolcount);
READ_NODE_FIELD(funccolnames);
READ_NODE_FIELD(funccoltypes);
READ_NODE_FIELD(funccoltypmods);
READ_NODE_FIELD(funccolcollations);
READ_BITMAPSET_FIELD(funcparams);
READ_DONE();
}
Redesign tablesample method API, and do extensive code review. The original implementation of TABLESAMPLE modeled the tablesample method API on index access methods, which wasn't a good choice because, without specialized DDL commands, there's no way to build an extension that can implement a TSM. (Raw inserts into system catalogs are not an acceptable thing to do, because we can't undo them during DROP EXTENSION, nor will pg_upgrade behave sanely.) Instead adopt an API more like procedural language handlers or foreign data wrappers, wherein the only SQL-level support object needed is a single handler function identified by having a special return type. This lets us get rid of the supporting catalog altogether, so that no custom DDL support is needed for the feature. Adjust the API so that it can support non-constant tablesample arguments (the original coding assumed we could evaluate the argument expressions at ExecInitSampleScan time, which is undesirable even if it weren't outright unsafe), and discourage sampling methods from looking at invisible tuples. Make sure that the BERNOULLI and SYSTEM methods are genuinely repeatable within and across queries, as required by the SQL standard, and deal more honestly with methods that can't support that requirement. Make a full code-review pass over the tablesample additions, and fix assorted bugs, omissions, infelicities, and cosmetic issues (such as failure to put the added code stanzas in a consistent ordering). Improve EXPLAIN's output of tablesample plans, too. Back-patch to 9.5 so that we don't have to support the original API in production.
2015-07-25 20:39:00 +02:00
/*
* _readTableSampleClause
*/
static TableSampleClause *
_readTableSampleClause(void)
{
READ_LOCALS(TableSampleClause);
READ_OID_FIELD(tsmhandler);
READ_NODE_FIELD(args);
READ_NODE_FIELD(repeatable);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readDefElem
*/
static DefElem *
_readDefElem(void)
{
READ_LOCALS(DefElem);
READ_STRING_FIELD(defnamespace);
READ_STRING_FIELD(defname);
READ_NODE_FIELD(arg);
READ_ENUM_FIELD(defaction, DefElemAction);
Add location field to DefElem Add a location field to the DefElem struct, used to parse many utility commands. Update various error messages to supply error position information. To propogate the error position information in a more systematic way, create a ParseState in standard_ProcessUtility() and pass that to interested functions implementing the utility commands. This seems better than passing the query string and then reassembling a parse state ad hoc, which violates the encapsulation of the ParseState type. Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com>
2016-09-06 18:00:00 +02:00
READ_LOCATION_FIELD(location);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
/*
* Stuff from plannodes.h.
*/
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readPlannedStmt
*/
static PlannedStmt *
_readPlannedStmt(void)
{
READ_LOCALS(PlannedStmt);
READ_ENUM_FIELD(commandType, CmdType);
pg_stat_statements: Widen query IDs from 32 bits to 64 bits. This takes advantage of the infrastructure introduced by commit 81c5e46c490e2426db243eada186995da5bb0ba7 to greatly reduce the likelihood that two different queries will end up with the same query ID. It's still possible, of course, but whereas before it the chances of a collision reached 25% around 50,000 queries, it will now take more than 3 billion queries. Backward incompatibility: Because the type exposed at the SQL level is int8, users may now see negative query IDs in the pg_stat_statements view (and also, query IDs more than 4 billion, which was the old limit). Patch by me, reviewed by Michael Paquier and Peter Geoghegan. Discussion: http://postgr.es/m/CA+TgmobG_Kp4cBKFmsznUAaM1GWW6hhRNiZC0KjRMOOeYnz5Yw@mail.gmail.com
2017-10-12 01:52:46 +02:00
READ_UINT64_FIELD(queryId);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_BOOL_FIELD(hasReturning);
READ_BOOL_FIELD(hasModifyingCTE);
READ_BOOL_FIELD(canSetTag);
READ_BOOL_FIELD(transientPlan);
Avoid invalidating all foreign-join cached plans when user mappings change. We must not push down a foreign join when the foreign tables involved should be accessed under different user mappings. Previously we tried to enforce that rule literally during planning, but that meant that the resulting plans were dependent on the current contents of the pg_user_mapping catalog, and we had to blow away all cached plans containing any remote join when anything at all changed in pg_user_mapping. This could have been improved somewhat, but the fact that a syscache inval callback has very limited info about what changed made it hard to do better within that design. Instead, let's change the planner to not consider user mappings per se, but to allow a foreign join if both RTEs have the same checkAsUser value. If they do, then they necessarily will use the same user mapping at runtime, and we don't need to know specifically which one that is. Post-plan-time changes in pg_user_mapping no longer require any plan invalidation. This rule does give up some optimization ability, to wit where two foreign table references come from views with different owners or one's from a view and one's directly in the query, but nonetheless the same user mapping would have applied. We'll sacrifice the first case, but to not regress more than we have to in the second case, allow a foreign join involving both zero and nonzero checkAsUser values if the nonzero one is the same as the prevailing effective userID. In that case, mark the plan as only runnable by that userID. The plancache code already had a notion of plans being userID-specific, in order to support RLS. It was a little confused though, in particular lacking clarity of thought as to whether it was the rewritten query or just the finished plan that's dependent on the userID. Rearrange that code so that it's clearer what depends on which, and so that the same logic applies to both RLS-injected role dependency and foreign-join-injected role dependency. Note that this patch doesn't remove the other issue mentioned in the original complaint, which is that while we'll reliably stop using a foreign join if it's disallowed in a new context, we might fail to start using a foreign join if it's now allowed, but we previously created a generic cached plan that didn't use one. It was agreed that the chance of winning that way was not high enough to justify the much larger number of plan invalidations that would have to occur if we tried to cause it to happen. In passing, clean up randomly-varying spelling of EXPLAIN commands in postgres_fdw.sql, and fix a COSTS ON example that had been allowed to leak into the committed tests. This reverts most of commits fbe5a3fb7 and 5d4171d1c, which were the previous attempt at ensuring we wouldn't push down foreign joins that span permissions contexts. Etsuro Fujita and Tom Lane Discussion: <d49c1e5b-f059-20f4-c132-e9752ee0113e@lab.ntt.co.jp>
2016-07-15 23:22:56 +02:00
READ_BOOL_FIELD(dependsOnRole);
READ_BOOL_FIELD(parallelModeNeeded);
Fix incorrect field type for PlannedStmt.jitFlags in outfuncs/readfuncs. This field was a bool at one point, but now it's an int. Spotted by Hari Babu; trivial patch is by Ashutosh Bapat. Discussion: https://postgr.es/m/CAJrrPGedKiFE2fqntSauUfhapCksOJzam+QtHfSgx86LhXLeOQ@mail.gmail.com
2018-04-28 22:46:24 +02:00
READ_INT_FIELD(jitFlags);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(planTree);
READ_NODE_FIELD(rtable);
READ_NODE_FIELD(resultRelations);
Further adjust EXPLAIN's choices of table alias names. This patch causes EXPLAIN to always assign a separate table alias to the parent RTE of an append relation (inheritance set); before, such RTEs were ignored if not actually scanned by the plan. Since the child RTEs now always have that same alias to start with (cf. commit 55a1954da), the net effect is that the parent RTE usually gets the alias used or implied by the query text, and the children all get that alias with "_N" appended. (The exception to "usually" is if there are duplicate aliases in different subtrees of the original query; then some of those original RTEs will also have "_N" appended.) This results in more uniform output for partitioned-table plans than we had before: the partitioned table itself gets the original alias, and all child tables have aliases with "_N", rather than the previous behavior where one of the children would get an alias without "_N". The reason for giving the parent RTE an alias, even if it isn't scanned by the plan, is that we now use the parent's alias to qualify Vars that refer to an appendrel output column and appear above the Append or MergeAppend that computes the appendrel. But below the append, Vars refer to some one of the child relations, and are displayed that way. This seems clearer than the old behavior where a Var that could carry values from any child relation was displayed as if it referred to only one of them. While at it, change ruleutils.c so that the code paths used by EXPLAIN deal in Plan trees not PlanState trees. This effectively reverts a decision made in commit 1cc29fe7c, which seemed like a good idea at the time to make ruleutils.c consistent with explain.c. However, it's problematic because we'd really like to allow executor startup pruning to remove all the children of an append node when possible, leaving no child PlanState to resolve Vars against. (That's not done here, but will be in the next patch.) This requires different handling of subplans and initplans than before, but is otherwise a pretty straightforward change. Discussion: https://postgr.es/m/001001d4f44b$2a2cca50$7e865ef0$@lab.ntt.co.jp
2019-12-11 23:05:18 +01:00
READ_NODE_FIELD(appendRelations);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(subplans);
READ_BITMAPSET_FIELD(rewindPlanIDs);
READ_NODE_FIELD(rowMarks);
READ_NODE_FIELD(relationOids);
READ_NODE_FIELD(invalItems);
Track in the plan the types associated with PARAM_EXEC parameters. Up until now, we only tracked the number of parameters, which was sufficient to allocate an array of Datums of the appropriate size, but not sufficient to, for example, know how to serialize a Datum stored in one of those slots. An upcoming patch wants to do that, so add this tracking to make it possible. Patch by me, reviewed by Tom Lane and Amit Kapila. Discussion: http://postgr.es/m/CA+TgmoYqpxDKn8koHdW8BEKk8FMUL0=e8m2Qe=M+r0UBjr3tuQ@mail.gmail.com
2017-11-13 21:24:12 +01:00
READ_NODE_FIELD(paramExecTypes);
Change representation of statement lists, and add statement location info. This patch makes several changes that improve the consistency of representation of lists of statements. It's always been the case that the output of parse analysis is a list of Query nodes, whatever the types of the individual statements in the list. This patch brings similar consistency to the outputs of raw parsing and planning steps: * The output of raw parsing is now always a list of RawStmt nodes; the statement-type-dependent nodes are one level down from that. * The output of pg_plan_queries() is now always a list of PlannedStmt nodes, even for utility statements. In the case of a utility statement, "planning" just consists of wrapping a CMD_UTILITY PlannedStmt around the utility node. This list representation is now used in Portal and CachedPlan plan lists, replacing the former convention of intermixing PlannedStmts with bare utility-statement nodes. Now, every list of statements has a consistent head-node type depending on how far along it is in processing. This allows changing many places that formerly used generic "Node *" pointers to use a more specific pointer type, thus reducing the number of IsA() tests and casts needed, as well as improving code clarity. Also, the post-parse-analysis representation of DECLARE CURSOR is changed so that it looks more like EXPLAIN, PREPARE, etc. That is, the contained SELECT remains a child of the DeclareCursorStmt rather than getting flipped around to be the other way. It's now true for both Query and PlannedStmt that utilityStmt is non-null if and only if commandType is CMD_UTILITY. That allows simplifying a lot of places that were testing both fields. (I think some of those were just defensive programming, but in many places, it was actually necessary to avoid confusing DECLARE CURSOR with SELECT.) Because PlannedStmt carries a canSetTag field, we're also able to get rid of some ad-hoc rules about how to reconstruct canSetTag for a bare utility statement; specifically, the assumption that a utility is canSetTag if and only if it's the only one in its list. While I see no near-term need for relaxing that restriction, it's nice to get rid of the ad-hocery. The API of ProcessUtility() is changed so that what it's passed is the wrapper PlannedStmt not just the bare utility statement. This will affect all users of ProcessUtility_hook, but the changes are pretty trivial; see the affected contrib modules for examples of the minimum change needed. (Most compilers should give pointer-type-mismatch warnings for uncorrected code.) There's also a change in the API of ExplainOneQuery_hook, to pass through cursorOptions instead of expecting hook functions to know what to pick. This is needed because of the DECLARE CURSOR changes, but really should have been done in 9.6; it's unlikely that any extant hook functions know about using CURSOR_OPT_PARALLEL_OK. Finally, teach gram.y to save statement boundary locations in RawStmt nodes, and pass those through to Query and PlannedStmt nodes. This allows more intelligent handling of cases where a source query string contains multiple statements. This patch doesn't actually do anything with the information, but a follow-on patch will. (Passing this information through cleanly is the true motivation for these changes; while I think this is all good cleanup, it's unlikely we'd have bothered without this end goal.) catversion bump because addition of location fields to struct Query affects stored rules. This patch is by me, but it owes a good deal to Fabien Coelho who did a lot of preliminary work on the problem, and also reviewed the patch. Discussion: https://postgr.es/m/alpine.DEB.2.20.1612200926310.29821@lancre
2017-01-14 22:02:35 +01:00
READ_NODE_FIELD(utilityStmt);
READ_LOCATION_FIELD(stmt_location);
Reconcile nodes/*funcs.c. The stmt_len changes do not affect behavior. LimitPath has no other support functions, so that part changes only debugging output.
2020-05-26 01:23:48 +02:00
READ_INT_FIELD(stmt_len);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* ReadCommonPlan
* Assign the basic stuff of all nodes that inherit from Plan
*/
static void
ReadCommonPlan(Plan *local_node)
{
READ_TEMP_LOCALS();
READ_FLOAT_FIELD(startup_cost);
READ_FLOAT_FIELD(total_cost);
READ_FLOAT_FIELD(plan_rows);
READ_INT_FIELD(plan_width);
Make sequential scans parallel-aware. In addition, this path fills in a number of missing bits and pieces in the parallel infrastructure. Paths and plans now have a parallel_aware flag indicating whether whatever parallel-aware logic they have should be engaged. It is believed that we will need this flag for a number of path/plan types, not just sequential scans, which is why the flag is generic rather than part of the SeqScan structures specifically. Also, execParallel.c now gives parallel nodes a chance to initialize their PlanState nodes from the DSM during parallel worker startup. Amit Kapila, with a fair amount of adjustment by me. Review of previous patch versions by Haribabu Kommi and others.
2015-11-11 14:57:52 +01:00
READ_BOOL_FIELD(parallel_aware);
Mark finished Plan nodes with parallel_safe flags. We'd managed to avoid doing this so far, but it seems pretty obvious that it would be forced on us some day, and this is much the cleanest way of approaching the open problem that parallel-unsafe subplans are being transmitted to parallel workers. Anyway there's no space cost due to alignment considerations, and the time cost is pretty minimal since we're just copying the flag from the corresponding Path node. (At least in most cases ... some of the klugier spots in createplan.c have to work a bit harder.) In principle we could perhaps get rid of SubPlan.parallel_safe, but I thought it better to keep that in case there are reasons to consider a SubPlan unsafe even when its child plan is parallel-safe. This patch doesn't actually do anything with the new flags, but I thought I'd commit it separately anyway. Note: although this touches outfuncs/readfuncs, there's no need for a catversion bump because Plan trees aren't stored on disk. Discussion: https://postgr.es/m/87tw5x4vcu.fsf@credativ.de
2017-04-12 21:13:23 +02:00
READ_BOOL_FIELD(parallel_safe);
Add support for asynchronous execution. This implements asynchronous execution, which runs multiple parts of a non-parallel-aware Append concurrently rather than serially to improve performance when possible. Currently, the only node type that can be run concurrently is a ForeignScan that is an immediate child of such an Append. In the case where such ForeignScans access data on different remote servers, this would run those ForeignScans concurrently, and overlap the remote operations to be performed simultaneously, so it'll improve the performance especially when the operations involve time-consuming ones such as remote join and remote aggregation. We may extend this to other node types such as joins or aggregates over ForeignScans in the future. This also adds the support for postgres_fdw, which is enabled by the table-level/server-level option "async_capable". The default is false. Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit is mostly based on the patch proposed by Robert Haas, but also uses stuff from the patch proposed by Kyotaro Horiguchi and from the patch proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and others. Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
READ_BOOL_FIELD(async_capable);
Parallel executor support. This code provides infrastructure for a parallel leader to start up parallel workers to execute subtrees of the plan tree being executed in the master. User-supplied parameters from ParamListInfo are passed down, but PARAM_EXEC parameters are not. Various other constructs, such as initplans, subplans, and CTEs, are also not currently shared. Nevertheless, there's enough here to support a basic implementation of parallel query, and we can lift some of the current restrictions as needed. Amit Kapila and Robert Haas
2015-09-29 03:55:57 +02:00
READ_INT_FIELD(plan_node_id);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(targetlist);
READ_NODE_FIELD(qual);
READ_NODE_FIELD(lefttree);
READ_NODE_FIELD(righttree);
READ_NODE_FIELD(initPlan);
READ_BITMAPSET_FIELD(extParam);
READ_BITMAPSET_FIELD(allParam);
}
/*
* _readResult
*/
static Result *
_readResult(void)
{
READ_LOCALS(Result);
ReadCommonPlan(&local_node->plan);
READ_NODE_FIELD(resconstantqual);
READ_DONE();
}
Move targetlist SRF handling from expression evaluation to new executor node. Evaluation of set returning functions (SRFs_ in the targetlist (like SELECT generate_series(1,5)) so far was done in the expression evaluation (i.e. ExecEvalExpr()) and projection (i.e. ExecProject/ExecTargetList) code. This meant that most executor nodes performing projection, and most expression evaluation functions, had to deal with the possibility that an evaluated expression could return a set of return values. That's bad because it leads to repeated code in a lot of places. It also, and that's my (Andres's) motivation, made it a lot harder to implement a more efficient way of doing expression evaluation. To fix this, introduce a new executor node (ProjectSet) that can evaluate targetlists containing one or more SRFs. To avoid the complexity of the old way of handling nested expressions returning sets (e.g. having to pass up ExprDoneCond, and dealing with arguments to functions returning sets etc.), those SRFs can only be at the top level of the node's targetlist. The planner makes sure (via split_pathtarget_at_srfs()) that SRF evaluation is only necessary in ProjectSet nodes and that SRFs are only present at the top level of the node's targetlist. If there are nested SRFs the planner creates multiple stacked ProjectSet nodes. The ProjectSet nodes always get input from an underlying node. We also discussed and prototyped evaluating targetlist SRFs using ROWS FROM(), but that turned out to be more complicated than we'd hoped. While moving SRF evaluation to ProjectSet would allow to retain the old "least common multiple" behavior when multiple SRFs are present in one targetlist (i.e. continue returning rows until all SRFs are at the end of their input at the same time), we decided to instead only return rows till all SRFs are exhausted, returning NULL for already exhausted ones. We deemed the previous behavior to be too confusing, unexpected and actually not particularly useful. As a side effect, the previously prohibited case of multiple set returning arguments to a function, is now allowed. Not because it's particularly desirable, but because it ends up working and there seems to be no argument for adding code to prohibit it. Currently the behavior for COALESCE and CASE containing SRFs has changed, returning multiple rows from the expression, even when the SRF containing "arm" of the expression is not evaluated. That's because the SRFs are evaluated in a separate ProjectSet node. As that's quite confusing, we're likely to instead prohibit SRFs in those places. But that's still being discussed, and the code would reside in places not touched here, so that's a task for later. There's a lot of, now superfluous, code dealing with set return expressions around. But as the changes to get rid of those are verbose largely boring, it seems better for readability to keep the cleanup as a separate commit. Author: Tom Lane and Andres Freund Discussion: https://postgr.es/m/20160822214023.aaxz5l4igypowyri@alap3.anarazel.de
2017-01-18 21:46:50 +01:00
/*
* _readProjectSet
*/
static ProjectSet *
_readProjectSet(void)
{
READ_LOCALS_NO_FIELDS(ProjectSet);
ReadCommonPlan(&local_node->plan);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readModifyTable
*/
static ModifyTable *
_readModifyTable(void)
{
READ_LOCALS(ModifyTable);
ReadCommonPlan(&local_node->plan);
READ_ENUM_FIELD(operation, CmdType);
READ_BOOL_FIELD(canSetTag);
READ_UINT_FIELD(nominalRelation);
Remove some unnecessary fields from Plan trees. In the wake of commit f2343653f, we no longer need some fields that were used before to control executor lock acquisitions: * PlannedStmt.nonleafResultRelations can go away entirely. * partitioned_rels can go away from Append, MergeAppend, and ModifyTable. However, ModifyTable still needs to know the RT index of the partition root table if any, which was formerly kept in the first entry of that list. Add a new field "rootRelation" to remember that. rootRelation is partly redundant with nominalRelation, in that if it's set it will have the same value as nominalRelation. However, the latter field has a different purpose so it seems best to keep them distinct. Amit Langote, reviewed by David Rowley and Jesper Pedersen, and whacked around a bit more by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp
2018-10-07 20:33:17 +02:00
READ_UINT_FIELD(rootRelation);
Allow UPDATE to move rows between partitions. When an UPDATE causes a row to no longer match the partition constraint, try to move it to a different partition where it does match the partition constraint. In essence, the UPDATE is split into a DELETE from the old partition and an INSERT into the new one. This can lead to surprising behavior in concurrency scenarios because EvalPlanQual rechecks won't work as they normally did; the known problems are documented. (There is a pending patch to improve the situation further, but it needs more review.) Amit Khandekar, reviewed and tested by Amit Langote, David Rowley, Rajkumar Raghuwanshi, Dilip Kumar, Amul Sul, Thomas Munro, Álvaro Herrera, Amit Kapila, and me. A few final revisions by me. Discussion: http://postgr.es/m/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb=yJO3u3QhyP8EEPQ@mail.gmail.com
2018-01-19 21:33:06 +01:00
READ_BOOL_FIELD(partColsUpdated);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(resultRelations);
Rework planning and execution of UPDATE and DELETE. This patch makes two closely related sets of changes: 1. For UPDATE, the subplan of the ModifyTable node now only delivers the new values of the changed columns (i.e., the expressions computed in the query's SET clause) plus row identity information such as CTID. ModifyTable must re-fetch the original tuple to merge in the old values of any unchanged columns. The core advantage of this is that the changed columns are uniform across all tables of an inherited or partitioned target relation, whereas the other columns might not be. A secondary advantage, when the UPDATE involves joins, is that less data needs to pass through the plan tree. The disadvantage of course is an extra fetch of each tuple to be updated. However, that seems to be very nearly free in context; even worst-case tests don't show it to add more than a couple percent to the total query cost. At some point it might be interesting to combine the re-fetch with the tuple access that ModifyTable must do anyway to mark the old tuple dead; but that would require a good deal of refactoring and it seems it wouldn't buy all that much, so this patch doesn't attempt it. 2. For inherited UPDATE/DELETE, instead of generating a separate subplan for each target relation, we now generate a single subplan that is just exactly like a SELECT's plan, then stick ModifyTable on top of that. To let ModifyTable know which target relation a given incoming row refers to, a tableoid junk column is added to the row identity information. This gets rid of the horrid hack that was inheritance_planner(), eliminating O(N^2) planning cost and memory consumption in cases where there were many unprunable target relations. Point 2 of course requires point 1, so that there is a uniform definition of the non-junk columns to be returned by the subplan. We can't insist on uniform definition of the row identity junk columns however, if we want to keep the ability to have both plain and foreign tables in a partitioning hierarchy. Since it wouldn't scale very far to have every child table have its own row identity column, this patch includes provisions to merge similar row identity columns into one column of the subplan result. In particular, we can merge the whole-row Vars typically used as row identity by FDWs into one column by pretending they are type RECORD. (It's still okay for the actual composite Datums to be labeled with the table's rowtype OID, though.) There is more that can be done to file down residual inefficiencies in this patch, but it seems to be committable now. FDW authors should note several API changes: * The argument list for AddForeignUpdateTargets() has changed, and so has the method it must use for adding junk columns to the query. Call add_row_identity_var() instead of manipulating the parse tree directly. You might want to reconsider exactly what you're adding, too. * PlanDirectModify() must now work a little harder to find the ForeignScan plan node; if the foreign table is part of a partitioning hierarchy then the ForeignScan might not be the direct child of ModifyTable. See postgres_fdw for sample code. * To check whether a relation is a target relation, it's no longer sufficient to compare its relid to root->parse->resultRelation. Instead, check it against all_result_relids or leaf_result_relids, as appropriate. Amit Langote and Tom Lane Discussion: https://postgr.es/m/CA+HiwqHpHdqdDn48yCEhynnniahH78rwcrv1rEX65-fsZGBOLQ@mail.gmail.com
2021-03-31 17:52:34 +02:00
READ_NODE_FIELD(updateColnosLists);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(withCheckOptionLists);
READ_NODE_FIELD(returningLists);
READ_NODE_FIELD(fdwPrivLists);
Directly modify foreign tables. postgres_fdw can now sent an UPDATE or DELETE statement directly to the foreign server in simple cases, rather than sending a SELECT FOR UPDATE statement and then updating or deleting rows one-by-one. Etsuro Fujita, reviewed by Rushabh Lathia, Shigeru Hanada, Kyotaro Horiguchi, Albe Laurenz, Thom Brown, and me.
2016-03-18 18:48:58 +01:00
READ_BITMAPSET_FIELD(fdwDirectModifyPlans);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_ENUM_FIELD(onConflictAction, OnConflictAction);
READ_NODE_FIELD(arbiterIndexes);
READ_NODE_FIELD(onConflictSet);
Fix mishandling of resjunk columns in ON CONFLICT ... UPDATE tlists. It's unusual to have any resjunk columns in an ON CONFLICT ... UPDATE list, but it can happen when MULTIEXPR_SUBLINK SubPlans are present. If it happens, the ON CONFLICT UPDATE code path would end up storing tuples that include the values of the extra resjunk columns. That's fairly harmless in the short run, but if new columns are added to the table then the values would become accessible, possibly leading to malfunctions if they don't match the datatypes of the new columns. This had escaped notice through a confluence of missing sanity checks, including * There's no cross-check that a tuple presented to heap_insert or heap_update matches the table rowtype. While it's difficult to check that fully at reasonable cost, we can easily add assertions that there aren't too many columns. * The output-column-assignment cases in execExprInterp.c lacked any sanity checks on the output column numbers, which seems like an oversight considering there are plenty of assertion checks on input column numbers. Add assertions there too. * We failed to apply nodeModifyTable's ExecCheckPlanOutput() to the ON CONFLICT UPDATE tlist. That wouldn't have caught this specific error, since that function is chartered to ignore resjunk columns; but it sure seems like a bad omission now that we've seen this bug. In HEAD, the right way to fix this is to make the processing of ON CONFLICT UPDATE tlists work the same as regular UPDATE tlists now do, that is don't add "SET x = x" entries, and use ExecBuildUpdateProjection to evaluate the tlist and combine it with old values of the not-set columns. This adds a little complication to ExecBuildUpdateProjection, but allows removal of a comparable amount of now-dead code from the planner. In the back branches, the most expedient solution seems to be to (a) use an output slot for the ON CONFLICT UPDATE projection that actually matches the target table, and then (b) invent a variant of ExecBuildProjectionInfo that can be told to not store values resulting from resjunk columns, so it doesn't try to store into nonexistent columns of the output slot. (We can't simply ignore the resjunk columns altogether; they have to be evaluated for MULTIEXPR_SUBLINK to work.) This works back to v10. In 9.6, projections work much differently and we can't cheaply give them such an option. The 9.6 version of this patch works by inserting a JunkFilter when it's necessary to get rid of resjunk columns. In addition, v11 and up have the reverse problem when trying to perform ON CONFLICT UPDATE on a partitioned table. Through a further oversight, adjust_partition_tlist() discarded resjunk columns when re-ordering the ON CONFLICT UPDATE tlist to match a partition. This accidentally prevented the storing-bogus-tuples problem, but at the cost that MULTIEXPR_SUBLINK cases didn't work, typically crashing if more than one row has to be updated. Fix by preserving resjunk columns in that routine. (I failed to resist the temptation to add more assertions there too, and to do some minor code beautification.) Per report from Andres Freund. Back-patch to all supported branches. Security: CVE-2021-32028
2021-05-10 17:02:29 +02:00
READ_NODE_FIELD(onConflictCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(onConflictWhere);
READ_UINT_FIELD(exclRelRTI);
READ_NODE_FIELD(exclRelTlist);
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
READ_NODE_FIELD(mergeActionLists);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readAppend
*/
static Append *
_readAppend(void)
{
READ_LOCALS(Append);
ReadCommonPlan(&local_node->plan);
Further adjust EXPLAIN's choices of table alias names. This patch causes EXPLAIN to always assign a separate table alias to the parent RTE of an append relation (inheritance set); before, such RTEs were ignored if not actually scanned by the plan. Since the child RTEs now always have that same alias to start with (cf. commit 55a1954da), the net effect is that the parent RTE usually gets the alias used or implied by the query text, and the children all get that alias with "_N" appended. (The exception to "usually" is if there are duplicate aliases in different subtrees of the original query; then some of those original RTEs will also have "_N" appended.) This results in more uniform output for partitioned-table plans than we had before: the partitioned table itself gets the original alias, and all child tables have aliases with "_N", rather than the previous behavior where one of the children would get an alias without "_N". The reason for giving the parent RTE an alias, even if it isn't scanned by the plan, is that we now use the parent's alias to qualify Vars that refer to an appendrel output column and appear above the Append or MergeAppend that computes the appendrel. But below the append, Vars refer to some one of the child relations, and are displayed that way. This seems clearer than the old behavior where a Var that could carry values from any child relation was displayed as if it referred to only one of them. While at it, change ruleutils.c so that the code paths used by EXPLAIN deal in Plan trees not PlanState trees. This effectively reverts a decision made in commit 1cc29fe7c, which seemed like a good idea at the time to make ruleutils.c consistent with explain.c. However, it's problematic because we'd really like to allow executor startup pruning to remove all the children of an append node when possible, leaving no child PlanState to resolve Vars against. (That's not done here, but will be in the next patch.) This requires different handling of subplans and initplans than before, but is otherwise a pretty straightforward change. Discussion: https://postgr.es/m/001001d4f44b$2a2cca50$7e865ef0$@lab.ntt.co.jp
2019-12-11 23:05:18 +01:00
READ_BITMAPSET_FIELD(apprelids);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(appendplans);
Add support for asynchronous execution. This implements asynchronous execution, which runs multiple parts of a non-parallel-aware Append concurrently rather than serially to improve performance when possible. Currently, the only node type that can be run concurrently is a ForeignScan that is an immediate child of such an Append. In the case where such ForeignScans access data on different remote servers, this would run those ForeignScans concurrently, and overlap the remote operations to be performed simultaneously, so it'll improve the performance especially when the operations involve time-consuming ones such as remote join and remote aggregation. We may extend this to other node types such as joins or aggregates over ForeignScans in the future. This also adds the support for postgres_fdw, which is enabled by the table-level/server-level option "async_capable". The default is false. Robert Haas, Kyotaro Horiguchi, Thomas Munro, and myself. This commit is mostly based on the patch proposed by Robert Haas, but also uses stuff from the patch proposed by Kyotaro Horiguchi and from the patch proposed by Thomas Munro. Reviewed by Kyotaro Horiguchi, Konstantin Knizhnik, Andrey Lepikhov, Movead Li, Thomas Munro, Justin Pryzby, and others. Discussion: https://postgr.es/m/CA%2BTgmoaXQEt4tZ03FtQhnzeDEMzBck%2BLrni0UWHVVgOTnA6C1w%40mail.gmail.com Discussion: https://postgr.es/m/CA%2BhUKGLBRyu0rHrDCMC4%3DRn3252gogyp1SjOgG8SEKKZv%3DFwfQ%40mail.gmail.com Discussion: https://postgr.es/m/20200228.170650.667613673625155850.horikyota.ntt%40gmail.com
2021-03-31 11:45:00 +02:00
READ_INT_FIELD(nasyncplans);
Support Parallel Append plan nodes. When we create an Append node, we can spread out the workers over the subplans instead of piling on to each subplan one at a time, which should typically be a bit more efficient, both because the startup cost of any plan executed entirely by one worker is paid only once and also because of reduced contention. We can also construct Append plans using a mix of partial and non-partial subplans, which may allow for parallelism in places that otherwise couldn't support it. Unfortunately, this patch doesn't handle the important case of parallelizing UNION ALL by running each branch in a separate worker; the executor infrastructure is added here, but more planner work is needed. Amit Khandekar, Robert Haas, Amul Sul, reviewed and tested by Ashutosh Bapat, Amit Langote, Rafia Sabih, Amit Kapila, and Rajkumar Raghuwanshi. Discussion: http://postgr.es/m/CAJ3gD9dy0K_E8r727heqXoBmWZ83HwLFwdcaSSmBQ1+S+vRuUQ@mail.gmail.com
2017-12-05 23:28:39 +01:00
READ_INT_FIELD(first_partial_plan);
Fix run-time partition pruning for appends with multiple source rels. The previous coding here supposed that if run-time partitioning applied to a particular Append/MergeAppend plan, then all child plans of that node must be members of a single partitioning hierarchy. This is totally wrong, since an Append could be formed from a UNION ALL: we could have multiple hierarchies sharing the same Append, or child plans that aren't part of any hierarchy. To fix, restructure the related plan-time and execution-time data structures so that we can have a separate list or array for each partitioning hierarchy. Also track subplans that are not part of any hierarchy, and make sure they don't get pruned. Per reports from Phil Florent and others. Back-patch to v11, since the bug originated there. David Rowley, with a lot of cosmetic adjustments by me; thanks also to Amit Langote for review. Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com
2018-08-02 01:42:46 +02:00
READ_NODE_FIELD(part_prune_info);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readMergeAppend
*/
static MergeAppend *
_readMergeAppend(void)
{
READ_LOCALS(MergeAppend);
ReadCommonPlan(&local_node->plan);
Further adjust EXPLAIN's choices of table alias names. This patch causes EXPLAIN to always assign a separate table alias to the parent RTE of an append relation (inheritance set); before, such RTEs were ignored if not actually scanned by the plan. Since the child RTEs now always have that same alias to start with (cf. commit 55a1954da), the net effect is that the parent RTE usually gets the alias used or implied by the query text, and the children all get that alias with "_N" appended. (The exception to "usually" is if there are duplicate aliases in different subtrees of the original query; then some of those original RTEs will also have "_N" appended.) This results in more uniform output for partitioned-table plans than we had before: the partitioned table itself gets the original alias, and all child tables have aliases with "_N", rather than the previous behavior where one of the children would get an alias without "_N". The reason for giving the parent RTE an alias, even if it isn't scanned by the plan, is that we now use the parent's alias to qualify Vars that refer to an appendrel output column and appear above the Append or MergeAppend that computes the appendrel. But below the append, Vars refer to some one of the child relations, and are displayed that way. This seems clearer than the old behavior where a Var that could carry values from any child relation was displayed as if it referred to only one of them. While at it, change ruleutils.c so that the code paths used by EXPLAIN deal in Plan trees not PlanState trees. This effectively reverts a decision made in commit 1cc29fe7c, which seemed like a good idea at the time to make ruleutils.c consistent with explain.c. However, it's problematic because we'd really like to allow executor startup pruning to remove all the children of an append node when possible, leaving no child PlanState to resolve Vars against. (That's not done here, but will be in the next patch.) This requires different handling of subplans and initplans than before, but is otherwise a pretty straightforward change. Discussion: https://postgr.es/m/001001d4f44b$2a2cca50$7e865ef0$@lab.ntt.co.jp
2019-12-11 23:05:18 +01:00
READ_BITMAPSET_FIELD(apprelids);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(mergeplans);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(sortColIdx, local_node->numCols);
READ_OID_ARRAY(sortOperators, local_node->numCols);
READ_OID_ARRAY(collations, local_node->numCols);
READ_BOOL_ARRAY(nullsFirst, local_node->numCols);
Fix run-time partition pruning for appends with multiple source rels. The previous coding here supposed that if run-time partitioning applied to a particular Append/MergeAppend plan, then all child plans of that node must be members of a single partitioning hierarchy. This is totally wrong, since an Append could be formed from a UNION ALL: we could have multiple hierarchies sharing the same Append, or child plans that aren't part of any hierarchy. To fix, restructure the related plan-time and execution-time data structures so that we can have a separate list or array for each partitioning hierarchy. Also track subplans that are not part of any hierarchy, and make sure they don't get pruned. Per reports from Phil Florent and others. Back-patch to v11, since the bug originated there. David Rowley, with a lot of cosmetic adjustments by me; thanks also to Amit Langote for review. Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com
2018-08-02 01:42:46 +02:00
READ_NODE_FIELD(part_prune_info);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readRecursiveUnion
*/
static RecursiveUnion *
_readRecursiveUnion(void)
{
READ_LOCALS(RecursiveUnion);
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(wtParam);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(dupColIdx, local_node->numCols);
READ_OID_ARRAY(dupOperators, local_node->numCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(dupCollations, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_LONG_FIELD(numGroups);
READ_DONE();
}
/*
* _readBitmapAnd
*/
static BitmapAnd *
_readBitmapAnd(void)
{
READ_LOCALS(BitmapAnd);
ReadCommonPlan(&local_node->plan);
READ_NODE_FIELD(bitmapplans);
READ_DONE();
}
/*
* _readBitmapOr
*/
static BitmapOr *
_readBitmapOr(void)
{
READ_LOCALS(BitmapOr);
ReadCommonPlan(&local_node->plan);
Support parallel bitmap heap scans. The index is scanned by a single process, but then all cooperating processes can iterate jointly over the resulting set of heap blocks. In the future, we might also want to support using a parallel bitmap index scan to set up for a parallel bitmap heap scan, but that's a job for another day. Dilip Kumar, with some corrections and cosmetic changes by me. The larger patch set of which this is a part has been reviewed and tested by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia Sabih, Haribabu Kommi, Thomas Munro, and me. Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
READ_BOOL_FIELD(isshared);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(bitmapplans);
READ_DONE();
}
/*
* ReadCommonScan
* Assign the basic stuff of all nodes that inherit from Scan
*/
static void
ReadCommonScan(Scan *local_node)
{
READ_TEMP_LOCALS();
ReadCommonPlan(&local_node->plan);
READ_UINT_FIELD(scanrelid);
}
/*
* _readScan
*/
static Scan *
_readScan(void)
{
READ_LOCALS_NO_FIELDS(Scan);
ReadCommonScan(local_node);
READ_DONE();
}
/*
* _readSeqScan
*/
static SeqScan *
_readSeqScan(void)
{
READ_LOCALS_NO_FIELDS(SeqScan);
Change SeqScan node to contain Scan node This makes the structure of all Scan-derived nodes the same, independent of whether they have additional fields. Discussion: https://www.postgresql.org/message-id/flat/c1097590-a6a4-486a-64b1-e1f9cc0533ce@enterprisedb.com
2021-08-08 16:55:51 +02:00
ReadCommonScan(&local_node->scan);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readSampleScan
*/
static SampleScan *
_readSampleScan(void)
{
READ_LOCALS(SampleScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(tablesample);
READ_DONE();
}
/*
* _readIndexScan
*/
static IndexScan *
_readIndexScan(void)
{
READ_LOCALS(IndexScan);
ReadCommonScan(&local_node->scan);
READ_OID_FIELD(indexid);
READ_NODE_FIELD(indexqual);
READ_NODE_FIELD(indexqualorig);
READ_NODE_FIELD(indexorderby);
READ_NODE_FIELD(indexorderbyorig);
READ_NODE_FIELD(indexorderbyops);
READ_ENUM_FIELD(indexorderdir, ScanDirection);
READ_DONE();
}
/*
* _readIndexOnlyScan
*/
static IndexOnlyScan *
_readIndexOnlyScan(void)
{
READ_LOCALS(IndexOnlyScan);
ReadCommonScan(&local_node->scan);
READ_OID_FIELD(indexid);
READ_NODE_FIELD(indexqual);
Fix index-only scan plans, take 2. Commit 4ace45677 failed to fix the problem fully, because the same issue of attempting to fetch a non-returnable index column can occur when rechecking the indexqual after using a lossy index operator. Moreover, it broke EXPLAIN for such indexquals (which indicates a gap in our test cases :-(). Revert the code changes of 4ace45677 in favor of adding a new field to struct IndexOnlyScan, containing a version of the indexqual that can be executed against the index-returned tuple without using any non-returnable columns. (The restrictions imposed by check_index_only guarantee this is possible, although we may have to recompute indexed expressions.) Support construction of that during setrefs.c processing by marking IndexOnlyScan.indextlist entries as resjunk if they can't be returned, rather than removing them entirely. (We could alternatively require setrefs.c to look up the IndexOptInfo again, but abusing resjunk this way seems like a reasonably safe way to avoid needing to do that.) This solution isn't great from an API-stability standpoint: if there are any extensions out there that build IndexOnlyScan structs directly, they'll be broken in the next minor releases. However, only a very invasive extension would be likely to do such a thing. There's no change in the Path representation, so typical planner extensions shouldn't have a problem. As before, back-patch to all supported branches. Discussion: https://postgr.es/m/3179992.1641150853@sss.pgh.pa.us Discussion: https://postgr.es/m/17350-b5bdcf476e5badbb@postgresql.org
2022-01-03 21:42:27 +01:00
READ_NODE_FIELD(recheckqual);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(indexorderby);
READ_NODE_FIELD(indextlist);
READ_ENUM_FIELD(indexorderdir, ScanDirection);
READ_DONE();
}
/*
* _readBitmapIndexScan
*/
static BitmapIndexScan *
_readBitmapIndexScan(void)
{
READ_LOCALS(BitmapIndexScan);
ReadCommonScan(&local_node->scan);
READ_OID_FIELD(indexid);
Support parallel bitmap heap scans. The index is scanned by a single process, but then all cooperating processes can iterate jointly over the resulting set of heap blocks. In the future, we might also want to support using a parallel bitmap index scan to set up for a parallel bitmap heap scan, but that's a job for another day. Dilip Kumar, with some corrections and cosmetic changes by me. The larger patch set of which this is a part has been reviewed and tested by (at least) Andres Freund, Amit Khandekar, Tushar Ahuja, Rafia Sabih, Haribabu Kommi, Thomas Munro, and me. Discussion: http://postgr.es/m/CAFiTN-uc4=0WxRGfCzs-xfkMYcSEWUC-Fon6thkJGjkh9i=13A@mail.gmail.com
2017-03-08 18:05:43 +01:00
READ_BOOL_FIELD(isshared);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(indexqual);
READ_NODE_FIELD(indexqualorig);
READ_DONE();
}
/*
* _readBitmapHeapScan
*/
static BitmapHeapScan *
_readBitmapHeapScan(void)
{
READ_LOCALS(BitmapHeapScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(bitmapqualorig);
READ_DONE();
}
/*
* _readTidScan
*/
static TidScan *
_readTidScan(void)
{
READ_LOCALS(TidScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(tidquals);
READ_DONE();
}
Add missing TidRangeScan readfunc Mistakenly forgotten in bb437f995
2021-02-27 11:21:21 +01:00
/*
* _readTidRangeScan
*/
static TidRangeScan *
_readTidRangeScan(void)
{
READ_LOCALS(TidRangeScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(tidrangequals);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readSubqueryScan
*/
static SubqueryScan *
_readSubqueryScan(void)
{
READ_LOCALS(SubqueryScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(subplan);
Allow asynchronous execution in more cases. In commit 27e1f1456, create_append_plan() only allowed the subplan created from a given subpath to be executed asynchronously when it was an async-capable ForeignPath. To extend coverage, this patch handles cases when the given subpath includes some other Path types as well that can be omitted in the plan processing, such as a ProjectionPath directly atop an async-capable ForeignPath, allowing asynchronous execution in partitioned-scan/partitioned-join queries with non-Var tlist expressions and more UNION queries. Andrey Lepikhov and Etsuro Fujita, reviewed by Alexander Pyhalov and Zhihong Yu. Discussion: https://postgr.es/m/659c37a8-3e71-0ff2-394c-f04428c76f08%40postgrespro.ru
2022-04-06 08:45:00 +02:00
READ_ENUM_FIELD(scanstatus, SubqueryScanStatus);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readFunctionScan
*/
static FunctionScan *
_readFunctionScan(void)
{
READ_LOCALS(FunctionScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(functions);
READ_BOOL_FIELD(funcordinality);
READ_DONE();
}
/*
* _readValuesScan
*/
static ValuesScan *
_readValuesScan(void)
{
READ_LOCALS(ValuesScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(values_lists);
READ_DONE();
}
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
/*
* _readTableFuncScan
*/
static TableFuncScan *
_readTableFuncScan(void)
{
READ_LOCALS(TableFuncScan);
ReadCommonScan(&local_node->scan);
READ_NODE_FIELD(tablefunc);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readCteScan
*/
static CteScan *
_readCteScan(void)
{
READ_LOCALS(CteScan);
ReadCommonScan(&local_node->scan);
READ_INT_FIELD(ctePlanId);
READ_INT_FIELD(cteParam);
READ_DONE();
}
Fix some probably-minor oversights in readfuncs.c. The system expects TABLEFUNC RTEs to have coltypes, coltypmods, and colcollations lists, but outfuncs doesn't dump them and readfuncs doesn't restore them. This doesn't cause obvious failures, because the only things that look at those fields are expandRTE() and get_rte_attribute_type(), which are mostly used during parse analysis, before anything would've passed the parsetree through outfuncs/readfuncs. But expandRTE() is used in build_physical_tlist(), which means that that function will return a wrong answer for a TABLEFUNC RTE that came from a view. Very accidentally, this doesn't cause serious problems, because what it will return is NIL which callers will interpret as "couldn't build a physical tlist because of dropped columns". So you still get a plan that works, though it's marginally less efficient than it could be. There are also some other expandRTE() calls associated with transformation of whole-row Vars in the planner. I have been unable to exhibit misbehavior from that, and it may be unreachable in any case that anyone would care about ... but I'm not entirely convinced, so this seems like something we should back- patch a fix for. Fortunately, we can fix it without forcing a change of stored rules and a catversion bump, because we can just copy these lists from the subsidiary TableFunc object. readfuncs.c was also missing support for NamedTuplestoreScan plan nodes. This accidentally fails to break parallel query because a query using a named tuplestore would never be considered parallel-safe anyway. However, project policy since parallel query came in is that all plan node types should have outfuncs/readfuncs support, so this is clearly an oversight that should be repaired. Noted while fooling around with a patch to test outfuncs/readfuncs more thoroughly. That exposed some other issues too, but these are the only ones that seem worth back-patching. Back-patch to v10 where both of these features came in. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 19:02:27 +02:00
/*
* _readNamedTuplestoreScan
*/
static NamedTuplestoreScan *
_readNamedTuplestoreScan(void)
{
READ_LOCALS(NamedTuplestoreScan);
ReadCommonScan(&local_node->scan);
READ_STRING_FIELD(enrname);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readWorkTableScan
*/
static WorkTableScan *
_readWorkTableScan(void)
{
READ_LOCALS(WorkTableScan);
ReadCommonScan(&local_node->scan);
READ_INT_FIELD(wtParam);
READ_DONE();
}
/*
* _readForeignScan
*/
static ForeignScan *
_readForeignScan(void)
{
READ_LOCALS(ForeignScan);
ReadCommonScan(&local_node->scan);
Fix missed update in _readForeignScan(). Blatant fail in 0bf3ae88af330496517722e391e7c975e6bad219. Caught by buildfarm member mandrill.
2016-03-19 06:20:34 +01:00
READ_ENUM_FIELD(operation, CmdType);
Standardize nodes/*funcs.c cosmetics for ForeignScan.resultRelation. catversion bump due to readfuncs.c field order change.
2021-06-06 09:08:21 +02:00
READ_UINT_FIELD(resultRelation);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_OID_FIELD(fs_server);
READ_NODE_FIELD(fdw_exprs);
READ_NODE_FIELD(fdw_private);
READ_NODE_FIELD(fdw_scan_tlist);
Allow FDWs to push down quals without breaking EvalPlanQual rechecks. This fixes a long-standing bug which was discovered while investigating the interaction between the new join pushdown code and the EvalPlanQual machinery: if a ForeignScan appears on the inner side of a paramaterized nestloop, an EPQ recheck would re-return the original tuple even if it no longer satisfied the pushed-down quals due to changed parameter values. This fix adds a new member to ForeignScan and ForeignScanState and a new argument to make_foreignscan, and requires changes to FDWs which push down quals to populate that new argument with a list of quals they have chosen to push down. Therefore, I'm only back-patching to 9.5, even though the bug is not new in 9.5. Etsuro Fujita, reviewed by me and by Kyotaro Horiguchi.
2015-10-15 19:00:40 +02:00
READ_NODE_FIELD(fdw_recheck_quals);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_BITMAPSET_FIELD(fs_relids);
READ_BOOL_FIELD(fsSystemCol);
READ_DONE();
}
Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me.
2015-11-12 13:40:31 +01:00
/*
* _readCustomScan
*/
static CustomScan *
_readCustomScan(void)
{
READ_LOCALS(CustomScan);
Rework custom scans to work more like the new extensible node stuff. Per discussion, the new extensible node framework is thought to be better designed than the custom path/scan/scanstate stuff we added in PostgreSQL 9.5. Rework the latter to be more like the former. This is not backward-compatible, but we generally don't promise that for C APIs, and there probably aren't many people using this yet anyway. KaiGai Kohei, reviewed by Petr Jelinek and me. Some further cosmetic changes by me.
2016-03-29 17:00:18 +02:00
char *custom_name;
Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me.
2015-11-12 13:40:31 +01:00
const CustomScanMethods *methods;
ReadCommonScan(&local_node->scan);
READ_UINT_FIELD(flags);
READ_NODE_FIELD(custom_plans);
READ_NODE_FIELD(custom_exprs);
READ_NODE_FIELD(custom_private);
READ_NODE_FIELD(custom_scan_tlist);
READ_BITMAPSET_FIELD(custom_relids);
Rework custom scans to work more like the new extensible node stuff. Per discussion, the new extensible node framework is thought to be better designed than the custom path/scan/scanstate stuff we added in PostgreSQL 9.5. Rework the latter to be more like the former. This is not backward-compatible, but we generally don't promise that for C APIs, and there probably aren't many people using this yet anyway. KaiGai Kohei, reviewed by Petr Jelinek and me. Some further cosmetic changes by me.
2016-03-29 17:00:18 +02:00
/* Lookup CustomScanMethods by CustomName */
Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me.
2015-11-12 13:40:31 +01:00
token = pg_strtok(&length); /* skip methods: */
Rework custom scans to work more like the new extensible node stuff. Per discussion, the new extensible node framework is thought to be better designed than the custom path/scan/scanstate stuff we added in PostgreSQL 9.5. Rework the latter to be more like the former. This is not backward-compatible, but we generally don't promise that for C APIs, and there probably aren't many people using this yet anyway. KaiGai Kohei, reviewed by Petr Jelinek and me. Some further cosmetic changes by me.
2016-03-29 17:00:18 +02:00
token = pg_strtok(&length); /* CustomName */
custom_name = nullable_string(token, length);
methods = GetCustomScanMethods(custom_name, false);
Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me.
2015-11-12 13:40:31 +01:00
local_node->methods = methods;
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* ReadCommonJoin
* Assign the basic stuff of all nodes that inherit from Join
*/
static void
ReadCommonJoin(Join *local_node)
{
READ_TEMP_LOCALS();
ReadCommonPlan(&local_node->plan);
READ_ENUM_FIELD(jointype, JoinType);
Optimize joins when the inner relation can be proven unique. If there can certainly be no more than one matching inner row for a given outer row, then the executor can move on to the next outer row as soon as it's found one match; there's no need to continue scanning the inner relation for this outer row. This saves useless scanning in nestloop and hash joins. In merge joins, it offers the opportunity to skip mark/restore processing, because we know we have not advanced past the first possible match for the next outer row. Of course, the devil is in the details: the proof of uniqueness must depend only on joinquals (not otherquals), and if we want to skip mergejoin mark/restore then it must depend only on merge clauses. To avoid adding more planning overhead than absolutely necessary, the present patch errs in the conservative direction: there are cases where inner_unique or skip_mark_restore processing could be used, but it will not do so because it's not sure that the uniqueness proof depended only on "safe" clauses. This could be improved later. David Rowley, reviewed and rather heavily editorialized on by me Discussion: https://postgr.es/m/CAApHDvqF6Sw-TK98bW48TdtFJ+3a7D2mFyZ7++=D-RyPsL76gw@mail.gmail.com
2017-04-08 04:20:03 +02:00
READ_BOOL_FIELD(inner_unique);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(joinqual);
}
/*
* _readNestLoop
*/
static NestLoop *
_readNestLoop(void)
{
READ_LOCALS(NestLoop);
ReadCommonJoin(&local_node->join);
READ_NODE_FIELD(nestParams);
READ_DONE();
}
/*
* _readMergeJoin
*/
static MergeJoin *
_readMergeJoin(void)
{
int numCols;
READ_LOCALS(MergeJoin);
ReadCommonJoin(&local_node->join);
Optimize joins when the inner relation can be proven unique. If there can certainly be no more than one matching inner row for a given outer row, then the executor can move on to the next outer row as soon as it's found one match; there's no need to continue scanning the inner relation for this outer row. This saves useless scanning in nestloop and hash joins. In merge joins, it offers the opportunity to skip mark/restore processing, because we know we have not advanced past the first possible match for the next outer row. Of course, the devil is in the details: the proof of uniqueness must depend only on joinquals (not otherquals), and if we want to skip mergejoin mark/restore then it must depend only on merge clauses. To avoid adding more planning overhead than absolutely necessary, the present patch errs in the conservative direction: there are cases where inner_unique or skip_mark_restore processing could be used, but it will not do so because it's not sure that the uniqueness proof depended only on "safe" clauses. This could be improved later. David Rowley, reviewed and rather heavily editorialized on by me Discussion: https://postgr.es/m/CAApHDvqF6Sw-TK98bW48TdtFJ+3a7D2mFyZ7++=D-RyPsL76gw@mail.gmail.com
2017-04-08 04:20:03 +02:00
READ_BOOL_FIELD(skip_mark_restore);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(mergeclauses);
numCols = list_length(local_node->mergeclauses);
READ_OID_ARRAY(mergeFamilies, numCols);
READ_OID_ARRAY(mergeCollations, numCols);
READ_INT_ARRAY(mergeStrategies, numCols);
READ_BOOL_ARRAY(mergeNullsFirst, numCols);
READ_DONE();
}
/*
* _readHashJoin
*/
static HashJoin *
_readHashJoin(void)
{
READ_LOCALS(HashJoin);
ReadCommonJoin(&local_node->join);
READ_NODE_FIELD(hashclauses);
Fix representation of hash keys in Hash/HashJoin nodes. In 5f32b29c1819 I changed the creation of HashState.hashkeys to actually use HashState as the parent (instead of HashJoinState, which was incorrect, as they were executed below HashState), to fix the problem of hashkeys expressions otherwise relying on slot types appropriate for HashJoinState, rather than HashState as would be correct. That reliance was only introduced in 12, which is why it previously worked to use HashJoinState as the parent (although I'd be unsurprised if there were problematic cases). Unfortunately that's not a sufficient solution, because before this commit, the to-be-hashed expressions referenced inner/outer as appropriate for the HashJoin, not Hash. That didn't have obvious bad consequences, because the slots containing the tuples were put into ecxt_innertuple when hashing a tuple for HashState (even though Hash doesn't have an inner plan). There are less common cases where this can cause visible problems however (rather than just confusion when inspecting such executor trees). E.g. "ERROR: bogus varno: 65000", when explaining queries containing a HashJoin where the subsidiary Hash node's hash keys reference a subplan. While normally hashkeys aren't displayed by EXPLAIN, if one of those expressions references a subplan, that subplan may be printed as part of the Hash node - which then failed because an inner plan was referenced, and Hash doesn't have that. It seems quite possible that there's other broken cases, too. Fix the problem by properly splitting the expression for the HashJoin and Hash nodes at plan time, and have them reference the proper subsidiary node. While other workarounds are possible, fixing this correctly seems easy enough. It was a pretty ugly hack to have ExecInitHashJoin put the expression into the already initialized HashState, in the first place. I decided to not just split inner/outer hashkeys inside make_hashjoin(), but also to separate out hashoperators and hashcollations at plan time. Otherwise we would have ended up having two very similar loops, one at plan time and the other during executor startup. The work seems to more appropriately belong to plan time, anyway. Reported-By: Nikita Glukhov, Alexander Korotkov Author: Andres Freund Reviewed-By: Tom Lane, in an earlier version Discussion: https://postgr.es/m/CAPpHfdvGVegF_TKKRiBrSmatJL2dR9uwFCuR+teQ_8tEXU8mxg@mail.gmail.com Backpatch: 12-
2019-08-02 09:02:46 +02:00
READ_NODE_FIELD(hashoperators);
READ_NODE_FIELD(hashcollations);
READ_NODE_FIELD(hashkeys);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readMaterial
*/
static Material *
_readMaterial(void)
{
READ_LOCALS_NO_FIELDS(Material);
ReadCommonPlan(&local_node->plan);
READ_DONE();
}
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
/*
Change the name of the Result Cache node to Memoize "Result Cache" was never a great name for this node, but nobody managed to come up with another name that anyone liked enough. That was until David Johnston mentioned "Node Memoization", which Tom Lane revised to just "Memoize". People seem to like "Memoize", so let's do the rename. Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/20210708165145.GG1176@momjian.us Backpatch-through: 14, where Result Cache was introduced
2021-07-14 02:43:58 +02:00
* _readMemoize
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
*/
Change the name of the Result Cache node to Memoize "Result Cache" was never a great name for this node, but nobody managed to come up with another name that anyone liked enough. That was until David Johnston mentioned "Node Memoization", which Tom Lane revised to just "Memoize". People seem to like "Memoize", so let's do the rename. Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/20210708165145.GG1176@momjian.us Backpatch-through: 14, where Result Cache was introduced
2021-07-14 02:43:58 +02:00
static Memoize *
_readMemoize(void)
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
{
Change the name of the Result Cache node to Memoize "Result Cache" was never a great name for this node, but nobody managed to come up with another name that anyone liked enough. That was until David Johnston mentioned "Node Memoization", which Tom Lane revised to just "Memoize". People seem to like "Memoize", so let's do the rename. Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/20210708165145.GG1176@momjian.us Backpatch-through: 14, where Result Cache was introduced
2021-07-14 02:43:58 +02:00
READ_LOCALS(Memoize);
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(numKeys);
READ_OID_ARRAY(hashOperators, local_node->numKeys);
READ_OID_ARRAY(collations, local_node->numKeys);
READ_NODE_FIELD(param_exprs);
READ_BOOL_FIELD(singlerow);
Allow Memoize to operate in binary comparison mode Memoize would always use the hash equality operator for the cache key types to determine if the current set of parameters were the same as some previously cached set. Certain types such as floating points where -0.0 and +0.0 differ in their binary representation but are classed as equal by the hash equality operator may cause problems as unless the join uses the same operator it's possible that whichever join operator is being used would be able to distinguish the two values. In which case we may accidentally return in the incorrect rows out of the cache. To fix this here we add a binary mode to Memoize to allow it to the current set of parameters to previously cached values by comparing bit-by-bit rather than logically using the hash equality operator. This binary mode is always used for LATERAL joins and it's used for normal joins when any of the join operators are not hashable. Reported-by: Tom Lane Author: David Rowley Discussion: https://postgr.es/m/3004308.1632952496@sss.pgh.pa.us Backpatch-through: 14, where Memoize was added
2021-11-23 22:06:59 +01:00
READ_BOOL_FIELD(binary_mode);
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
READ_UINT_FIELD(est_entries);
Flush Memoize cache when non-key parameters change, take 2 It's possible that a subplan below a Memoize node contains a parameter from above the Memoize node. If this parameter changes then cache entries may become out-dated due to the new parameter value. Previously Memoize was mistakenly not aware of this. We fix this here by flushing the cache whenever a parameter that's not part of the cache key changes. Bug: #17213 Reported by: Elvis Pranskevichus Author: David Rowley Discussion: https://postgr.es/m/17213-988ed34b225a2862@postgresql.org Backpatch-through: 14, where Memoize was added
2021-11-24 11:29:14 +01:00
READ_BITMAPSET_FIELD(keyparamids);
Add Result Cache executor node (take 2) Here we add a new executor node type named "Result Cache". The planner can include this node type in the plan to have the executor cache the results from the inner side of parameterized nested loop joins. This allows caching of tuples for sets of parameters so that in the event that the node sees the same parameter values again, it can just return the cached tuples instead of rescanning the inner side of the join all over again. Internally, result cache uses a hash table in order to quickly find tuples that have been previously cached. For certain data sets, this can significantly improve the performance of joins. The best cases for using this new node type are for join problems where a large portion of the tuples from the inner side of the join have no join partner on the outer side of the join. In such cases, hash join would have to hash values that are never looked up, thus bloating the hash table and possibly causing it to multi-batch. Merge joins would have to skip over all of the unmatched rows. If we use a nested loop join with a result cache, then we only cache tuples that have at least one join partner on the outer side of the join. The benefits of using a parameterized nested loop with a result cache increase when there are fewer distinct values being looked up and the number of lookups of each value is large. Also, hash probes to lookup the cache can be much faster than the hash probe in a hash join as it's common that the result cache's hash table is much smaller than the hash join's due to result cache only caching useful tuples rather than all tuples from the inner side of the join. This variation in hash probe performance is more significant when the hash join's hash table no longer fits into the CPU's L3 cache, but the result cache's hash table does. The apparent "random" access of hash buckets with each hash probe can cause a poor L3 cache hit ratio for large hash tables. Smaller hash tables generally perform better. The hash table used for the cache limits itself to not exceeding work_mem * hash_mem_multiplier in size. We maintain a dlist of keys for this cache and when we're adding new tuples and realize we've exceeded the memory budget, we evict cache entries starting with the least recently used ones until we have enough memory to add the new tuples to the cache. For parameterized nested loop joins, we now consider using one of these result cache nodes in between the nested loop node and its inner node. We determine when this might be useful based on cost, which is primarily driven off of what the expected cache hit ratio will be. Estimating the cache hit ratio relies on having good distinct estimates on the nested loop's parameters. For now, the planner will only consider using a result cache for parameterized nested loop joins. This works for both normal joins and also for LATERAL type joins to subqueries. It is possible to use this new node for other uses in the future. For example, to cache results from correlated subqueries. However, that's not done here due to some difficulties obtaining a distinct estimation on the outer plan to calculate the estimated cache hit ratio. Currently we plan the inner plan before planning the outer plan so there is no good way to know if a result cache would be useful or not since we can't estimate the number of times the subplan will be called until the outer plan is generated. The functionality being added here is newly introducing a dependency on the return value of estimate_num_groups() during the join search. Previously, during the join search, we only ever needed to perform selectivity estimations. With this commit, we need to use estimate_num_groups() in order to estimate what the hit ratio on the result cache will be. In simple terms, if we expect 10 distinct values and we expect 1000 outer rows, then we'll estimate the hit ratio to be 99%. Since cache hits are very cheap compared to scanning the underlying nodes on the inner side of the nested loop join, then this will significantly reduce the planner's cost for the join. However, it's fairly easy to see here that things will go bad when estimate_num_groups() incorrectly returns a value that's significantly lower than the actual number of distinct values. If this happens then that may cause us to make use of a nested loop join with a result cache instead of some other join type, such as a merge or hash join. Our distinct estimations have been known to be a source of trouble in the past, so the extra reliance on them here could cause the planner to choose slower plans than it did previous to having this feature. Distinct estimations are also fairly hard to estimate accurately when several tables have been joined already or when a WHERE clause filters out a set of values that are correlated to the expressions we're estimating the number of distinct value for. For now, the costing we perform during query planning for result caches does put quite a bit of faith in the distinct estimations being accurate. When these are accurate then we should generally see faster execution times for plans containing a result cache. However, in the real world, we may find that we need to either change the costings to put less trust in the distinct estimations being accurate or perhaps even disable this feature by default. There's always an element of risk when we teach the query planner to do new tricks that it decides to use that new trick at the wrong time and causes a regression. Users may opt to get the old behavior by turning the feature off using the enable_resultcache GUC. Currently, this is enabled by default. It remains to be seen if we'll maintain that setting for the release. Additionally, the name "Result Cache" is the best name I could think of for this new node at the time I started writing the patch. Nobody seems to strongly dislike the name. A few people did suggest other names but no other name seemed to dominate in the brief discussion that there was about names. Let's allow the beta period to see if the current name pleases enough people. If there's some consensus on a better name, then we can change it before the release. Please see the 2nd discussion link below for the discussion on the "Result Cache" name. Author: David Rowley Reviewed-by: Andy Fan, Justin Pryzby, Zhihong Yu, Hou Zhijie Tested-By: Konstantin Knizhnik Discussion: https://postgr.es/m/CAApHDvrPcQyQdWERGYWx8J%2B2DLUNgXu%2BfOSbQ1UscxrunyXyrQ%40mail.gmail.com Discussion: https://postgr.es/m/CAApHDvq=yQXr5kqhRviT2RhNKwToaWr9JAN5t+5_PzhuRJ3wvg@mail.gmail.com
2021-04-02 03:10:56 +02:00
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
Implement Incremental Sort Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com
2020-04-06 21:33:28 +02:00
* ReadCommonSort
* Assign the basic stuff of all nodes that inherit from Sort
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
*/
Implement Incremental Sort Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com
2020-04-06 21:33:28 +02:00
static void
ReadCommonSort(Sort *local_node)
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
{
Implement Incremental Sort Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com
2020-04-06 21:33:28 +02:00
READ_TEMP_LOCALS();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(sortColIdx, local_node->numCols);
READ_OID_ARRAY(sortOperators, local_node->numCols);
READ_OID_ARRAY(collations, local_node->numCols);
READ_BOOL_ARRAY(nullsFirst, local_node->numCols);
Implement Incremental Sort Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com
2020-04-06 21:33:28 +02:00
}
/*
* _readSort
*/
static Sort *
_readSort(void)
{
READ_LOCALS_NO_FIELDS(Sort);
ReadCommonSort(local_node);
READ_DONE();
}
/*
* _readIncrementalSort
*/
static IncrementalSort *
_readIncrementalSort(void)
{
READ_LOCALS(IncrementalSort);
ReadCommonSort(&local_node->sort);
READ_INT_FIELD(nPresortedCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readGroup
*/
static Group *
_readGroup(void)
{
READ_LOCALS(Group);
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(grpColIdx, local_node->numCols);
READ_OID_ARRAY(grpOperators, local_node->numCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(grpCollations, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readAgg
*/
static Agg *
_readAgg(void)
{
READ_LOCALS(Agg);
ReadCommonPlan(&local_node->plan);
READ_ENUM_FIELD(aggstrategy, AggStrategy);
Rethink node-level representation of partial-aggregation modes. The original coding had three separate booleans representing partial aggregation behavior, which was confusing, unreadable, and error-prone, not least because the booleans weren't always listed in the same order. It was also inadequate for the allegedly-desirable future extension to support intermediate partial aggregation, because we'd need separate markers for serialization and deserialization in such a case. Merge these bools into an enum "AggSplit" to provide symbolic names for the supported operating modes (and document what those are). By assigning the values of the enum constants carefully, we can treat AggSplit values as options bitmasks so that tests of what to do aren't noticeably more expensive than before. While at it, get rid of Aggref.aggoutputtype. That's not needed since commit 59a3795c2 got rid of setrefs.c's special-purpose Aggref comparison code, and it likewise seemed more confusing than helpful. Assorted comment cleanup as well (there's still more that I want to do in that line). catversion bump for change in Aggref node contents. Should be the last one for partial-aggregation changes. Discussion: <29309.1466699160@sss.pgh.pa.us>
2016-06-26 20:33:38 +02:00
READ_ENUM_FIELD(aggsplit, AggSplit);
Make the upper part of the planner work by generating and comparing Paths. I've been saying we needed to do this for more than five years, and here it finally is. This patch removes the ever-growing tangle of spaghetti logic that grouping_planner() used to use to try to identify the best plan for post-scan/join query steps. Now, there is (nearly) independent consideration of each execution step, and entirely separate construction of Paths to represent each of the possible ways to do that step. We choose the best Path or set of Paths using the same add_path() logic that's been used inside query_planner() for years. In addition, this patch removes the old restriction that subquery_planner() could return only a single Plan. It now returns a RelOptInfo containing a set of Paths, just as query_planner() does, and the parent query level can use each of those Paths as the basis of a SubqueryScanPath at its level. This allows finding some optimizations that we missed before, wherein a subquery was capable of returning presorted data and thereby avoiding a sort in the parent level, making the overall cost cheaper even though delivering sorted output was not the cheapest plan for the subquery in isolation. (A couple of regression test outputs change in consequence of that. However, there is very little change in visible planner behavior overall, because the point of this patch is not to get immediate planning benefits but to create the infrastructure for future improvements.) There is a great deal left to do here. This patch unblocks a lot of planner work that was basically impractical in the old code structure, such as allowing FDWs to implement remote aggregation, or rewriting plan_set_operations() to allow consideration of multiple implementation orders for set operations. (The latter will likely require a full rewrite of plan_set_operations(); what I've done here is only to fix it to return Paths not Plans.) I have also left unfinished some localized refactoring in createplan.c and planner.c, because it was not necessary to get this patch to a working state. Thanks to Robert Haas, David Rowley, and Amit Kapila for review.
2016-03-07 21:58:22 +01:00
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(grpColIdx, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_OID_ARRAY(grpOperators, local_node->numCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(grpCollations, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_LONG_FIELD(numGroups);
Fix commit c11cb17d. I neglected to update copyfuncs/outfuncs/readfuncs. Discussion: https://postgr.es/m/12491.1582833409%40sss.pgh.pa.us
2020-02-28 18:32:35 +01:00
READ_UINT64_FIELD(transitionSpace);
Fix improper repetition of previous results from a hashed aggregate. ExecReScanAgg's check for whether it could re-use a previously calculated hashtable neglected the possibility that the Agg node might reference PARAM_EXEC Params that are not referenced by its input plan node. That's okay if the Params are in upper tlist or qual expressions; but if one appears in aggregate input expressions, then the hashtable contents need to be recomputed when the Param's value changes. To avoid unnecessary performance degradation in the case of a Param that isn't within an aggregate input, add logic to the planner to determine which Params are within aggregate inputs. This requires a new field in struct Agg, but fortunately we never write plans to disk, so this isn't an initdb-forcing change. Per report from Jeevan Chalke. This has been broken since forever, so back-patch to all supported branches. Andrew Gierth, with minor adjustments by me Report: <CAM2+6=VY8ykfLT5Q8vb9B6EbeBk-NGuLbT6seaQ+Fq4zXvrDcA@mail.gmail.com>
2016-08-24 20:37:50 +02:00
READ_BITMAPSET_FIELD(aggParams);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(groupingSets);
READ_NODE_FIELD(chain);
READ_DONE();
}
/*
* _readWindowAgg
*/
static WindowAgg *
_readWindowAgg(void)
{
READ_LOCALS(WindowAgg);
ReadCommonPlan(&local_node->plan);
READ_UINT_FIELD(winref);
READ_INT_FIELD(partNumCols);
READ_ATTRNUMBER_ARRAY(partColIdx, local_node->partNumCols);
READ_OID_ARRAY(partOperators, local_node->partNumCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(partCollations, local_node->partNumCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_INT_FIELD(ordNumCols);
READ_ATTRNUMBER_ARRAY(ordColIdx, local_node->ordNumCols);
READ_OID_ARRAY(ordOperators, local_node->ordNumCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(ordCollations, local_node->ordNumCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_INT_FIELD(frameOptions);
READ_NODE_FIELD(startOffset);
READ_NODE_FIELD(endOffset);
Teach planner and executor about monotonic window funcs Window functions such as row_number() always return a value higher than the previously returned value for tuples in any given window partition. Traditionally queries such as; SELECT * FROM ( SELECT *, row_number() over (order by c) rn FROM t ) t WHERE rn <= 10; were executed fairly inefficiently. Neither the query planner nor the executor knew that once rn made it to 11 that nothing further would match the outer query's WHERE clause. It would blindly continue until all tuples were exhausted from the subquery. Here we implement means to make the above execute more efficiently. This is done by way of adding a pg_proc.prosupport function to various of the built-in window functions and adding supporting code to allow the support function to inform the planner if the window function is monotonically increasing, monotonically decreasing, both or neither. The planner is then able to make use of that information and possibly allow the executor to short-circuit execution by way of adding a "run condition" to the WindowAgg to allow it to determine if some of its execution work can be skipped. This "run condition" is not like a normal filter. These run conditions are only built using quals comparing values to monotonic window functions. For monotonic increasing functions, quals making use of the btree operators for <, <= and = can be used (assuming the window function column is on the left). You can see here that once such a condition becomes false that a monotonic increasing function could never make it subsequently true again. For monotonically decreasing functions the >, >= and = btree operators for the given type can be used for run conditions. The best-case situation for this is when there is a single WindowAgg node without a PARTITION BY clause. Here when the run condition becomes false the WindowAgg node can simply return NULL. No more tuples will ever match the run condition. It's a little more complex when there is a PARTITION BY clause. In this case, we cannot return NULL as we must still process other partitions. To speed this case up we pull tuples from the outer plan to check if they're from the same partition and simply discard them if they are. When we find a tuple belonging to another partition we start processing as normal again until the run condition becomes false or we run out of tuples to process. When there are multiple WindowAgg nodes to evaluate then this complicates the situation. For intermediate WindowAggs we must ensure we always return all tuples to the calling node. Any filtering done could lead to incorrect results in WindowAgg nodes above. For all intermediate nodes, we can still save some work when the run condition becomes false. We've no need to evaluate the WindowFuncs anymore. Other WindowAgg nodes cannot reference the value of these and these tuples will not appear in the final result anyway. The savings here are small in comparison to what can be saved in the top-level WingowAgg, but still worthwhile. Intermediate WindowAgg nodes never filter out tuples, but here we change WindowAgg so that the top-level WindowAgg filters out tuples that don't match the intermediate WindowAgg node's run condition. Such filters appear in the "Filter" clause in EXPLAIN for the top-level WindowAgg node. Here we add prosupport functions to allow the above to work for; row_number(), rank(), dense_rank(), count(*) and count(expr). It appears technically possible to do the same for min() and max(), however, it seems unlikely to be useful enough, so that's not done here. Bump catversion Author: David Rowley Reviewed-by: Andy Fan, Zhihong Yu Discussion: https://postgr.es/m/CAApHDvqvp3At8++yF8ij06sdcoo1S_b2YoaT9D4Nf+MObzsrLQ@mail.gmail.com
2022-04-08 00:34:36 +02:00
READ_NODE_FIELD(runCondition);
READ_NODE_FIELD(runConditionOrig);
Support all SQL:2011 options for window frame clauses. This patch adds the ability to use "RANGE offset PRECEDING/FOLLOWING" frame boundaries in window functions. We'd punted on that back in the original patch to add window functions, because it was not clear how to do it in a reasonably data-type-extensible fashion. That problem is resolved here by adding the ability for btree operator classes to provide an "in_range" support function that defines how to add or subtract the RANGE offset value. Factoring it this way also allows the operator class to avoid overflow problems near the ends of the datatype's range, if it wishes to expend effort on that. (In the committed patch, the integer opclasses handle that issue, but it did not seem worth the trouble to avoid overflow failures for datetime types.) The patch includes in_range support for the integer_ops opfamily (int2/int4/int8) as well as the standard datetime types. Support for other numeric types has been requested, but that seems like suitable material for a follow-on patch. In addition, the patch adds GROUPS mode which counts the offset in ORDER-BY peer groups rather than rows, and it adds the frame_exclusion options specified by SQL:2011. As far as I can see, we are now fully up to spec on window framing options. Existing behaviors remain unchanged, except that I changed the errcode for a couple of existing error reports to meet the SQL spec's expectation that negative "offset" values should be reported as SQLSTATE 22013. Internally and in relevant parts of the documentation, we now consistently use the terminology "offset PRECEDING/FOLLOWING" rather than "value PRECEDING/FOLLOWING", since the term "value" is confusingly vague. Oliver Ford, reviewed and whacked around some by me Discussion: https://postgr.es/m/CAGMVOdu9sivPAxbNN0X+q19Sfv9edEPv=HibOJhB14TJv_RCQg@mail.gmail.com
2018-02-07 06:06:50 +01:00
READ_OID_FIELD(startInRangeFunc);
READ_OID_FIELD(endInRangeFunc);
READ_OID_FIELD(inRangeColl);
READ_BOOL_FIELD(inRangeAsc);
READ_BOOL_FIELD(inRangeNullsFirst);
Teach planner and executor about monotonic window funcs Window functions such as row_number() always return a value higher than the previously returned value for tuples in any given window partition. Traditionally queries such as; SELECT * FROM ( SELECT *, row_number() over (order by c) rn FROM t ) t WHERE rn <= 10; were executed fairly inefficiently. Neither the query planner nor the executor knew that once rn made it to 11 that nothing further would match the outer query's WHERE clause. It would blindly continue until all tuples were exhausted from the subquery. Here we implement means to make the above execute more efficiently. This is done by way of adding a pg_proc.prosupport function to various of the built-in window functions and adding supporting code to allow the support function to inform the planner if the window function is monotonically increasing, monotonically decreasing, both or neither. The planner is then able to make use of that information and possibly allow the executor to short-circuit execution by way of adding a "run condition" to the WindowAgg to allow it to determine if some of its execution work can be skipped. This "run condition" is not like a normal filter. These run conditions are only built using quals comparing values to monotonic window functions. For monotonic increasing functions, quals making use of the btree operators for <, <= and = can be used (assuming the window function column is on the left). You can see here that once such a condition becomes false that a monotonic increasing function could never make it subsequently true again. For monotonically decreasing functions the >, >= and = btree operators for the given type can be used for run conditions. The best-case situation for this is when there is a single WindowAgg node without a PARTITION BY clause. Here when the run condition becomes false the WindowAgg node can simply return NULL. No more tuples will ever match the run condition. It's a little more complex when there is a PARTITION BY clause. In this case, we cannot return NULL as we must still process other partitions. To speed this case up we pull tuples from the outer plan to check if they're from the same partition and simply discard them if they are. When we find a tuple belonging to another partition we start processing as normal again until the run condition becomes false or we run out of tuples to process. When there are multiple WindowAgg nodes to evaluate then this complicates the situation. For intermediate WindowAggs we must ensure we always return all tuples to the calling node. Any filtering done could lead to incorrect results in WindowAgg nodes above. For all intermediate nodes, we can still save some work when the run condition becomes false. We've no need to evaluate the WindowFuncs anymore. Other WindowAgg nodes cannot reference the value of these and these tuples will not appear in the final result anyway. The savings here are small in comparison to what can be saved in the top-level WingowAgg, but still worthwhile. Intermediate WindowAgg nodes never filter out tuples, but here we change WindowAgg so that the top-level WindowAgg filters out tuples that don't match the intermediate WindowAgg node's run condition. Such filters appear in the "Filter" clause in EXPLAIN for the top-level WindowAgg node. Here we add prosupport functions to allow the above to work for; row_number(), rank(), dense_rank(), count(*) and count(expr). It appears technically possible to do the same for min() and max(), however, it seems unlikely to be useful enough, so that's not done here. Bump catversion Author: David Rowley Reviewed-by: Andy Fan, Zhihong Yu Discussion: https://postgr.es/m/CAApHDvqvp3At8++yF8ij06sdcoo1S_b2YoaT9D4Nf+MObzsrLQ@mail.gmail.com
2022-04-08 00:34:36 +02:00
READ_BOOL_FIELD(topWindow);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readUnique
*/
static Unique *
_readUnique(void)
{
READ_LOCALS(Unique);
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(uniqColIdx, local_node->numCols);
READ_OID_ARRAY(uniqOperators, local_node->numCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(uniqCollations, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
Fix readfuncs/outfuncs problems in last night's Gather patch. KaiGai Kohei, with one correction by me.
2015-10-01 15:15:36 +02:00
/*
* _readGather
*/
static Gather *
_readGather(void)
{
READ_LOCALS(Gather);
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(num_workers);
Force rescanning of parallel-aware scan nodes below a Gather[Merge]. The ExecReScan machinery contains various optimizations for postponing or skipping rescans of plan subtrees; for example a HashAgg node may conclude that it can re-use the table it built before, instead of re-reading its input subtree. But that is wrong if the input contains a parallel-aware table scan node, since the portion of the table scanned by the leader process is likely to vary from one rescan to the next. This explains the timing-dependent buildfarm failures we saw after commit a2b70c89c. The established mechanism for showing that a plan node's output is potentially variable is to mark it as depending on some runtime Param. Hence, to fix this, invent a dummy Param (one that has a PARAM_EXEC parameter number, but carries no actual value) associated with each Gather or GatherMerge node, mark parallel-aware nodes below that node as dependent on that Param, and arrange for ExecReScanGather[Merge] to flag that Param as changed whenever the Gather[Merge] node is rescanned. This solution breaks an undocumented assumption made by the parallel executor logic, namely that all rescans of nodes below a Gather[Merge] will happen synchronously during the ReScan of the top node itself. But that's fundamentally contrary to the design of the ExecReScan code, and so was doomed to fail someday anyway (even if you want to argue that the bug being fixed here wasn't a failure of that assumption). A follow-on patch will address that issue. In the meantime, the worst that's expected to happen is that given very bad timing luck, the leader might have to do all the work during a rescan, because workers think they have nothing to do, if they are able to start up before the eventual ReScan of the leader's parallel-aware table scan node has reset the shared scan state. Although this problem exists in 9.6, there does not seem to be any way for it to manifest there. Without GatherMerge, it seems that a plan tree that has a rescan-short-circuiting node below Gather will always also have one above it that will short-circuit in the same cases, preventing the Gather from being rescanned. Hence we won't take the risk of back-patching this change into 9.6. But v10 needs it. Discussion: https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
2017-08-30 15:29:55 +02:00
READ_INT_FIELD(rescan_param);
Fix readfuncs/outfuncs problems in last night's Gather patch. KaiGai Kohei, with one correction by me.
2015-10-01 15:15:36 +02:00
READ_BOOL_FIELD(single_copy);
Introduce a new GUC force_parallel_mode for testing purposes. When force_parallel_mode = true, we enable the parallel mode restrictions for all queries for which this is believed to be safe. For the subset of those queries believed to be safe to run entirely within a worker, we spin up a worker and run the query there instead of running it in the original process. When force_parallel_mode = regress, make additional changes to allow the regression tests to run cleanly even though parallel workers have been injected under the hood. Taken together, this facilitates both better user testing and better regression testing of the parallelism code. Robert Haas, with help from Amit Kapila and Rushabh Lathia.
2016-02-07 17:39:22 +01:00
READ_BOOL_FIELD(invisible);
Pass InitPlan values to workers via Gather (Merge). If a PARAM_EXEC parameter is used below a Gather (Merge) but the InitPlan that computes it is attached to or above the Gather (Merge), force the value to be computed before starting parallelism and pass it down to all workers. This allows us to use parallelism in cases where it previously would have had to be rejected as unsafe. We do - in this case - lose the optimization that the value is only computed if it's actually used. An alternative strategy would be to have the first worker that needs the value compute it, but one downside of that approach is that we'd then need to select a parallel-safe path to compute the parameter value; it couldn't for example contain a Gather (Merge) node. At some point in the future, we might want to consider both approaches. Independent of that consideration, there is a great deal more work that could be done to make more kinds of PARAM_EXEC parameters parallel-safe. This infrastructure could be used to allow a Gather (Merge) on the inner side of a nested loop (although that's not a very appealing plan) and cases where the InitPlan is attached below the Gather (Merge) could be addressed as well using various techniques. But this is a good start. Amit Kapila, reviewed and revised by me. Reviewing and testing from Kuntal Ghosh, Haribabu Kommi, and Tushar Ahuja. Discussion: http://postgr.es/m/CAA4eK1LV0Y1AUV4cUCdC+sYOx0Z0-8NAJ2Pd9=UKsbQ5Sr7+JQ@mail.gmail.com
2017-11-16 18:06:14 +01:00
READ_BITMAPSET_FIELD(initParam);
Fix readfuncs/outfuncs problems in last night's Gather patch. KaiGai Kohei, with one correction by me.
2015-10-01 15:15:36 +02:00
READ_DONE();
}
Add a Gather Merge executor node. Like Gather, we spawn multiple workers and run the same plan in each one; however, Gather Merge is used when each worker produces the same output ordering and we want to preserve that output ordering while merging together the streams of tuples from various workers. (In a way, Gather Merge is like a hybrid of Gather and MergeAppend.) This works out to a win if it saves us from having to perform an expensive Sort. In cases where only a small amount of data would need to be sorted, it may actually be faster to use a regular Gather node and then sort the results afterward, because Gather Merge sometimes needs to wait synchronously for tuples whereas a pure Gather generally doesn't. But if this avoids an expensive sort then it's a win. Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro, and Neha Sharma, and reviewed and revised by me. Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com
2017-03-09 13:40:36 +01:00
/*
* _readGatherMerge
*/
static GatherMerge *
_readGatherMerge(void)
{
READ_LOCALS(GatherMerge);
ReadCommonPlan(&local_node->plan);
READ_INT_FIELD(num_workers);
Force rescanning of parallel-aware scan nodes below a Gather[Merge]. The ExecReScan machinery contains various optimizations for postponing or skipping rescans of plan subtrees; for example a HashAgg node may conclude that it can re-use the table it built before, instead of re-reading its input subtree. But that is wrong if the input contains a parallel-aware table scan node, since the portion of the table scanned by the leader process is likely to vary from one rescan to the next. This explains the timing-dependent buildfarm failures we saw after commit a2b70c89c. The established mechanism for showing that a plan node's output is potentially variable is to mark it as depending on some runtime Param. Hence, to fix this, invent a dummy Param (one that has a PARAM_EXEC parameter number, but carries no actual value) associated with each Gather or GatherMerge node, mark parallel-aware nodes below that node as dependent on that Param, and arrange for ExecReScanGather[Merge] to flag that Param as changed whenever the Gather[Merge] node is rescanned. This solution breaks an undocumented assumption made by the parallel executor logic, namely that all rescans of nodes below a Gather[Merge] will happen synchronously during the ReScan of the top node itself. But that's fundamentally contrary to the design of the ExecReScan code, and so was doomed to fail someday anyway (even if you want to argue that the bug being fixed here wasn't a failure of that assumption). A follow-on patch will address that issue. In the meantime, the worst that's expected to happen is that given very bad timing luck, the leader might have to do all the work during a rescan, because workers think they have nothing to do, if they are able to start up before the eventual ReScan of the leader's parallel-aware table scan node has reset the shared scan state. Although this problem exists in 9.6, there does not seem to be any way for it to manifest there. Without GatherMerge, it seems that a plan tree that has a rescan-short-circuiting node below Gather will always also have one above it that will short-circuit in the same cases, preventing the Gather from being rescanned. Hence we won't take the risk of back-patching this change into 9.6. But v10 needs it. Discussion: https://postgr.es/m/CAA4eK1JkByysFJNh9M349u_nNjqETuEnY_y1VUc_kJiU0bxtaQ@mail.gmail.com
2017-08-30 15:29:55 +02:00
READ_INT_FIELD(rescan_param);
Add a Gather Merge executor node. Like Gather, we spawn multiple workers and run the same plan in each one; however, Gather Merge is used when each worker produces the same output ordering and we want to preserve that output ordering while merging together the streams of tuples from various workers. (In a way, Gather Merge is like a hybrid of Gather and MergeAppend.) This works out to a win if it saves us from having to perform an expensive Sort. In cases where only a small amount of data would need to be sorted, it may actually be faster to use a regular Gather node and then sort the results afterward, because Gather Merge sometimes needs to wait synchronously for tuples whereas a pure Gather generally doesn't. But if this avoids an expensive sort then it's a win. Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro, and Neha Sharma, and reviewed and revised by me. Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com
2017-03-09 13:40:36 +01:00
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(sortColIdx, local_node->numCols);
READ_OID_ARRAY(sortOperators, local_node->numCols);
READ_OID_ARRAY(collations, local_node->numCols);
READ_BOOL_ARRAY(nullsFirst, local_node->numCols);
Pass InitPlan values to workers via Gather (Merge). If a PARAM_EXEC parameter is used below a Gather (Merge) but the InitPlan that computes it is attached to or above the Gather (Merge), force the value to be computed before starting parallelism and pass it down to all workers. This allows us to use parallelism in cases where it previously would have had to be rejected as unsafe. We do - in this case - lose the optimization that the value is only computed if it's actually used. An alternative strategy would be to have the first worker that needs the value compute it, but one downside of that approach is that we'd then need to select a parallel-safe path to compute the parameter value; it couldn't for example contain a Gather (Merge) node. At some point in the future, we might want to consider both approaches. Independent of that consideration, there is a great deal more work that could be done to make more kinds of PARAM_EXEC parameters parallel-safe. This infrastructure could be used to allow a Gather (Merge) on the inner side of a nested loop (although that's not a very appealing plan) and cases where the InitPlan is attached below the Gather (Merge) could be addressed as well using various techniques. But this is a good start. Amit Kapila, reviewed and revised by me. Reviewing and testing from Kuntal Ghosh, Haribabu Kommi, and Tushar Ahuja. Discussion: http://postgr.es/m/CAA4eK1LV0Y1AUV4cUCdC+sYOx0Z0-8NAJ2Pd9=UKsbQ5Sr7+JQ@mail.gmail.com
2017-11-16 18:06:14 +01:00
READ_BITMAPSET_FIELD(initParam);
Add a Gather Merge executor node. Like Gather, we spawn multiple workers and run the same plan in each one; however, Gather Merge is used when each worker produces the same output ordering and we want to preserve that output ordering while merging together the streams of tuples from various workers. (In a way, Gather Merge is like a hybrid of Gather and MergeAppend.) This works out to a win if it saves us from having to perform an expensive Sort. In cases where only a small amount of data would need to be sorted, it may actually be faster to use a regular Gather node and then sort the results afterward, because Gather Merge sometimes needs to wait synchronously for tuples whereas a pure Gather generally doesn't. But if this avoids an expensive sort then it's a win. Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro, and Neha Sharma, and reviewed and revised by me. Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com
2017-03-09 13:40:36 +01:00
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readHash
*/
static Hash *
_readHash(void)
{
READ_LOCALS(Hash);
ReadCommonPlan(&local_node->plan);
Fix representation of hash keys in Hash/HashJoin nodes. In 5f32b29c1819 I changed the creation of HashState.hashkeys to actually use HashState as the parent (instead of HashJoinState, which was incorrect, as they were executed below HashState), to fix the problem of hashkeys expressions otherwise relying on slot types appropriate for HashJoinState, rather than HashState as would be correct. That reliance was only introduced in 12, which is why it previously worked to use HashJoinState as the parent (although I'd be unsurprised if there were problematic cases). Unfortunately that's not a sufficient solution, because before this commit, the to-be-hashed expressions referenced inner/outer as appropriate for the HashJoin, not Hash. That didn't have obvious bad consequences, because the slots containing the tuples were put into ecxt_innertuple when hashing a tuple for HashState (even though Hash doesn't have an inner plan). There are less common cases where this can cause visible problems however (rather than just confusion when inspecting such executor trees). E.g. "ERROR: bogus varno: 65000", when explaining queries containing a HashJoin where the subsidiary Hash node's hash keys reference a subplan. While normally hashkeys aren't displayed by EXPLAIN, if one of those expressions references a subplan, that subplan may be printed as part of the Hash node - which then failed because an inner plan was referenced, and Hash doesn't have that. It seems quite possible that there's other broken cases, too. Fix the problem by properly splitting the expression for the HashJoin and Hash nodes at plan time, and have them reference the proper subsidiary node. While other workarounds are possible, fixing this correctly seems easy enough. It was a pretty ugly hack to have ExecInitHashJoin put the expression into the already initialized HashState, in the first place. I decided to not just split inner/outer hashkeys inside make_hashjoin(), but also to separate out hashoperators and hashcollations at plan time. Otherwise we would have ended up having two very similar loops, one at plan time and the other during executor startup. The work seems to more appropriately belong to plan time, anyway. Reported-By: Nikita Glukhov, Alexander Korotkov Author: Andres Freund Reviewed-By: Tom Lane, in an earlier version Discussion: https://postgr.es/m/CAPpHfdvGVegF_TKKRiBrSmatJL2dR9uwFCuR+teQ_8tEXU8mxg@mail.gmail.com Backpatch: 12-
2019-08-02 09:02:46 +02:00
READ_NODE_FIELD(hashkeys);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_OID_FIELD(skewTable);
READ_INT_FIELD(skewColumn);
READ_BOOL_FIELD(skewInherit);
Add parallel-aware hash joins. Introduce parallel-aware hash joins that appear in EXPLAIN plans as Parallel Hash Join with Parallel Hash. While hash joins could already appear in parallel queries, they were previously always parallel-oblivious and had a partial subplan only on the outer side, meaning that the work of the inner subplan was duplicated in every worker. After this commit, the planner will consider using a partial subplan on the inner side too, using the Parallel Hash node to divide the work over the available CPU cores and combine its results in shared memory. If the join needs to be split into multiple batches in order to respect work_mem, then workers process different batches as much as possible and then work together on the remaining batches. The advantages of a parallel-aware hash join over a parallel-oblivious hash join used in a parallel query are that it: * avoids wasting memory on duplicated hash tables * avoids wasting disk space on duplicated batch files * divides the work of building the hash table over the CPUs One disadvantage is that there is some communication between the participating CPUs which might outweigh the benefits of parallelism in the case of small hash tables. This is avoided by the planner's existing reluctance to supply partial plans for small scans, but it may be necessary to estimate synchronization costs in future if that situation changes. Another is that outer batch 0 must be written to disk if multiple batches are required. A potential future advantage of parallel-aware hash joins is that right and full outer joins could be supported, since there is a single set of matched bits for each hashtable, but that is not yet implemented. A new GUC enable_parallel_hash is defined to control the feature, defaulting to on. Author: Thomas Munro Reviewed-By: Andres Freund, Robert Haas Tested-By: Rafia Sabih, Prabhat Sahu Discussion: https://postgr.es/m/CAEepm=2W=cOkiZxcg6qiFQP-dHUe09aqTrEMM7yJDrHMhDv_RA@mail.gmail.com https://postgr.es/m/CAEepm=37HKyJ4U6XOLi=JgfSHM3o6B-GaeO-6hkOmneTDkH+Uw@mail.gmail.com
2017-12-21 08:39:21 +01:00
READ_FLOAT_FIELD(rows_total);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readSetOp
*/
static SetOp *
_readSetOp(void)
{
READ_LOCALS(SetOp);
ReadCommonPlan(&local_node->plan);
READ_ENUM_FIELD(cmd, SetOpCmd);
READ_ENUM_FIELD(strategy, SetOpStrategy);
READ_INT_FIELD(numCols);
READ_ATTRNUMBER_ARRAY(dupColIdx, local_node->numCols);
READ_OID_ARRAY(dupOperators, local_node->numCols);
Collations with nondeterministic comparison This adds a flag "deterministic" to collations. If that is false, such a collation disables various optimizations that assume that strings are equal only if they are byte-wise equal. That then allows use cases such as case-insensitive or accent-insensitive comparisons or handling of strings with different Unicode normal forms. This functionality is only supported with the ICU provider. At least glibc doesn't appear to have any locales that work in a nondeterministic way, so it's not worth supporting this for the libc provider. The term "deterministic comparison" in this context is from Unicode Technical Standard #10 (https://unicode.org/reports/tr10/#Deterministic_Comparison). This patch makes changes in three areas: - CREATE COLLATION DDL changes and system catalog changes to support this new flag. - Many executor nodes and auxiliary code are extended to track collations. Previously, this code would just throw away collation information, because the eventually-called user-defined functions didn't use it since they only cared about equality, which didn't need collation information. - String data type functions that do equality comparisons and hashing are changed to take the (non-)deterministic flag into account. For comparison, this just means skipping various shortcuts and tie breakers that use byte-wise comparison. For hashing, we first need to convert the input string to a canonical "sort key" using the ICU analogue of strxfrm(). Reviewed-by: Daniel Verite <daniel@manitou-mail.org> Reviewed-by: Peter Geoghegan <pg@bowt.ie> Discussion: https://www.postgresql.org/message-id/flat/1ccc668f-4cbc-0bef-af67-450b47cdfee7@2ndquadrant.com
2019-03-22 12:09:32 +01:00
READ_OID_ARRAY(dupCollations, local_node->numCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_INT_FIELD(flagColIdx);
READ_INT_FIELD(firstFlag);
READ_LONG_FIELD(numGroups);
READ_DONE();
}
/*
* _readLockRows
*/
static LockRows *
_readLockRows(void)
{
READ_LOCALS(LockRows);
ReadCommonPlan(&local_node->plan);
READ_NODE_FIELD(rowMarks);
READ_INT_FIELD(epqParam);
READ_DONE();
}
/*
* _readLimit
*/
static Limit *
_readLimit(void)
{
READ_LOCALS(Limit);
ReadCommonPlan(&local_node->plan);
READ_NODE_FIELD(limitOffset);
READ_NODE_FIELD(limitCount);
Support FETCH FIRST WITH TIES WITH TIES is an option to the FETCH FIRST N ROWS clause (the SQL standard's spelling of LIMIT), where you additionally get rows that compare equal to the last of those N rows by the columns in the mandatory ORDER BY clause. There was a proposal by Andrew Gierth to implement this functionality in a more powerful way that would yield more features, but the other patch had not been finished at this time, so we decided to use this one for now in the spirit of incremental development. Author: Surafel Temesgen <surafel3000@gmail.com> Reviewed-by: Álvaro Herrera <alvherre@alvh.no-ip.org> Reviewed-by: Tomas Vondra <tomas.vondra@2ndquadrant.com> Discussion: https://postgr.es/m/CALAY4q9ky7rD_A4vf=FVQvCGngm3LOes-ky0J6euMrg=_Se+ag@mail.gmail.com Discussion: https://postgr.es/m/87o8wvz253.fsf@news-spur.riddles.org.uk
2020-04-07 22:22:13 +02:00
READ_ENUM_FIELD(limitOption, LimitOption);
READ_INT_FIELD(uniqNumCols);
READ_ATTRNUMBER_ARRAY(uniqColIdx, local_node->uniqNumCols);
READ_OID_ARRAY(uniqOperators, local_node->uniqNumCols);
READ_OID_ARRAY(uniqCollations, local_node->uniqNumCols);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_DONE();
}
/*
* _readNestLoopParam
*/
static NestLoopParam *
_readNestLoopParam(void)
{
READ_LOCALS(NestLoopParam);
READ_INT_FIELD(paramno);
READ_NODE_FIELD(paramval);
READ_DONE();
}
/*
* _readPlanRowMark
*/
static PlanRowMark *
_readPlanRowMark(void)
{
READ_LOCALS(PlanRowMark);
READ_UINT_FIELD(rti);
READ_UINT_FIELD(prti);
READ_UINT_FIELD(rowmarkId);
READ_ENUM_FIELD(markType, RowMarkType);
READ_INT_FIELD(allMarkTypes);
READ_ENUM_FIELD(strength, LockClauseStrength);
READ_ENUM_FIELD(waitPolicy, LockWaitPolicy);
READ_BOOL_FIELD(isParent);
READ_DONE();
}
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
static PartitionPruneInfo *
_readPartitionPruneInfo(void)
{
READ_LOCALS(PartitionPruneInfo);
Fix run-time partition pruning for appends with multiple source rels. The previous coding here supposed that if run-time partitioning applied to a particular Append/MergeAppend plan, then all child plans of that node must be members of a single partitioning hierarchy. This is totally wrong, since an Append could be formed from a UNION ALL: we could have multiple hierarchies sharing the same Append, or child plans that aren't part of any hierarchy. To fix, restructure the related plan-time and execution-time data structures so that we can have a separate list or array for each partitioning hierarchy. Also track subplans that are not part of any hierarchy, and make sure they don't get pruned. Per reports from Phil Florent and others. Back-patch to v11, since the bug originated there. David Rowley, with a lot of cosmetic adjustments by me; thanks also to Amit Langote for review. Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com
2018-08-02 01:42:46 +02:00
READ_NODE_FIELD(prune_infos);
READ_BITMAPSET_FIELD(other_subplans);
READ_DONE();
}
static PartitionedRelPruneInfo *
_readPartitionedRelPruneInfo(void)
{
READ_LOCALS(PartitionedRelPruneInfo);
Centralize executor's opening/closing of Relations for rangetable entries. Create an array estate->es_relations[] paralleling the es_range_table, and store references to Relations (relcache entries) there, so that any given RT entry is opened and closed just once per executor run. Scan nodes typically still call ExecOpenScanRelation, but ExecCloseScanRelation is no more; relation closing is now done centrally in ExecEndPlan. This is slightly more complex than one would expect because of the interactions with relcache references held in ResultRelInfo nodes. The general convention is now that ResultRelInfo->ri_RelationDesc does not represent a separate relcache reference and so does not need to be explicitly closed; but there is an exception for ResultRelInfos in the es_trig_target_relations list, which are manufactured by ExecGetTriggerResultRel and have to be cleaned up by ExecCleanUpTriggerState. (That much was true all along, but these ResultRelInfos are now more different from others than they used to be.) To allow the partition pruning logic to make use of es_relations[] rather than having its own relcache references, adjust PartitionedRelPruneInfo to store an RT index rather than a relation OID. Amit Langote, reviewed by David Rowley and Jesper Pedersen, some mods by me Discussion: https://postgr.es/m/468c85d9-540e-66a2-1dde-fec2b741e688@lab.ntt.co.jp
2018-10-04 20:03:37 +02:00
READ_UINT_FIELD(rtindex);
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
READ_BITMAPSET_FIELD(present_parts);
READ_INT_FIELD(nparts);
Assorted cosmetic cleanup of run-time-partition-pruning code. Use "subplan" rather than "subnode" to refer to the child plans of a partitioning Append; this seems a bit more specific and hence clearer. Improve assorted comments. No non-cosmetic changes. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-11 00:24:34 +02:00
READ_INT_ARRAY(subplan_map, local_node->nparts);
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
READ_INT_ARRAY(subpart_map, local_node->nparts);
Allow ATTACH PARTITION with only ShareUpdateExclusiveLock. We still require AccessExclusiveLock on the partition itself, because otherwise an insert that violates the newly-imposed partition constraint could be in progress at the same time that we're changing that constraint; only the lock level on the parent relation is weakened. To make this safe, we have to cope with (at least) three separate problems. First, relevant DDL might commit while we're in the process of building a PartitionDesc. If so, find_inheritance_children() might see a new partition while the RELOID system cache still has the old partition bound cached, and even before invalidation messages have been queued. To fix that, if we see that the pg_class tuple seems to be missing or to have a null relpartbound, refetch the value directly from the table. We can't get the wrong value, because DETACH PARTITION still requires AccessExclusiveLock throughout; if we ever want to change that, this will need more thought. In testing, I found it quite difficult to hit even the null-relpartbound case; the race condition is extremely tight, but the theoretical risk is there. Second, successive calls to RelationGetPartitionDesc might not return the same answer. The query planner will get confused if lookup up the PartitionDesc for a particular relation does not return a consistent answer for the entire duration of query planning. Likewise, query execution will get confused if the same relation seems to have a different PartitionDesc at different times. Invent a new PartitionDirectory concept and use it to ensure consistency. This ensures that a single invocation of either the planner or the executor sees the same view of the PartitionDesc from beginning to end, but it does not guarantee that the planner and the executor see the same view. Since this allows pointers to old PartitionDesc entries to survive even after a relcache rebuild, also postpone removing the old PartitionDesc entry until we're certain no one is using it. For the most part, it seems to be OK for the planner and executor to have different views of the PartitionDesc, because the executor will just ignore any concurrently added partitions which were unknown at plan time; those partitions won't be part of the inheritance expansion, but invalidation messages will trigger replanning at some point. Normally, this happens by the time the very next command is executed, but if the next command acquires no locks and executes a prepared query, it can manage not to notice until a new transaction is started. We might want to tighten that up, but it's material for a separate patch. There would still be a small window where a query that started just after an ATTACH PARTITION command committed might fail to notice its results -- but only if the command starts before the commit has been acknowledged to the user. All in all, the warts here around serializability seem small enough to be worth accepting for the considerable advantage of being able to add partitions without a full table lock. Although in general the consequences of new partitions showing up between planning and execution are limited to the query not noticing the new partitions, run-time partition pruning will get confused in that case, so that's the third problem that this patch fixes. Run-time partition pruning assumes that indexes into the PartitionDesc are stable between planning and execution. So, add code so that if new partitions are added between plan time and execution time, the indexes stored in the subplan_map[] and subpart_map[] arrays within the plan's PartitionedRelPruneInfo get adjusted accordingly. There does not seem to be a simple way to generalize this scheme to cope with partitions that are removed, mostly because they could then get added back again with different bounds, but it works OK for added partitions. This code does not try to ensure that every backend participating in a parallel query sees the same view of the PartitionDesc. That currently doesn't matter, because we never pass PartitionDesc indexes between backends. Each backend will ignore the concurrently added partitions which it notices, and it doesn't matter if different backends are ignoring different sets of concurrently added partitions. If in the future that matters, for example because we allow writes in parallel query and want all participants to do tuple routing to the same set of partitions, the PartitionDirectory concept could be improved to share PartitionDescs across backends. There is a draft patch to serialize and restore PartitionDescs on the thread where this patch was discussed, which may be a useful place to start. Patch by me. Thanks to Alvaro Herrera, David Rowley, Simon Riggs, Amit Langote, and Michael Paquier for discussion, and to Alvaro Herrera for some review. Discussion: http://postgr.es/m/CA+Tgmobt2upbSocvvDej3yzokd7AkiT+PvgFH+a9-5VV1oJNSQ@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoZE0r9-cyA-aY6f8WFEROaDLLL7Vf81kZ8MtFCkxpeQSw@mail.gmail.com Discussion: http://postgr.es/m/CA+TgmoY13KQZF-=HNTrt9UYWYx3_oYOQpu9ioNT49jGgiDpUEA@mail.gmail.com
2019-03-07 17:13:12 +01:00
READ_OID_ARRAY(relid_map, local_node->nparts);
Restructure creation of run-time pruning steps. Previously, gen_partprune_steps() always built executor pruning steps using all suitable clauses, including those containing PARAM_EXEC Params. This meant that the pruning steps were only completely safe for executor run-time (scan start) pruning. To prune at executor startup, we had to ignore the steps involving exec Params. But this doesn't really work in general, since there may be logic changes needed as well --- for example, pruning according to the last operator's btree strategy is the wrong thing if we're not applying that operator. The rules embodied in gen_partprune_steps() and its minions are sufficiently complicated that tracking their incremental effects in other logic seems quite impractical. Short of a complete redesign, the only safe fix seems to be to run gen_partprune_steps() twice, once to create executor startup pruning steps and then again for run-time pruning steps. We can save a few cycles however by noting during the first scan whether we rejected any clauses because they involved exec Params --- if not, we don't need to do the second scan. In support of this, refactor the internal APIs in partprune.c to make more use of passing information in the GeneratePruningStepsContext struct, rather than as separate arguments. This is, I hope, the last piece of our response to a bug report from Alan Jackson. Back-patch to v11 where this code came in. Discussion: https://postgr.es/m/FAD28A83-AC73-489E-A058-2681FA31D648@tvsquared.com
2019-05-18 01:44:19 +02:00
READ_NODE_FIELD(initial_pruning_steps);
READ_NODE_FIELD(exec_pruning_steps);
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
READ_BITMAPSET_FIELD(execparamids);
READ_DONE();
}
static PartitionPruneStepOp *
_readPartitionPruneStepOp(void)
{
READ_LOCALS(PartitionPruneStepOp);
READ_INT_FIELD(step.step_id);
READ_INT_FIELD(opstrategy);
READ_NODE_FIELD(exprs);
READ_NODE_FIELD(cmpfns);
READ_BITMAPSET_FIELD(nullkeys);
READ_DONE();
}
static PartitionPruneStepCombine *
_readPartitionPruneStepCombine(void)
{
READ_LOCALS(PartitionPruneStepCombine);
READ_INT_FIELD(step.step_id);
READ_ENUM_FIELD(combineOp, PartitionPruneCombineOp);
READ_NODE_FIELD(source_stepids);
READ_DONE();
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* _readPlanInvalItem
*/
static PlanInvalItem *
_readPlanInvalItem(void)
{
READ_LOCALS(PlanInvalItem);
READ_INT_FIELD(cacheId);
READ_UINT_FIELD(hashValue);
READ_DONE();
}
/*
* _readSubPlan
*/
static SubPlan *
_readSubPlan(void)
{
READ_LOCALS(SubPlan);
READ_ENUM_FIELD(subLinkType, SubLinkType);
READ_NODE_FIELD(testexpr);
READ_NODE_FIELD(paramIds);
READ_INT_FIELD(plan_id);
READ_STRING_FIELD(plan_name);
READ_OID_FIELD(firstColType);
READ_INT_FIELD(firstColTypmod);
READ_OID_FIELD(firstColCollation);
READ_BOOL_FIELD(useHashTable);
READ_BOOL_FIELD(unknownEqFalse);
Allow parallel workers to execute subplans. This doesn't do anything to make Param nodes anything other than parallel-restricted, so this only helps with uncorrelated subplans, and it's not necessarily very cheap because each worker will run the subplan separately (just as a Hash Join will build a separate copy of the hash table in each participating process), but it's a first step toward supporting cases that are more likely to help in practice, and is occasionally useful on its own. Amit Kapila, reviewed and tested by Rafia Sabih, Dilip Kumar, and me. Discussion: http://postgr.es/m/CAA4eK1+e8Z45D2n+rnDMDYsVEb5iW7jqaCH_tvPMYau=1Rru9w@mail.gmail.com
2017-02-15 00:09:47 +01:00
READ_BOOL_FIELD(parallel_safe);
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
READ_NODE_FIELD(setParam);
READ_NODE_FIELD(parParam);
READ_NODE_FIELD(args);
READ_FLOAT_FIELD(startup_cost);
READ_FLOAT_FIELD(per_call_cost);
READ_DONE();
}
/*
* _readAlternativeSubPlan
*/
static AlternativeSubPlan *
_readAlternativeSubPlan(void)
{
READ_LOCALS(AlternativeSubPlan);
READ_NODE_FIELD(subplans);
READ_DONE();
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
/*
* _readExtensibleNode
*/
static ExtensibleNode *
_readExtensibleNode(void)
{
const ExtensibleNodeMethods *methods;
ExtensibleNode *local_node;
const char *extnodename;
pgindent run for 9.6
2016-06-10 00:02:36 +02:00
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
READ_TEMP_LOCALS();
Fix typo in comment. Amit Langote
2016-07-07 22:13:37 +02:00
token = pg_strtok(&length); /* skip :extnodename */
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
token = pg_strtok(&length); /* get extnodename */
extnodename = nullable_string(token, length);
if (!extnodename)
elog(ERROR, "extnodename has to be supplied");
methods = GetExtensibleNodeMethods(extnodename, false);
local_node = (ExtensibleNode *) newNode(methods->node_size,
T_ExtensibleNode);
local_node->extnodename = extnodename;
/* deserialize the private fields */
methods->nodeRead(local_node);
READ_DONE();
}
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
/*
* _readPartitionBoundSpec
*/
static PartitionBoundSpec *
_readPartitionBoundSpec(void)
{
READ_LOCALS(PartitionBoundSpec);
READ_CHAR_FIELD(strategy);
Allow a partitioned table to have a default partition. Any tuples that don't route to any other partition will route to the default partition. Jeevan Ladhe, Beena Emerson, Ashutosh Bapat, Rahila Syed, and Robert Haas, with review and testing at various stages by (at least) Rushabh Lathia, Keith Fiske, Amit Langote, Amul Sul, Rajkumar Raghuanshi, Sven Kunze, Kyotaro Horiguchi, Thom Brown, Rafia Sabih, and Dilip Kumar. Discussion: http://postgr.es/m/CAH2L28tbN4SYyhS7YV1YBWcitkqbhSWfQCy0G=apRcC_PEO-bg@mail.gmail.com Discussion: http://postgr.es/m/CAOG9ApEYj34fWMcvBMBQ-YtqR9fTdXhdN82QEKG0SVZ6zeL1xg@mail.gmail.com
2017-09-08 23:28:04 +02:00
READ_BOOL_FIELD(is_default);
Add hash partitioning. Hash partitioning is useful when you want to partition a growing data set evenly. This can be useful to keep table sizes reasonable, which makes maintenance operations such as VACUUM faster, or to enable partition-wise join. At present, we still depend on constraint exclusion for partitioning pruning, and the shape of the partition constraints for hash partitioning is such that that doesn't work. Work is underway to fix that, which should both improve performance and make partitioning pruning work with hash partitioning. Amul Sul, reviewed and tested by Dilip Kumar, Ashutosh Bapat, Yugo Nagata, Rajkumar Raghuwanshi, Jesper Pedersen, and by me. A few final tweaks also by me. Discussion: http://postgr.es/m/CAAJ_b96fhpJAP=ALbETmeLk1Uni_GFZD938zgenhF49qgDTjaQ@mail.gmail.com
2017-11-10 00:07:25 +01:00
READ_INT_FIELD(modulus);
READ_INT_FIELD(remainder);
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
READ_NODE_FIELD(listdatums);
READ_NODE_FIELD(lowerdatums);
READ_NODE_FIELD(upperdatums);
Fix omission of locations in outfuncs/readfuncs partitioning node support. We could have limped along without this for v10, which was my intention when I annotated the bug in commit 76a3df6e5. But consensus is that it's better to fix it now and take the cost of a post-beta1 initdb (which is needed because these node types are stored in pg_class.relpartbound). Since we're forcing initdb anyway, take the opportunity to make the node type identification strings match the node struct names, instead of being randomly different from them. Discussion: https://postgr.es/m/E1dFBEX-0004wt-8t@gemulon.postgresql.org
2017-05-30 17:32:41 +02:00
READ_LOCATION_FIELD(location);
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
READ_DONE();
}
/*
* _readPartitionRangeDatum
*/
static PartitionRangeDatum *
_readPartitionRangeDatum(void)
{
READ_LOCALS(PartitionRangeDatum);
Use MINVALUE/MAXVALUE instead of UNBOUNDED for range partition bounds. Previously, UNBOUNDED meant no lower bound when used in the FROM list, and no upper bound when used in the TO list, which was OK for single-column range partitioning, but problematic with multiple columns. For example, an upper bound of (10.0, UNBOUNDED) would not be collocated with a lower bound of (10.0, UNBOUNDED), thus making it difficult or impossible to define contiguous multi-column range partitions in some cases. Fix this by using MINVALUE and MAXVALUE instead of UNBOUNDED to represent a partition column that is unbounded below or above respectively. This syntax removes any ambiguity, and ensures that if one partition's lower bound equals another partition's upper bound, then the partitions are contiguous. Also drop the constraint prohibiting finite values after an unbounded column, and just document the fact that any values after MINVALUE or MAXVALUE are ignored. Previously it was necessary to repeat UNBOUNDED multiple times, which was needlessly verbose. Note: Forces a post-PG 10 beta2 initdb. Report by Amul Sul, original patch by Amit Langote with some additional hacking by me. Discussion: https://postgr.es/m/CAAJ_b947mowpLdxL3jo3YLKngRjrq9+Ej4ymduQTfYR+8=YAYQ@mail.gmail.com
2017-07-21 10:20:47 +02:00
READ_ENUM_FIELD(kind, PartitionRangeDatumKind);
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
READ_NODE_FIELD(value);
Fix omission of locations in outfuncs/readfuncs partitioning node support. We could have limped along without this for v10, which was my intention when I annotated the bug in commit 76a3df6e5. But consensus is that it's better to fix it now and take the cost of a post-beta1 initdb (which is needed because these node types are stored in pg_class.relpartbound). Since we're forcing initdb anyway, take the opportunity to make the node type identification strings match the node struct names, instead of being randomly different from them. Discussion: https://postgr.es/m/E1dFBEX-0004wt-8t@gemulon.postgresql.org
2017-05-30 17:32:41 +02:00
READ_LOCATION_FIELD(location);
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
READ_DONE();
}
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* parseNodeString
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
* Given a character string representing a node tree, parseNodeString creates
* the internal node structure.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
* The string to be read must already have been loaded into pg_strtok().
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Node *
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
parseNodeString(void)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
void *return_value;
pgindent run.
2003-08-04 02:43:34 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
READ_TEMP_LOCALS();
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Add stack depth checks to key recursive functions in backend/nodes/*.c. Although copyfuncs.c has a check_stack_depth call in its recursion, equalfuncs.c, outfuncs.c, and readfuncs.c lacked one. This seems unwise. Likewise fix planstate_tree_walker(), in branches where that exists. Discussion: https://postgr.es/m/30253.1544286631@sss.pgh.pa.us
2018-12-10 17:12:43 +01:00
/* Guard against stack overflow due to overly complex expressions */
check_stack_depth();
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
token = pg_strtok(&length);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
#define MATCH(tokname, namelen) \
Use memcmp() rather than strncmp() when shorter string length is known. It appears that this will be faster for all but the shortest strings; at least one some platforms, memcmp() can use word-at-a-time comparisons. Noah Misch, somewhat pared down.
2010-12-22 04:11:40 +01:00
(length == namelen && memcmp(token, tokname, namelen) == 0)
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
if (MATCH("QUERY", 5))
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
return_value = _readQuery();
WITH CHECK OPTION support for auto-updatable VIEWs For simple views which are automatically updatable, this patch allows the user to specify what level of checking should be done on records being inserted or updated. For 'LOCAL CHECK', new tuples are validated against the conditionals of the view they are being inserted into, while for 'CASCADED CHECK' the new tuples are validated against the conditionals for all views involved (from the top down). This option is part of the SQL specification. Dean Rasheed, reviewed by Pavel Stehule
2013-07-18 23:10:16 +02:00
else if (MATCH("WITHCHECKOPTION", 15))
return_value = _readWithCheckOption();
Rearrange the querytree representation of ORDER BY/GROUP BY/DISTINCT items as per my recent proposal: 1. Fold SortClause and GroupClause into a single node type SortGroupClause. We were already relying on them to be struct-equivalent, so using two node tags wasn't accomplishing much except to get in the way of comparing items with equal(). 2. Add an "eqop" field to SortGroupClause to carry the associated equality operator. This is cheap for the parser to get at the same time it's looking up the sort operator, and storing it eliminates the need for repeated not-so-cheap lookups during planning. In future this will also let us represent GROUP/DISTINCT operations on datatypes that have hash opclasses but no btree opclasses (ie, they have equality but no natural sort order). The previous representation simply didn't work for that, since its only indicator of comparison semantics was a sort operator. 3. Add a hasDistinctOn boolean to struct Query to explicitly record whether the distinctClause came from DISTINCT or DISTINCT ON. This allows removing some complicated and not 100% bulletproof code that attempted to figure that out from the distinctClause alone. This patch doesn't in itself create any new capability, but it's necessary infrastructure for future attempts to use hash-based grouping for DISTINCT and UNION/INTERSECT/EXCEPT.
2008-08-02 23:32:01 +02:00
else if (MATCH("SORTGROUPCLAUSE", 15))
return_value = _readSortGroupClause();
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
else if (MATCH("GROUPINGSET", 11))
return_value = _readGroupingSet();
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
else if (MATCH("WINDOWCLAUSE", 12))
return_value = _readWindowClause();
Improve the representation of FOR UPDATE/FOR SHARE so that we can support both FOR UPDATE and FOR SHARE in one command, as well as both NOWAIT and normal WAIT behavior. The more general code is actually simpler and cleaner.
2006-04-30 20:30:40 +02:00
else if (MATCH("ROWMARKCLAUSE", 13))
return_value = _readRowMarkClause();
SEARCH and CYCLE clauses This adds the SQL standard feature that adds the SEARCH and CYCLE clauses to recursive queries to be able to do produce breadth- or depth-first search orders and detect cycles. These clauses can be rewritten into queries using existing syntax, and that is what this patch does in the rewriter. Reviewed-by: Vik Fearing <vik@postgresfriends.org> Reviewed-by: Pavel Stehule <pavel.stehule@gmail.com> Discussion: https://www.postgresql.org/message-id/flat/db80ceee-6f97-9b4a-8ee8-3ba0c58e5be2@2ndquadrant.com
2021-02-01 13:54:59 +01:00
else if (MATCH("CTESEARCHCLAUSE", 15))
return_value = _readCTESearchClause();
else if (MATCH("CTECYCLECLAUSE", 14))
return_value = _readCTECycleClause();
Implement SQL-standard WITH clauses, including WITH RECURSIVE. There are some unimplemented aspects: recursive queries must use UNION ALL (should allow UNION too), and we don't have SEARCH or CYCLE clauses. These might or might not get done for 8.4, but even without them it's a pretty useful feature. There are also a couple of small loose ends and definitional quibbles, which I'll send a memo about to pgsql-hackers shortly. But let's land the patch now so we can get on with other development. Yoshiyuki Asaba, with lots of help from Tatsuo Ishii and Tom Lane
2008-10-04 23:56:55 +02:00
else if (MATCH("COMMONTABLEEXPR", 15))
return_value = _readCommonTableExpr();
Add support for MERGE SQL command MERGE performs actions that modify rows in the target table using a source table or query. MERGE provides a single SQL statement that can conditionally INSERT/UPDATE/DELETE rows -- a task that would otherwise require multiple PL statements. For example, MERGE INTO target AS t USING source AS s ON t.tid = s.sid WHEN MATCHED AND t.balance > s.delta THEN UPDATE SET balance = t.balance - s.delta WHEN MATCHED THEN DELETE WHEN NOT MATCHED AND s.delta > 0 THEN INSERT VALUES (s.sid, s.delta) WHEN NOT MATCHED THEN DO NOTHING; MERGE works with regular tables, partitioned tables and inheritance hierarchies, including column and row security enforcement, as well as support for row and statement triggers and transition tables therein. MERGE is optimized for OLTP and is parameterizable, though also useful for large scale ETL/ELT. MERGE is not intended to be used in preference to existing single SQL commands for INSERT, UPDATE or DELETE since there is some overhead. MERGE can be used from PL/pgSQL. MERGE does not support targetting updatable views or foreign tables, and RETURNING clauses are not allowed either. These limitations are likely fixable with sufficient effort. Rewrite rules are also not supported, but it's not clear that we'd want to support them. Author: Pavan Deolasee <pavan.deolasee@gmail.com> Author: Álvaro Herrera <alvherre@alvh.no-ip.org> Author: Amit Langote <amitlangote09@gmail.com> Author: Simon Riggs <simon.riggs@enterprisedb.com> Reviewed-by: Peter Eisentraut <peter.eisentraut@enterprisedb.com> Reviewed-by: Andres Freund <andres@anarazel.de> (earlier versions) Reviewed-by: Peter Geoghegan <pg@bowt.ie> (earlier versions) Reviewed-by: Robert Haas <robertmhaas@gmail.com> (earlier versions) Reviewed-by: Japin Li <japinli@hotmail.com> Reviewed-by: Justin Pryzby <pryzby@telsasoft.com> Reviewed-by: Tomas Vondra <tomas.vondra@enterprisedb.com> Reviewed-by: Zhihong Yu <zyu@yugabyte.com> Discussion: https://postgr.es/m/CANP8+jKitBSrB7oTgT9CY2i1ObfOt36z0XMraQc+Xrz8QB0nXA@mail.gmail.com Discussion: https://postgr.es/m/CAH2-WzkJdBuxj9PO=2QaO9-3h3xGbQPZ34kJH=HukRekwM-GZg@mail.gmail.com Discussion: https://postgr.es/m/20201231134736.GA25392@alvherre.pgsql
2022-03-28 16:45:58 +02:00
else if (MATCH("MERGEWHENCLAUSE", 15))
return_value = _readMergeWhenClause();
else if (MATCH("MERGEACTION", 11))
return_value = _readMergeAction();
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
else if (MATCH("SETOPERATIONSTMT", 16))
Reimplementation of UNION/INTERSECT/EXCEPT. INTERSECT/EXCEPT now meet the SQL92 semantics, including support for ALL option. All three can be used in subqueries and views. DISTINCT and ORDER BY work now in views, too. This rewrite fixes many problems with cross-datatype UNIONs and INSERT/SELECT where the SELECT yields different datatypes than the INSERT needs. I did that by making UNION subqueries and SELECT in INSERT be treated like subselects-in-FROM, thereby allowing an extra level of targetlist where the datatype conversions can be inserted safely. INITDB NEEDED!
2000-10-05 21:11:39 +02:00
return_value = _readSetOperationStmt();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("ALIAS", 5))
return_value = _readAlias();
else if (MATCH("RANGEVAR", 8))
return_value = _readRangeVar();
Remove the Query structure from the executor's API. This allows us to stop storing mostly-redundant Query trees in prepared statements, portals, etc. To replace Query, a new node type called PlannedStmt is inserted by the planner at the top of a completed plan tree; this carries just the fields of Query that are still needed at runtime. The statement lists kept in portals etc. now consist of intermixed PlannedStmt and bare utility-statement nodes --- no Query. This incidentally allows us to remove some fields from Query and Plan nodes that shouldn't have been there in the first place. Still to do: simplify the execution-time range table; at the moment the range table passed to the executor still contains Query trees for subqueries. initdb forced due to change of stored rules.
2007-02-20 18:32:18 +01:00
else if (MATCH("INTOCLAUSE", 10))
return_value = _readIntoClause();
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
else if (MATCH("TABLEFUNC", 9))
return_value = _readTableFunc();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("VAR", 3))
return_value = _readVar();
else if (MATCH("CONST", 5))
return_value = _readConst();
else if (MATCH("PARAM", 5))
return_value = _readParam();
else if (MATCH("AGGREF", 6))
return_value = _readAggref();
Support GROUPING SETS, CUBE and ROLLUP. This SQL standard functionality allows to aggregate data by different GROUP BY clauses at once. Each grouping set returns rows with columns grouped by in other sets set to NULL. This could previously be achieved by doing each grouping as a separate query, conjoined by UNION ALLs. Besides being considerably more concise, grouping sets will in many cases be faster, requiring only one scan over the underlying data. The current implementation of grouping sets only supports using sorting for input. Individual sets that share a sort order are computed in one pass. If there are sets that don't share a sort order, additional sort & aggregation steps are performed. These additional passes are sourced by the previous sort step; thus avoiding repeated scans of the source data. The code is structured in a way that adding support for purely using hash aggregation or a mix of hashing and sorting is possible. Sorting was chosen to be supported first, as it is the most generic method of implementation. Instead of, as in an earlier versions of the patch, representing the chain of sort and aggregation steps as full blown planner and executor nodes, all but the first sort are performed inside the aggregation node itself. This avoids the need to do some unusual gymnastics to handle having to return aggregated and non-aggregated tuples from underlying nodes, as well as having to shut down underlying nodes early to limit memory usage. The optimizer still builds Sort/Agg node to describe each phase, but they're not part of the plan tree, but instead additional data for the aggregation node. They're a convenient and preexisting way to describe aggregation and sorting. The first (and possibly only) sort step is still performed as a separate execution step. That retains similarity with existing group by plans, makes rescans fairly simple, avoids very deep plans (leading to slow explains) and easily allows to avoid the sorting step if the underlying data is sorted by other means. A somewhat ugly side of this patch is having to deal with a grammar ambiguity between the new CUBE keyword and the cube extension/functions named cube (and rollup). To avoid breaking existing deployments of the cube extension it has not been renamed, neither has cube been made a reserved keyword. Instead precedence hacking is used to make GROUP BY cube(..) refer to the CUBE grouping sets feature, and not the function cube(). To actually group by a function cube(), unlikely as that might be, the function name has to be quoted. Needs a catversion bump because stored rules may change. Author: Andrew Gierth and Atri Sharma, with contributions from Andres Freund Reviewed-By: Andres Freund, Noah Misch, Tom Lane, Svenne Krap, Tomas Vondra, Erik Rijkers, Marti Raudsepp, Pavel Stehule Discussion: CAOeZVidmVRe2jU6aMk_5qkxnB7dfmPROzM7Ur8JPW5j8Y5X-Lw@mail.gmail.com
2015-05-16 03:40:59 +02:00
else if (MATCH("GROUPINGFUNC", 12))
return_value = _readGroupingFunc();
Support window functions a la SQL:2008. Hitoshi Harada, with some kibitzing from Heikki and Tom.
2008-12-28 19:54:01 +01:00
else if (MATCH("WINDOWFUNC", 10))
return_value = _readWindowFunc();
Renaming for new subscripting mechanism Over at patch https://commitfest.postgresql.org/21/1062/ Dmitry wants to introduce a more generic subscription mechanism, which allows subscripting not only arrays but also other object types such as JSONB. That functionality is introduced in a largish invasive patch, out of which this internal renaming patch was extracted. Author: Dmitry Dolgov Reviewed-by: Tom Lane, Arthur Zakirov Discussion: https://postgr.es/m/CA+q6zcUK4EqPAu7XRRO5CCjMwhz5zvg+rfWuLzVoxp_5sKS6=w@mail.gmail.com
2019-02-01 16:50:32 +01:00
else if (MATCH("SUBSCRIPTINGREF", 15))
return_value = _readSubscriptingRef();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("FUNCEXPR", 8))
return_value = _readFuncExpr();
Support use of function argument names to identify which actual arguments match which function parameters. The syntax uses AS, for example funcname(value AS arg1, anothervalue AS arg2) Pavel Stehule
2009-10-08 04:39:25 +02:00
else if (MATCH("NAMEDARGEXPR", 12))
return_value = _readNamedArgExpr();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("OPEXPR", 6))
return_value = _readOpExpr();
else if (MATCH("DISTINCTEXPR", 12))
return_value = _readDistinctExpr();
Revise collation derivation method and expression-tree representation. All expression nodes now have an explicit output-collation field, unless they are known to only return a noncollatable data type (such as boolean or record). Also, nodes that can invoke collation-aware functions store a separate field that is the collation value to pass to the function. This avoids confusion that arises when a function has collatable inputs and noncollatable output type, or vice versa. Also, replace the parser's on-the-fly collation assignment method with a post-pass over the completed expression tree. This allows us to use a more complex (and hopefully more nearly spec-compliant) assignment rule without paying for it in extra storage in every expression node. Fix assorted bugs in the planner's handling of collations by making collation one of the defining properties of an EquivalenceClass and by converting CollateExprs into discardable RelabelType nodes during expression preprocessing.
2011-03-20 01:29:08 +01:00
else if (MATCH("NULLIFEXPR", 10))
return_value = _readNullIfExpr();
Support expressions of the form 'scalar op ANY (array)' and 'scalar op ALL (array)', where the operator is applied between the lefthand scalar and each element of the array. The operator must yield boolean; the result of the construct is the OR or AND of the per-element results, respectively. Original coding by Joe Conway, after an idea of Peter's. Rewritten by Tom to keep the implementation strictly separate from subqueries.
2003-06-29 02:33:44 +02:00
else if (MATCH("SCALARARRAYOPEXPR", 17))
return_value = _readScalarArrayOpExpr();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("BOOLEXPR", 8))
return_value = _readBoolExpr();
else if (MATCH("SUBLINK", 7))
return_value = _readSubLink();
else if (MATCH("FIELDSELECT", 11))
return_value = _readFieldSelect();
Support assignment to subfields of composite columns in UPDATE and INSERT. As a side effect, cause subscripts in INSERT targetlists to do something more or less sensible; previously we evaluated such subscripts and then effectively ignored them. Another side effect is that UPDATE-ing an element or slice of an array value that is NULL now produces a non-null result, namely an array containing just the assigned-to positions.
2004-06-09 21:08:20 +02:00
else if (MATCH("FIELDSTORE", 10))
return_value = _readFieldStore();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("RELABELTYPE", 11))
return_value = _readRelabelType();
Downgrade implicit casts to text to be assignment-only, except for the ones from the other string-category types; this eliminates a lot of surprising interpretations that the parser could formerly make when there was no directly applicable operator. Create a general mechanism that supports casts to and from the standard string types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's I/O functions. These new casts are assignment-only in the to-string direction, explicit-only in the other, and therefore should create no surprising behavior. Remove a bunch of thereby-obsoleted datatype-specific casting functions. The "general mechanism" is a new expression node type CoerceViaIO that can actually convert between *any* two datatypes if their external text representations are compatible. This is more general than needed for the immediate feature, but might be useful in plpgsql or other places in future. This commit does nothing about the issue that applying the concatenation operator || to non-text types will now fail, often with strange error messages due to misinterpreting the operator as array concatenation. Since it often (not always) worked before, we should either make it succeed or at least give a more user-friendly error; but details are still under debate. Peter Eisentraut and Tom Lane
2007-06-05 23:31:09 +02:00
else if (MATCH("COERCEVIAIO", 11))
return_value = _readCoerceViaIO();
Fix array coercion expressions to ensure that the correct volatility is seen by code inspecting the expression. The best way to do this seems to be to drop the original representation as a function invocation, and instead make a special expression node type that represents applying the element-type coercion function to each array element. In this way the element function is exposed and will be checked for volatility. Per report from Guillaume Smet.
2007-03-28 01:21:12 +02:00
else if (MATCH("ARRAYCOERCEEXPR", 15))
return_value = _readArrayCoerceExpr();
Instead of supposing (wrongly, in the general case) that the rowtype of an inheritance child table is binary-compatible with the rowtype of its parent, invent an expression node type that does the conversion correctly. Fixes the new bug exhibited by Kris Shannon as well as a lot of old bugs that would only show up when using multiple inheritance or after altering the parent table.
2004-12-12 00:26:51 +01:00
else if (MATCH("CONVERTROWTYPEEXPR", 18))
return_value = _readConvertRowtypeExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("COLLATEEXPR", 11))
Split CollateClause into separate raw and analyzed node types. CollateClause is now used only in raw grammar output, and CollateExpr after parse analysis. This is for clarity and to avoid carrying collation names in post-analysis parse trees: that's both wasteful and possibly misleading, since the collation's name could be changed while the parsetree still exists. Also, clean up assorted infelicities and omissions in processing of the node type.
2011-03-11 22:27:51 +01:00
return_value = _readCollateExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("CASEEXPR", 8))
Add support for the CASE statement in the rewrite handling. Allows (at least some) rules and views. Still some trouble (crashes) with target CASE columns spanning tables, but lots now works.
1998-12-14 01:02:17 +01:00
return_value = _readCaseExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("CASEWHEN", 8))
Add support for the CASE statement in the rewrite handling. Allows (at least some) rules and views. Still some trouble (crashes) with target CASE columns spanning tables, but lots now works.
1998-12-14 01:02:17 +01:00
return_value = _readCaseWhen();
Reimplement CASE val WHEN compval1 THEN ... WHEN compval2 THEN ... END so that the 'val' is computed only once, per recent discussion. The speedup is not much when 'val' is just a simple variable, but could be significant for larger expressions. More importantly this avoids issues with multiple evaluations of a volatile 'val', and it allows the CASE expression to be reverse-listed in its original form by ruleutils.c.
2004-03-17 21:48:43 +01:00
else if (MATCH("CASETESTEXPR", 12))
return_value = _readCaseTestExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("ARRAYEXPR", 9))
First phase of work on array improvements. ARRAY[x,y,z] constructor expressions, ARRAY(sub-SELECT) expressions, some array functions. Polymorphic functions using ANYARRAY/ANYELEMENT argument and return types. Some regression tests in place, documentation is lacking. Joe Conway, with some kibitzing from Tom Lane.
2003-04-09 01:20:04 +02:00
return_value = _readArrayExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("ROWEXPR", 7))
Promote row expressions to full-fledged citizens of the expression syntax, rather than allowing them only in a few special cases as before. In particular you can now pass a ROW() construct to a function that accepts a rowtype parameter. Internal generation of RowExprs fixes a number of corner cases that used to not work very well, such as referencing the whole-row result of a JOIN or subquery. This represents a further step in the work I started a month or so back to make rowtype values into first-class citizens.
2004-05-11 00:44:49 +02:00
return_value = _readRowExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("ROWCOMPAREEXPR", 14))
Implement SQL-compliant treatment of row comparisons for < <= > >= cases (previously we only did = and <> correctly). Also, allow row comparisons with any operators that are in btree opclasses, not only those with these specific names. This gets rid of a whole lot of indefensible assumptions about the behavior of particular operators based on their names ... though it's still true that IN and NOT IN expand to "= ANY". The patch adds a RowCompareExpr expression node type, and makes some changes in the representation of ANY/ALL/ROWCOMPARE SubLinks so that they can share code with RowCompareExpr. I have not yet done anything about making RowCompareExpr an indexable operator, but will look at that soon. initdb forced due to changes in stored rules.
2005-12-28 02:30:02 +01:00
return_value = _readRowCompareExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("COALESCEEXPR", 12))
COALESCE() and NULLIF() are now first-class expressions, not macros that turn into CASE expressions. They evaluate their arguments at most once. Patch by Kris Jurka, review and (very light) editorializing by me.
2003-02-16 03:30:39 +01:00
return_value = _readCoalesceExpr();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("MINMAXEXPR", 10))
Add Oracle-compatible GREATEST and LEAST functions. Pavel Stehule
2005-06-27 00:05:42 +02:00
return_value = _readMinMaxExpr();
Improve parsetree representation of special functions such as CURRENT_DATE. We implement a dozen or so parameterless functions that the SQL standard defines special syntax for. Up to now, that was done by converting them into more or less ad-hoc constructs such as "'now'::text::date". That's messy for multiple reasons: it exposes what should be implementation details to users, and performance is worse than it needs to be in several cases. To improve matters, invent a new expression node type SQLValueFunction that can represent any of these parameterless functions. Bump catversion because this changes stored parsetrees for rules. Discussion: <30058.1463091294@sss.pgh.pa.us>
2016-08-17 02:33:01 +02:00
else if (MATCH("SQLVALUEFUNCTION", 16))
return_value = _readSQLValueFunction();
Code review for XML patch. Instill a bit of sanity in the location of the XmlExpr code in various lists, use a representation that has some hope of reverse-listing correctly (though it's still a de-escaping function shy of correctness), generally try to make it look more like Postgres coding conventions.
2006-12-24 01:29:20 +01:00
else if (MATCH("XMLEXPR", 7))
return_value = _readXmlExpr();
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
else if (MATCH("NULLTEST", 8))
Add IS UNKNOWN, IS NOT UNKNOWN boolean tests, fix the existing boolean tests to return the correct results per SQL9x when given NULL inputs. Reimplement these tests as well as IS [NOT] NULL to have their own expression node types, instead of depending on special functions. From Joe Conway, with a little help from Tom Lane.
2001-06-20 00:39:12 +02:00
return_value = _readNullTest();
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
else if (MATCH("BOOLEANTEST", 11))
Add IS UNKNOWN, IS NOT UNKNOWN boolean tests, fix the existing boolean tests to return the correct results per SQL9x when given NULL inputs. Reimplement these tests as well as IS [NOT] NULL to have their own expression node types, instead of depending on special functions. From Joe Conway, with a little help from Tom Lane.
2001-06-20 00:39:12 +02:00
return_value = _readBooleanTest();
Determine the set of constraints applied to a domain at executor startup, not in the parser; this allows ALTER DOMAIN to work correctly with domain constraint operations stored in rules. Rod Taylor; code review by Tom Lane.
2003-02-03 22:15:45 +01:00
else if (MATCH("COERCETODOMAIN", 14))
return_value = _readCoerceToDomain();
else if (MATCH("COERCETODOMAINVALUE", 19))
return_value = _readCoerceToDomainValue();
Code review for UPDATE tab SET col = DEFAULT patch ... whack it around so it has some chance of working in rules ...
2003-07-03 18:34:26 +02:00
else if (MATCH("SETTODEFAULT", 12))
return_value = _readSetToDefault();
Support UPDATE/DELETE WHERE CURRENT OF cursor_name, per SQL standard. Along the way, allow FOR UPDATE in non-WITH-HOLD cursors; there may once have been a reason to disallow that, but it seems to work now, and it's really rather necessary if you want to select a row via a cursor and then update it in a concurrent-safe fashion. Original patch by Arul Shaji, rather heavily editorialized by Tom Lane.
2007-06-11 03:16:30 +02:00
else if (MATCH("CURRENTOFEXPR", 13))
return_value = _readCurrentOfExpr();
Code review for NextValueExpr expression node type. Add missing infrastructure for this node type, notably in ruleutils.c where its lack could demonstrably cause EXPLAIN to fail. Add outfuncs/readfuncs support. (outfuncs support is useful today for debugging purposes. The readfuncs support may never be needed, since at present it would only matter for parallel query and NextValueExpr should never appear in a parallelizable query; but it seems like a bad idea to have a primnode type that isn't fully supported here.) Teach planner infrastructure that NextValueExpr is a volatile, parallel-unsafe, non-leaky expression node with cost cpu_operator_cost. Given its limited scope of usage, there *might* be no live bug today from the lack of that knowledge, but it's certainly going to bite us on the rear someday. Teach pg_stat_statements about the new node type, too. While at it, also teach cost_qual_eval() that MinMaxExpr, SQLValueFunction, XmlExpr, and CoerceToDomain should be charged as cpu_operator_cost. Failing to do this for SQLValueFunction was an oversight in my commit 0bb51aa96. The others are longer-standing oversights, but no time like the present to fix them. (In principle, CoerceToDomain could have cost much higher than this, but it doesn't presently seem worth trying to examine the domain's constraints here.) Modify execExprInterp.c to execute NextValueExpr as an out-of-line function; it seems quite unlikely to me that it's worth insisting that it be inlined in all expression eval methods. Besides, providing the out-of-line function doesn't stop anyone from inlining if they want to. Adjust some places where NextValueExpr support had been inserted with the aid of a dartboard rather than keeping it in the same order as elsewhere. Discussion: https://postgr.es/m/23862.1499981661@sss.pgh.pa.us
2017-07-14 21:25:43 +02:00
else if (MATCH("NEXTVALUEEXPR", 13))
return_value = _readNextValueExpr();
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
else if (MATCH("INFERENCEELEM", 13))
return_value = _readInferenceElem();
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
else if (MATCH("TARGETENTRY", 11))
return_value = _readTargetEntry();
else if (MATCH("RANGETBLREF", 11))
return_value = _readRangeTblRef();
else if (MATCH("JOINEXPR", 8))
return_value = _readJoinExpr();
else if (MATCH("FROMEXPR", 8))
return_value = _readFromExpr();
Add support for INSERT ... ON CONFLICT DO NOTHING/UPDATE. The newly added ON CONFLICT clause allows to specify an alternative to raising a unique or exclusion constraint violation error when inserting. ON CONFLICT refers to constraints that can either be specified using a inference clause (by specifying the columns of a unique constraint) or by naming a unique or exclusion constraint. DO NOTHING avoids the constraint violation, without touching the pre-existing row. DO UPDATE SET ... [WHERE ...] updates the pre-existing tuple, and has access to both the tuple proposed for insertion and the existing tuple; the optional WHERE clause can be used to prevent an update from being executed. The UPDATE SET and WHERE clauses have access to the tuple proposed for insertion using the "magic" EXCLUDED alias, and to the pre-existing tuple using the table name or its alias. This feature is often referred to as upsert. This is implemented using a new infrastructure called "speculative insertion". It is an optimistic variant of regular insertion that first does a pre-check for existing tuples and then attempts an insert. If a violating tuple was inserted concurrently, the speculatively inserted tuple is deleted and a new attempt is made. If the pre-check finds a matching tuple the alternative DO NOTHING or DO UPDATE action is taken. If the insertion succeeds without detecting a conflict, the tuple is deemed inserted. To handle the possible ambiguity between the excluded alias and a table named excluded, and for convenience with long relation names, INSERT INTO now can alias its target table. Bumps catversion as stored rules change. Author: Peter Geoghegan, with significant contributions from Heikki Linnakangas and Andres Freund. Testing infrastructure by Jeff Janes. Reviewed-By: Heikki Linnakangas, Andres Freund, Robert Haas, Simon Riggs, Dean Rasheed, Stephen Frost and many others.
2015-05-08 05:31:36 +02:00
else if (MATCH("ONCONFLICTEXPR", 14))
return_value = _readOnConflictExpr();
Add readfuncs.c support for AppendRelInfo. This is made necessary by the fact that commit 6ef77cf46 added AppendRelInfos to plan trees. I'd concluded that this extra code was not necessary because we don't transmit that data to parallel workers ... but I forgot about -DWRITE_READ_PARSE_PLAN_TREES. Per buildfarm.
2019-12-12 01:08:16 +01:00
else if (MATCH("APPENDRELINFO", 13))
return_value = _readAppendRelInfo();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("RANGETBLENTRY", 13))
Phase 2 of read-only-plans project: restructure expression-tree nodes so that all executable expression nodes inherit from a common supertype Expr. This is somewhat of an exercise in code purity rather than any real functional advance, but getting rid of the extra Oper or Func node formerly used in each operator or function call should provide at least a little space and speed improvement. initdb forced by changes in stored-rules representation.
2002-12-12 16:49:42 +01:00
return_value = _readRangeTblEntry();
Support multi-argument UNNEST(), and TABLE() syntax for multiple functions. This patch adds the ability to write TABLE( function1(), function2(), ...) as a single FROM-clause entry. The result is the concatenation of the first row from each function, followed by the second row from each function, etc; with NULLs inserted if any function produces fewer rows than others. This is believed to be a much more useful behavior than what Postgres currently does with multiple SRFs in a SELECT list. This syntax also provides a reasonable way to combine use of column definition lists with WITH ORDINALITY: put the column definition list inside TABLE(), where it's clear that it doesn't control the ordinality column as well. Also implement SQL-compliant multiple-argument UNNEST(), by turning UNNEST(a,b,c) into TABLE(unnest(a), unnest(b), unnest(c)). The SQL standard specifies TABLE() with only a single function, not multiple functions, and it seems to require an implicit UNNEST() which is not what this patch does. There may be something wrong with that reading of the spec, though, because if it's right then the spec's TABLE() is just a pointless alternative spelling of UNNEST(). After further review of that, we might choose to adopt a different syntax for what this patch does, but in any case this functionality seems clearly worthwhile. Andrew Gierth, reviewed by Zoltán Böszörményi and Heikki Linnakangas, and significantly revised by me
2013-11-22 01:37:02 +01:00
else if (MATCH("RANGETBLFUNCTION", 16))
return_value = _readRangeTblFunction();
Redesign tablesample method API, and do extensive code review. The original implementation of TABLESAMPLE modeled the tablesample method API on index access methods, which wasn't a good choice because, without specialized DDL commands, there's no way to build an extension that can implement a TSM. (Raw inserts into system catalogs are not an acceptable thing to do, because we can't undo them during DROP EXTENSION, nor will pg_upgrade behave sanely.) Instead adopt an API more like procedural language handlers or foreign data wrappers, wherein the only SQL-level support object needed is a single handler function identified by having a special return type. This lets us get rid of the supporting catalog altogether, so that no custom DDL support is needed for the feature. Adjust the API so that it can support non-constant tablesample arguments (the original coding assumed we could evaluate the argument expressions at ExecInitSampleScan time, which is undesirable even if it weren't outright unsafe), and discourage sampling methods from looking at invisible tuples. Make sure that the BERNOULLI and SYSTEM methods are genuinely repeatable within and across queries, as required by the SQL standard, and deal more honestly with methods that can't support that requirement. Make a full code-review pass over the tablesample additions, and fix assorted bugs, omissions, infelicities, and cosmetic issues (such as failure to put the added code stanzas in a consistent ordering). Improve EXPLAIN's output of tablesample plans, too. Back-patch to 9.5 so that we don't have to support the original API in production.
2015-07-25 20:39:00 +02:00
else if (MATCH("TABLESAMPLECLAUSE", 17))
return_value = _readTableSampleClause();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("NOTIFYSTMT", 10))
Restructure parsetree representation of DECLARE CURSOR: now it's a utility statement (DeclareCursorStmt) with a SELECT query dangling from it, rather than a SELECT query with a few unusual fields in it. Add code to determine whether a planned query can safely be run backwards. If DECLARE CURSOR specifies SCROLL, ensure that the plan can be run backwards by adding a Materialize plan node if it can't. Without SCROLL, you get an error if you try to fetch backwards from a cursor that can't handle it. (There is still some discussion about what the exact behavior should be, but this is necessary infrastructure in any case.) Along the way, make EXPLAIN DECLARE CURSOR work.
2003-03-10 04:53:52 +01:00
return_value = _readNotifyStmt();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("DEFELEM", 7))
return_value = _readDefElem();
Make node output prefix match node structure name In most cases, the prefix string in a node output is the upper case of the node structure name, e.g., MergeAppend -> MERGEAPPEND. There were a few exceptions that for either no apparent reason or perhaps minor aesthetic reasons deviated from this. In order to simplify this and perhaps allow automatic generation without having to deal with exception cases, make them all match. Discussion: https://www.postgresql.org/message-id/c091e5cd-45f8-69ee-6a9b-de86912cc7e7@enterprisedb.com
2021-09-15 16:35:41 +02:00
else if (MATCH("DECLARECURSORSTMT", 17))
Restructure parsetree representation of DECLARE CURSOR: now it's a utility statement (DeclareCursorStmt) with a SELECT query dangling from it, rather than a SELECT query with a few unusual fields in it. Add code to determine whether a planned query can safely be run backwards. If DECLARE CURSOR specifies SCROLL, ensure that the plan can be run backwards by adding a Materialize plan node if it can't. Without SCROLL, you get an error if you try to fetch backwards from a cursor that can't handle it. (There is still some discussion about what the exact behavior should be, but this is necessary infrastructure in any case.) Along the way, make EXPLAIN DECLARE CURSOR work.
2003-03-10 04:53:52 +01:00
return_value = _readDeclareCursorStmt();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("PLANNEDSTMT", 11))
return_value = _readPlannedStmt();
else if (MATCH("RESULT", 6))
return_value = _readResult();
Move targetlist SRF handling from expression evaluation to new executor node. Evaluation of set returning functions (SRFs_ in the targetlist (like SELECT generate_series(1,5)) so far was done in the expression evaluation (i.e. ExecEvalExpr()) and projection (i.e. ExecProject/ExecTargetList) code. This meant that most executor nodes performing projection, and most expression evaluation functions, had to deal with the possibility that an evaluated expression could return a set of return values. That's bad because it leads to repeated code in a lot of places. It also, and that's my (Andres's) motivation, made it a lot harder to implement a more efficient way of doing expression evaluation. To fix this, introduce a new executor node (ProjectSet) that can evaluate targetlists containing one or more SRFs. To avoid the complexity of the old way of handling nested expressions returning sets (e.g. having to pass up ExprDoneCond, and dealing with arguments to functions returning sets etc.), those SRFs can only be at the top level of the node's targetlist. The planner makes sure (via split_pathtarget_at_srfs()) that SRF evaluation is only necessary in ProjectSet nodes and that SRFs are only present at the top level of the node's targetlist. If there are nested SRFs the planner creates multiple stacked ProjectSet nodes. The ProjectSet nodes always get input from an underlying node. We also discussed and prototyped evaluating targetlist SRFs using ROWS FROM(), but that turned out to be more complicated than we'd hoped. While moving SRF evaluation to ProjectSet would allow to retain the old "least common multiple" behavior when multiple SRFs are present in one targetlist (i.e. continue returning rows until all SRFs are at the end of their input at the same time), we decided to instead only return rows till all SRFs are exhausted, returning NULL for already exhausted ones. We deemed the previous behavior to be too confusing, unexpected and actually not particularly useful. As a side effect, the previously prohibited case of multiple set returning arguments to a function, is now allowed. Not because it's particularly desirable, but because it ends up working and there seems to be no argument for adding code to prohibit it. Currently the behavior for COALESCE and CASE containing SRFs has changed, returning multiple rows from the expression, even when the SRF containing "arm" of the expression is not evaluated. That's because the SRFs are evaluated in a separate ProjectSet node. As that's quite confusing, we're likely to instead prohibit SRFs in those places. But that's still being discussed, and the code would reside in places not touched here, so that's a task for later. There's a lot of, now superfluous, code dealing with set return expressions around. But as the changes to get rid of those are verbose largely boring, it seems better for readability to keep the cleanup as a separate commit. Author: Tom Lane and Andres Freund Discussion: https://postgr.es/m/20160822214023.aaxz5l4igypowyri@alap3.anarazel.de
2017-01-18 21:46:50 +01:00
else if (MATCH("PROJECTSET", 10))
return_value = _readProjectSet();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("MODIFYTABLE", 11))
return_value = _readModifyTable();
else if (MATCH("APPEND", 6))
return_value = _readAppend();
else if (MATCH("MERGEAPPEND", 11))
return_value = _readMergeAppend();
else if (MATCH("RECURSIVEUNION", 14))
return_value = _readRecursiveUnion();
else if (MATCH("BITMAPAND", 9))
return_value = _readBitmapAnd();
else if (MATCH("BITMAPOR", 8))
return_value = _readBitmapOr();
else if (MATCH("SCAN", 4))
return_value = _readScan();
else if (MATCH("SEQSCAN", 7))
return_value = _readSeqScan();
else if (MATCH("SAMPLESCAN", 10))
return_value = _readSampleScan();
else if (MATCH("INDEXSCAN", 9))
return_value = _readIndexScan();
else if (MATCH("INDEXONLYSCAN", 13))
return_value = _readIndexOnlyScan();
else if (MATCH("BITMAPINDEXSCAN", 15))
return_value = _readBitmapIndexScan();
else if (MATCH("BITMAPHEAPSCAN", 14))
return_value = _readBitmapHeapScan();
else if (MATCH("TIDSCAN", 7))
return_value = _readTidScan();
Add missing TidRangeScan readfunc Mistakenly forgotten in bb437f995
2021-02-27 11:21:21 +01:00
else if (MATCH("TIDRANGESCAN", 12))
return_value = _readTidRangeScan();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("SUBQUERYSCAN", 12))
return_value = _readSubqueryScan();
else if (MATCH("FUNCTIONSCAN", 12))
return_value = _readFunctionScan();
else if (MATCH("VALUESSCAN", 10))
return_value = _readValuesScan();
Support XMLTABLE query expression XMLTABLE is defined by the SQL/XML standard as a feature that allows turning XML-formatted data into relational form, so that it can be used as a <table primary> in the FROM clause of a query. This new construct provides significant simplicity and performance benefit for XML data processing; what in a client-side custom implementation was reported to take 20 minutes can be executed in 400ms using XMLTABLE. (The same functionality was said to take 10 seconds using nested PostgreSQL XPath function calls, and 5 seconds using XMLReader under PL/Python). The implemented syntax deviates slightly from what the standard requires. First, the standard indicates that the PASSING clause is optional and that multiple XML input documents may be given to it; we make it mandatory and accept a single document only. Second, we don't currently support a default namespace to be specified. This implementation relies on a new executor node based on a hardcoded method table. (Because the grammar is fixed, there is no extensibility in the current approach; further constructs can be implemented on top of this such as JSON_TABLE, but they require changes to core code.) Author: Pavel Stehule, Álvaro Herrera Extensively reviewed by: Craig Ringer Discussion: https://postgr.es/m/CAFj8pRAgfzMD-LoSmnMGybD0WsEznLHWap8DO79+-GTRAPR4qA@mail.gmail.com
2017-03-08 16:39:37 +01:00
else if (MATCH("TABLEFUNCSCAN", 13))
return_value = _readTableFuncScan();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("CTESCAN", 7))
return_value = _readCteScan();
Fix some probably-minor oversights in readfuncs.c. The system expects TABLEFUNC RTEs to have coltypes, coltypmods, and colcollations lists, but outfuncs doesn't dump them and readfuncs doesn't restore them. This doesn't cause obvious failures, because the only things that look at those fields are expandRTE() and get_rte_attribute_type(), which are mostly used during parse analysis, before anything would've passed the parsetree through outfuncs/readfuncs. But expandRTE() is used in build_physical_tlist(), which means that that function will return a wrong answer for a TABLEFUNC RTE that came from a view. Very accidentally, this doesn't cause serious problems, because what it will return is NIL which callers will interpret as "couldn't build a physical tlist because of dropped columns". So you still get a plan that works, though it's marginally less efficient than it could be. There are also some other expandRTE() calls associated with transformation of whole-row Vars in the planner. I have been unable to exhibit misbehavior from that, and it may be unreachable in any case that anyone would care about ... but I'm not entirely convinced, so this seems like something we should back- patch a fix for. Fortunately, we can fix it without forcing a change of stored rules and a catversion bump, because we can just copy these lists from the subsidiary TableFunc object. readfuncs.c was also missing support for NamedTuplestoreScan plan nodes. This accidentally fails to break parallel query because a query using a named tuplestore would never be considered parallel-safe anyway. However, project policy since parallel query came in is that all plan node types should have outfuncs/readfuncs support, so this is clearly an oversight that should be repaired. Noted while fooling around with a patch to test outfuncs/readfuncs more thoroughly. That exposed some other issues too, but these are the only ones that seem worth back-patching. Back-patch to v10 where both of these features came in. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 19:02:27 +02:00
else if (MATCH("NAMEDTUPLESTORESCAN", 19))
return_value = _readNamedTuplestoreScan();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("WORKTABLESCAN", 13))
return_value = _readWorkTableScan();
else if (MATCH("FOREIGNSCAN", 11))
return_value = _readForeignScan();
Provide readfuncs support for custom scans. Commit a0d9f6e434bb56f7e5441b7988f3982feead33b3 added this support for all other plan node types; this fills in the gap. Since TextOutCustomScan complicates this and is pretty well useless, remove it. KaiGai Kohei, with some modifications by me.
2015-11-12 13:40:31 +01:00
else if (MATCH("CUSTOMSCAN", 10))
return_value = _readCustomScan();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("NESTLOOP", 8))
return_value = _readNestLoop();
else if (MATCH("MERGEJOIN", 9))
return_value = _readMergeJoin();
else if (MATCH("HASHJOIN", 8))
return_value = _readHashJoin();
else if (MATCH("MATERIAL", 8))
return_value = _readMaterial();
Change the name of the Result Cache node to Memoize "Result Cache" was never a great name for this node, but nobody managed to come up with another name that anyone liked enough. That was until David Johnston mentioned "Node Memoization", which Tom Lane revised to just "Memoize". People seem to like "Memoize", so let's do the rename. Reviewed-by: Justin Pryzby Discussion: https://postgr.es/m/20210708165145.GG1176@momjian.us Backpatch-through: 14, where Result Cache was introduced
2021-07-14 02:43:58 +02:00
else if (MATCH("MEMOIZE", 7))
return_value = _readMemoize();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("SORT", 4))
return_value = _readSort();
Implement Incremental Sort Incremental Sort is an optimized variant of multikey sort for cases when the input is already sorted by a prefix of the requested sort keys. For example when the relation is already sorted by (key1, key2) and we need to sort it by (key1, key2, key3) we can simply split the input rows into groups having equal values in (key1, key2), and only sort/compare the remaining column key3. This has a number of benefits: - Reduced memory consumption, because only a single group (determined by values in the sorted prefix) needs to be kept in memory. This may also eliminate the need to spill to disk. - Lower startup cost, because Incremental Sort produce results after each prefix group, which is beneficial for plans where startup cost matters (like for example queries with LIMIT clause). We consider both Sort and Incremental Sort, and decide based on costing. The implemented algorithm operates in two different modes: - Fetching a minimum number of tuples without check of equality on the prefix keys, and sorting on all columns when safe. - Fetching all tuples for a single prefix group and then sorting by comparing only the remaining (non-prefix) keys. We always start in the first mode, and employ a heuristic to switch into the second mode if we believe it's beneficial - the goal is to minimize the number of unnecessary comparions while keeping memory consumption below work_mem. This is a very old patch series. The idea was originally proposed by Alexander Korotkov back in 2013, and then revived in 2017. In 2018 the patch was taken over by James Coleman, who wrote and rewrote most of the current code. There were many reviewers/contributors since 2013 - I've done my best to pick the most active ones, and listed them in this commit message. Author: James Coleman, Alexander Korotkov Reviewed-by: Tomas Vondra, Andreas Karlsson, Marti Raudsepp, Peter Geoghegan, Robert Haas, Thomas Munro, Antonin Houska, Andres Freund, Alexander Kuzmenkov Discussion: https://postgr.es/m/CAPpHfdscOX5an71nHd8WSUH6GNOCf=V7wgDaTXdDd9=goN-gfA@mail.gmail.com Discussion: https://postgr.es/m/CAPpHfds1waRZ=NOmueYq0sx1ZSCnt+5QJvizT8ndT2=etZEeAQ@mail.gmail.com
2020-04-06 21:33:28 +02:00
else if (MATCH("INCREMENTALSORT", 15))
return_value = _readIncrementalSort();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("GROUP", 5))
return_value = _readGroup();
else if (MATCH("AGG", 3))
return_value = _readAgg();
else if (MATCH("WINDOWAGG", 9))
return_value = _readWindowAgg();
else if (MATCH("UNIQUE", 6))
return_value = _readUnique();
Fix readfuncs/outfuncs problems in last night's Gather patch. KaiGai Kohei, with one correction by me.
2015-10-01 15:15:36 +02:00
else if (MATCH("GATHER", 6))
return_value = _readGather();
Add a Gather Merge executor node. Like Gather, we spawn multiple workers and run the same plan in each one; however, Gather Merge is used when each worker produces the same output ordering and we want to preserve that output ordering while merging together the streams of tuples from various workers. (In a way, Gather Merge is like a hybrid of Gather and MergeAppend.) This works out to a win if it saves us from having to perform an expensive Sort. In cases where only a small amount of data would need to be sorted, it may actually be faster to use a regular Gather node and then sort the results afterward, because Gather Merge sometimes needs to wait synchronously for tuples whereas a pure Gather generally doesn't. But if this avoids an expensive sort then it's a win. Rushabh Lathia, reviewed and tested by Amit Kapila, Thomas Munro, and Neha Sharma, and reviewed and revised by me. Discussion: http://postgr.es/m/CAGPqQf09oPX-cQRpBKS0Gq49Z+m6KBxgxd_p9gX8CKk_d75HoQ@mail.gmail.com
2017-03-09 13:40:36 +01:00
else if (MATCH("GATHERMERGE", 11))
return_value = _readGatherMerge();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("HASH", 4))
return_value = _readHash();
else if (MATCH("SETOP", 5))
return_value = _readSetOp();
else if (MATCH("LOCKROWS", 8))
return_value = _readLockRows();
else if (MATCH("LIMIT", 5))
return_value = _readLimit();
else if (MATCH("NESTLOOPPARAM", 13))
return_value = _readNestLoopParam();
else if (MATCH("PLANROWMARK", 11))
return_value = _readPlanRowMark();
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
else if (MATCH("PARTITIONPRUNEINFO", 18))
return_value = _readPartitionPruneInfo();
Fix run-time partition pruning for appends with multiple source rels. The previous coding here supposed that if run-time partitioning applied to a particular Append/MergeAppend plan, then all child plans of that node must be members of a single partitioning hierarchy. This is totally wrong, since an Append could be formed from a UNION ALL: we could have multiple hierarchies sharing the same Append, or child plans that aren't part of any hierarchy. To fix, restructure the related plan-time and execution-time data structures so that we can have a separate list or array for each partitioning hierarchy. Also track subplans that are not part of any hierarchy, and make sure they don't get pruned. Per reports from Phil Florent and others. Back-patch to v11, since the bug originated there. David Rowley, with a lot of cosmetic adjustments by me; thanks also to Amit Langote for review. Discussion: https://postgr.es/m/HE1PR03MB17068BB27404C90B5B788BCABA7B0@HE1PR03MB1706.eurprd03.prod.outlook.com
2018-08-02 01:42:46 +02:00
else if (MATCH("PARTITIONEDRELPRUNEINFO", 23))
return_value = _readPartitionedRelPruneInfo();
Relocate partition pruning structs to a saner place. These struct definitions were originally dropped into primnodes.h, which is a poor choice since that's mainly intended for primitive expression node types; these are not in that category. What they are is auxiliary info in Plan trees, so move them to plannodes.h. For consistency, also relocate some related code that was apparently placed with the aid of a dartboard. There's no interesting code changes in this commit, just reshuffling. David Rowley and Tom Lane Discussion: https://postgr.es/m/CAFj8pRBjrufA3ocDm8o4LPGNye9Y+pm1b9kCwode4X04CULG3g@mail.gmail.com
2018-06-10 22:30:14 +02:00
else if (MATCH("PARTITIONPRUNESTEPOP", 20))
return_value = _readPartitionPruneStepOp();
else if (MATCH("PARTITIONPRUNESTEPCOMBINE", 25))
return_value = _readPartitionPruneStepCombine();
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
else if (MATCH("PLANINVALITEM", 13))
return_value = _readPlanInvalItem();
else if (MATCH("SUBPLAN", 7))
return_value = _readSubPlan();
else if (MATCH("ALTERNATIVESUBPLAN", 18))
return_value = _readAlternativeSubPlan();
Introduce extensible node types. An extensible node is always tagged T_Extensible, but the extnodename field identifies it more specifically; it may also include arbitrary private data. Extensible nodes can be copied, tested for equality, serialized, and deserialized, but the core system doesn't know anything about them otherwise. Some extensions may find it useful to include these nodes in fdw_private or custom_private lists in lieu of arm-wrestling their data into a format that the core code can understand. Along the way, so as not to burden the authors of such extensible node types too much, expose the functions for writing serialized tokens, and for serializing and deserializing bitmapsets. KaiGai Kohei, per a design suggested by me. Reviewed by Andres Freund and by me, and further edited by me.
2016-02-12 15:31:16 +01:00
else if (MATCH("EXTENSIBLENODE", 14))
return_value = _readExtensibleNode();
Fix omission of locations in outfuncs/readfuncs partitioning node support. We could have limped along without this for v10, which was my intention when I annotated the bug in commit 76a3df6e5. But consensus is that it's better to fix it now and take the cost of a post-beta1 initdb (which is needed because these node types are stored in pg_class.relpartbound). Since we're forcing initdb anyway, take the opportunity to make the node type identification strings match the node struct names, instead of being randomly different from them. Discussion: https://postgr.es/m/E1dFBEX-0004wt-8t@gemulon.postgresql.org
2017-05-30 17:32:41 +02:00
else if (MATCH("PARTITIONBOUNDSPEC", 18))
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
return_value = _readPartitionBoundSpec();
Fix omission of locations in outfuncs/readfuncs partitioning node support. We could have limped along without this for v10, which was my intention when I annotated the bug in commit 76a3df6e5. But consensus is that it's better to fix it now and take the cost of a post-beta1 initdb (which is needed because these node types are stored in pg_class.relpartbound). Since we're forcing initdb anyway, take the opportunity to make the node type identification strings match the node struct names, instead of being randomly different from them. Discussion: https://postgr.es/m/E1dFBEX-0004wt-8t@gemulon.postgresql.org
2017-05-30 17:32:41 +02:00
else if (MATCH("PARTITIONRANGEDATUM", 19))
Implement table partitioning. Table partitioning is like table inheritance and reuses much of the existing infrastructure, but there are some important differences. The parent is called a partitioned table and is always empty; it may not have indexes or non-inherited constraints, since those make no sense for a relation with no data of its own. The children are called partitions and contain all of the actual data. Each partition has an implicit partitioning constraint. Multiple inheritance is not allowed, and partitioning and inheritance can't be mixed. Partitions can't have extra columns and may not allow nulls unless the parent does. Tuples inserted into the parent are automatically routed to the correct partition, so tuple-routing ON INSERT triggers are not needed. Tuple routing isn't yet supported for partitions which are foreign tables, and it doesn't handle updates that cross partition boundaries. Currently, tables can be range-partitioned or list-partitioned. List partitioning is limited to a single column, but range partitioning can involve multiple columns. A partitioning "column" can be an expression. Because table partitioning is less general than table inheritance, it is hoped that it will be easier to reason about properties of partitions, and therefore that this will serve as a better foundation for a variety of possible optimizations, including query planner optimizations. The tuple routing based which this patch does based on the implicit partitioning constraints is an example of this, but it seems likely that many other useful optimizations are also possible. Amit Langote, reviewed and tested by Robert Haas, Ashutosh Bapat, Amit Kapila, Rajkumar Raghuwanshi, Corey Huinker, Jaime Casanova, Rushabh Lathia, Erik Rijkers, among others. Minor revisions by me.
2016-12-07 19:17:43 +01:00
return_value = _readPartitionRangeDatum();
Common SQL/JSON clauses This introduces some of the building blocks used by the SQL/JSON constructor and query functions. Specifically, it provides node executor and grammar support for the FORMAT JSON [ENCODING foo] clause, and values decorated with it, and for the RETURNING clause. The following SQL/JSON patches will leverage these. Nikita Glukhov (who probably deserves an award for perseverance). Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:00:49 +01:00
else if (MATCH("JSONFORMAT", 10))
return_value = _readJsonFormat();
else if (MATCH("JSONRETURNING", 13))
return_value = _readJsonReturning();
else if (MATCH("JSONVALUEEXPR", 13))
return_value = _readJsonValueExpr();
Make node output prefix match node structure name as done in e58136069687b9cf29c27281e227ac397d72141d
2022-04-12 16:18:01 +02:00
else if (MATCH("JSONCONSTRUCTOREXPR", 19))
SQL/JSON constructors This patch introduces the SQL/JSON standard constructors for JSON: JSON() JSON_ARRAY() JSON_ARRAYAGG() JSON_OBJECT() JSON_OBJECTAGG() For the most part these functions provide facilities that mimic existing json/jsonb functions. However, they also offer some useful additional functionality. In addition to text input, the JSON() function accepts bytea input, which it will decode and constuct a json value from. The other functions provide useful options for handling duplicate keys and null values. This series of patches will be followed by a consolidated documentation patch. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:10 +01:00
return_value = _readJsonConstructorExpr();
IS JSON predicate This patch intrdocuces the SQL standard IS JSON predicate. It operates on text and bytea values representing JSON as well as on the json and jsonb types. Each test has an IS and IS NOT variant. The tests are: IS JSON [VALUE] IS JSON ARRAY IS JSON OBJECT IS JSON SCALAR IS JSON WITH | WITHOUT UNIQUE KEYS These are mostly self-explanatory, but note that IS JSON WITHOUT UNIQUE KEYS is true whenever IS JSON is true, and IS JSON WITH UNIQUE KEYS is true whenever IS JSON is true except it IS JSON OBJECT is true and there are duplicate keys (which is never the case when applied to jsonb values). Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:02:53 +01:00
else if (MATCH("JSONISPREDICATE", 15))
return_value = _readJsonIsPredicate();
SQL/JSON query functions This introduces the SQL/JSON functions for querying JSON data using jsonpath expressions. The functions are: JSON_EXISTS() JSON_QUERY() JSON_VALUE() All of these functions only operate on jsonb. The workaround for now is to cast the argument to jsonb. JSON_EXISTS() tests if the jsonpath expression applied to the jsonb value yields any values. JSON_VALUE() must return a single value, and an error occurs if it tries to return multiple values. JSON_QUERY() must return a json object or array, and there are various WRAPPER options for handling scalar or multi-value results. Both these functions have options for handling EMPTY and ERROR conditions. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zihong Yu, Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/cd0bb935-0158-78a7-08b5-904886deac4b@postgrespro.ru
2022-03-03 19:11:14 +01:00
else if (MATCH("JSONBEHAVIOR", 12))
return_value = _readJsonBehavior();
else if (MATCH("JSONEXPR", 8))
return_value = _readJsonExpr();
else if (MATCH("JSONCOERCION", 12))
return_value = _readJsonCoercion();
else if (MATCH("JSONITEMCOERCIONS", 17))
return_value = _readJsonItemCoercions();
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
else if (MATCH("JSONTABLEPARENT", 15))
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
return_value = _readJsonTableParent();
Rename JsonIsPredicate.value_type, fix JSON backend/nodes/ infrastructure. I started out with the intention to rename value_type to item_type to avoid a collision with a typedef name that appears on some platforms. Along the way, I noticed that the adjacent field "format" was not being correctly handled by the backend/nodes/ infrastructure functions: copyfuncs.c erroneously treated it as a scalar, while equalfuncs, outfuncs, and readfuncs omitted handling it at all. This looks like it might be cosmetic at the moment because the field is always NULL after parse analysis; but that's likely a bug in itself, and the code's certainly not very future-proof. Let's fix it while we can still do so without forcing an initdb on beta testers. Further study found a few other inconsistencies in the backend/nodes/ infrastructure for the recently-added JSON node types, so fix those too. catversion bumped because of potential change in stored rules. Discussion: https://postgr.es/m/526703.1652385613@sss.pgh.pa.us
2022-05-13 17:40:01 +02:00
else if (MATCH("JSONTABLESIBLING", 16))
JSON_TABLE This feature allows jsonb data to be treated as a table and thus used in a FROM clause like other tabular data. Data can be selected from the jsonb using jsonpath expressions, and hoisted out of nested structures in the jsonb to form multiple rows, more or less like an outer join. Nikita Glukhov Reviewers have included (in no particular order) Andres Freund, Alexander Korotkov, Pavel Stehule, Andrew Alsup, Erik Rijkers, Zhihong Yu (whose name I previously misspelled), Himanshu Upadhyaya, Daniel Gustafsson, Justin Pryzby. Discussion: https://postgr.es/m/7e2cb85d-24cf-4abb-30a5-1a33715959bd@postgrespro.ru
2022-04-04 21:36:03 +02:00
return_value = _readJsonTableSibling();
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
else
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
{
elog(ERROR, "badly formatted node string \"%.32s\"...", token);
return_value = NULL; /* keep compiler quiet */
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
return (Node *) return_value;
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Restructure outfuncs and readfuncs to use macros in the same style as just done for copyfuncs/equalfuncs. Read functions in particular get a lot shorter than before, and it's much easier to compare an out function with the corresponding read function to make sure they agree. initdb forced due to small changes in nodestring format (regularizing a few cases that were formerly idiosyncratic).
2002-11-25 19:12:12 +01:00
/*
* readDatum
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
* Given a string representation of a constant, recreate the appropriate
* Datum. The string representation embeds length info, but not byValue,
* so we must be told that.
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
*/
Expose more out/readfuncs support functions. Previously bcac23d exposed a subset of support functions, namely the ones Kaigai found useful. In 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de I mentioned that there's some functions missing to use the facility in an external project. To avoid having to add functions piecemeal, add all the functions which are used to define READ_* and WRITE_* macros; users of the extensible node functionality are likely to need these. Additionally expose outDatum(), which doesn't have it's own WRITE_ macro, as it needs information from the embedding struct. Discussion: 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de
2016-04-08 23:26:36 +02:00
Datum
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
readDatum(bool typbyval)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
Size length,
i;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
int tokenLength;
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
Datum res;
char *s;
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
/*
* read the actual length of the value
*/
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
token = pg_strtok(&tokenLength);
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
length = atoui(token);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Add a little more error checking and reporting to readDatum().
2001-07-03 18:52:48 +02:00
token = pg_strtok(&tokenLength); /* read the '[' */
if (token == NULL || token[0] != '[')
Allow use of "z" flag in our printf calls, and use it where appropriate. Since C99, it's been standard for printf and friends to accept a "z" size modifier, meaning "whatever size size_t has". Up to now we've generally dealt with printing size_t values by explicitly casting them to unsigned long and using the "l" modifier; but this is really the wrong thing on platforms where pointers are wider than longs (such as Win64). So let's start using "z" instead. To ensure we can do that on all platforms, teach src/port/snprintf.c to understand "z", and add a configure test to force use of that implementation when the platform's version doesn't handle "z". Having done that, modify a bunch of places that were using the unsigned-long hack to use "z" instead. This patch doesn't pretend to have gotten everyplace that could benefit, but it catches many of them. I made an effort in particular to ensure that all uses of the same error message text were updated together, so as not to increase the number of translatable strings. It's possible that this change will result in format-string warnings from pre-C99 compilers. We might have to reconsider if there are any popular compilers that will warn about this; but let's start by seeing what the buildfarm thinks. Andres Freund, with a little additional work by me
2014-01-23 23:18:23 +01:00
elog(ERROR, "expected \"[\" to start datum, but got \"%s\"; length = %zu",
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
token ? token : "[NULL]", length);
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
if (typbyval)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
if (length > (Size) sizeof(Datum))
Allow use of "z" flag in our printf calls, and use it where appropriate. Since C99, it's been standard for printf and friends to accept a "z" size modifier, meaning "whatever size size_t has". Up to now we've generally dealt with printing size_t values by explicitly casting them to unsigned long and using the "l" modifier; but this is really the wrong thing on platforms where pointers are wider than longs (such as Win64). So let's start using "z" instead. To ensure we can do that on all platforms, teach src/port/snprintf.c to understand "z", and add a configure test to force use of that implementation when the platform's version doesn't handle "z". Having done that, modify a bunch of places that were using the unsigned-long hack to use "z" instead. This patch doesn't pretend to have gotten everyplace that could benefit, but it catches many of them. I made an effort in particular to ensure that all uses of the same error message text were updated together, so as not to increase the number of translatable strings. It's possible that this change will result in format-string warnings from pre-C99 compilers. We might have to reconsider if there are any popular compilers that will warn about this; but let's start by seeing what the buildfarm thinks. Andres Freund, with a little additional work by me
2014-01-23 23:18:23 +01:00
elog(ERROR, "byval datum but length = %zu", length);
Revise quoting conventions in outfuncs/readfuncs so that nodeRead doesn't choke on relation or attribute names containing spaces, quotes, or other special characters. This fixes a TODO item. It also forces initdb, since stored rule strings change.
2000-01-14 01:53:21 +01:00
res = (Datum) 0;
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
s = (char *) (&res);
Make outfuncs/readfuncs treat OIDs properly as unsigned values. Clean up inconsistent coding practices for handling Index values and booleans, too.
2001-01-08 01:31:43 +01:00
for (i = 0; i < (Size) sizeof(Datum); i++)
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
token = pg_strtok(&tokenLength);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
s[i] = (char) atoi(token);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
}
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
}
else if (length <= 0)
Revise quoting conventions in outfuncs/readfuncs so that nodeRead doesn't choke on relation or attribute names containing spaces, quotes, or other special characters. This fixes a TODO item. It also forces initdb, since stored rule strings change.
2000-01-14 01:53:21 +01:00
res = (Datum) NULL;
else
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
{
s = (char *) palloc(length);
for (i = 0; i < length; i++)
{
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
token = pg_strtok(&tokenLength);
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
s[i] = (char) atoi(token);
}
res = PointerGetDatum(s);
}
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Add a little more error checking and reporting to readDatum().
2001-07-03 18:52:48 +02:00
token = pg_strtok(&tokenLength); /* read the ']' */
Modify readfuncs so that recursive use of stringToNode will not crash and burn. Just for added luck, change reading of CONST nodes so that we do not need to consult pg_type rows while reading them; this means that no database access occurs during stringToNode. This requires changing the order in which const-node fields are written, which means an initdb is forced.
2001-01-07 02:08:48 +01:00
if (token == NULL || token[0] != ']')
Allow use of "z" flag in our printf calls, and use it where appropriate. Since C99, it's been standard for printf and friends to accept a "z" size modifier, meaning "whatever size size_t has". Up to now we've generally dealt with printing size_t values by explicitly casting them to unsigned long and using the "l" modifier; but this is really the wrong thing on platforms where pointers are wider than longs (such as Win64). So let's start using "z" instead. To ensure we can do that on all platforms, teach src/port/snprintf.c to understand "z", and add a configure test to force use of that implementation when the platform's version doesn't handle "z". Having done that, modify a bunch of places that were using the unsigned-long hack to use "z" instead. This patch doesn't pretend to have gotten everyplace that could benefit, but it catches many of them. I made an effort in particular to ensure that all uses of the same error message text were updated together, so as not to increase the number of translatable strings. It's possible that this change will result in format-string warnings from pre-C99 compilers. We might have to reconsider if there are any popular compilers that will warn about this; but let's start by seeing what the buildfarm thinks. Andres Freund, with a little additional work by me
2014-01-23 23:18:23 +01:00
elog(ERROR, "expected \"]\" to end datum, but got \"%s\"; length = %zu",
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
token ? token : "[NULL]", length);
Massive commit to run PGINDENT on all *.c and *.h files.
1997-09-07 07:04:48 +02:00
Postgres95 1.01 Distribution - Virgin Sources
1996-07-09 08:22:35 +02:00
return res;
}
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
/*
* readAttrNumberCols
*/
Expose more out/readfuncs support functions. Previously bcac23d exposed a subset of support functions, namely the ones Kaigai found useful. In 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de I mentioned that there's some functions missing to use the facility in an external project. To avoid having to add functions piecemeal, add all the functions which are used to define READ_* and WRITE_* macros; users of the extensible node functionality are likely to need these. Additionally expose outDatum(), which doesn't have it's own WRITE_ macro, as it needs information from the embedding struct. Discussion: 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de
2016-04-08 23:26:36 +02:00
AttrNumber *
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
readAttrNumberCols(int numCols)
{
int tokenLength,
i;
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token;
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
AttrNumber *attr_vals;
if (numCols <= 0)
return NULL;
attr_vals = (AttrNumber *) palloc(numCols * sizeof(AttrNumber));
for (i = 0; i < numCols; i++)
{
token = pg_strtok(&tokenLength);
attr_vals[i] = atoi(token);
}
return attr_vals;
}
/*
* readOidCols
*/
Expose more out/readfuncs support functions. Previously bcac23d exposed a subset of support functions, namely the ones Kaigai found useful. In 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de I mentioned that there's some functions missing to use the facility in an external project. To avoid having to add functions piecemeal, add all the functions which are used to define READ_* and WRITE_* macros; users of the extensible node functionality are likely to need these. Additionally expose outDatum(), which doesn't have it's own WRITE_ macro, as it needs information from the embedding struct. Discussion: 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de
2016-04-08 23:26:36 +02:00
Oid *
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
readOidCols(int numCols)
{
int tokenLength,
i;
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token;
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
Oid *oid_vals;
if (numCols <= 0)
return NULL;
oid_vals = (Oid *) palloc(numCols * sizeof(Oid));
for (i = 0; i < numCols; i++)
{
token = pg_strtok(&tokenLength);
oid_vals[i] = atooid(token);
}
return oid_vals;
}
/*
* readIntCols
*/
Expose more out/readfuncs support functions. Previously bcac23d exposed a subset of support functions, namely the ones Kaigai found useful. In 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de I mentioned that there's some functions missing to use the facility in an external project. To avoid having to add functions piecemeal, add all the functions which are used to define READ_* and WRITE_* macros; users of the extensible node functionality are likely to need these. Additionally expose outDatum(), which doesn't have it's own WRITE_ macro, as it needs information from the embedding struct. Discussion: 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de
2016-04-08 23:26:36 +02:00
int *
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
readIntCols(int numCols)
{
int tokenLength,
i;
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token;
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
int *int_vals;
if (numCols <= 0)
return NULL;
int_vals = (int *) palloc(numCols * sizeof(int));
for (i = 0; i < numCols; i++)
{
token = pg_strtok(&tokenLength);
int_vals[i] = atoi(token);
}
return int_vals;
}
/*
* readBoolCols
*/
Expose more out/readfuncs support functions. Previously bcac23d exposed a subset of support functions, namely the ones Kaigai found useful. In 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de I mentioned that there's some functions missing to use the facility in an external project. To avoid having to add functions piecemeal, add all the functions which are used to define READ_* and WRITE_* macros; users of the extensible node functionality are likely to need these. Additionally expose outDatum(), which doesn't have it's own WRITE_ macro, as it needs information from the embedding struct. Discussion: 20160304193704.elq773pyg5fyl3mi@alap3.anarazel.de
2016-04-08 23:26:36 +02:00
bool *
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
readBoolCols(int numCols)
{
int tokenLength,
i;
Add a debugging option to stress-test outfuncs.c and readfuncs.c. In the normal course of operation, query trees will be serialized only if they are stored as views or rules; and plan trees will be serialized only if they get passed to parallel-query workers. This leaves an awful lot of opportunity for bugs/oversights to not get detected, as indeed we've just been reminded of the hard way. To improve matters, this patch adds a new compile option WRITE_READ_PARSE_PLAN_TREES, which is modeled on the longstanding option COPY_PARSE_PLAN_TREES; but instead of passing all parse and plan trees through copyObject, it passes them through nodeToString + stringToNode. Enabling this option in a buildfarm animal or two will catch problems at least for cases that are exercised by the regression tests. A small problem with this idea is that readfuncs.c historically has discarded location fields, on the reasonable grounds that parse locations in a retrieved view are not relevant to the current query. But doing that in WRITE_READ_PARSE_PLAN_TREES breaks pg_stat_statements, and it could cause problems for future improvements that might try to report error locations at runtime. To fix that, provide a variant behavior in readfuncs.c that makes it restore location fields when told to. In passing, const-ify the string arguments of stringToNode and its subsidiary functions, just because it annoyed me that they weren't const already. Discussion: https://postgr.es/m/17114.1537138992@sss.pgh.pa.us
2018-09-18 23:11:54 +02:00
const char *token;
Add readfuncs.c support for plan nodes. For parallel query, we need to be able to pass a Plan to a worker, so that it knows what it's supposed to do. We could invent our own way of serializing plans for that purpose, but piggybacking on the existing node infrastructure seems like a much better idea. Initially, we'll probably only support a limited number of nodes within parallel workers, but this commit adds support for everything in plannodes.h except CustomScan, because doing it all at once seems easier than doing it piecemeal, and it makes testing this code easier, too. CustomScan is excluded because making that work requires a larger rework of that facility. Amit Kapila, reviewed and slightly revised by me.
2015-09-23 17:51:50 +02:00
bool *bool_vals;
if (numCols <= 0)
return NULL;
bool_vals = (bool *) palloc(numCols * sizeof(bool));
for (i = 0; i < numCols; i++)
{
token = pg_strtok(&tokenLength);
bool_vals[i] = strtobool(token);
}
return bool_vals;
}